Low Latency CAN communication

[pscheidler]

Hello,

We are using the USB-8502 (784662-01) for testing some equipment, and while the communication basically works, we would like to speed it up. Ultimately we want to send a message, wait 10 ms for a response, and if we don't get the response, re-send the message within about 1 ms. If we do get a response, we move on to the next part of the test. Communication is all 1 MHz CAN, 2 wire. Running 4 devices in parallel would be ideal, but not required.

We're writing the test in Python using the NIXNet library. We can get the response time down to about 40 ms, but we have had get dig pretty deep into the NIXNet library to get that fast.

Is 1 ms just too fast to resend a message? Any other way we can automate this?

[Hooovahh]

CAN itself is meant to be non-deterministic.  Reliable yes, deterministic no.  A CAN bus can have data written to it by any device on the bus at any time.  Imagine if all 4 devices write to the bus at the very same time.  What happens?  Well the ID in the CAN frame is the priority level, and 3 of the messages will be lost because the highest priority message will dominate.  So now what happens to the other 3 messages are they lost forever?  Nope, the hardware will automatically try to send their messages again.  3 will probably try to go at once, the highest priority will dominate, and 2 will be lost.  So what happens if we do this again but instead all 4 try to write things all the time?  Well there will be lots of lost messages and messages will continue to be sent and retried until they go out, that's just how CAN works.

So what does this do to our timing?  Like I said non-deterministic. 

The amount of jitter is likely on a very small amount and sub millisecond unless the bus is loaded very heavily like 70% or more.  I just mentioned this non-deterministic part of CAN to let you know sometimes it is outside your control.

How you use the XNet API will determine how fast it can respond as well.  If you are using an ISO 15765 protocol, then you know the request is on an ID, and the response is on another ID.  So you can create a Frame In Queued session type, then use the read, reading in a single frame with a long timeout like 1s.  This read will return immediately once that frame is seen once so it should be as fast as the hardware can return.  If instead you are in a loop reading frames over and over looking for the one ID you want, you will be using up lots more CPU resources, and will be slower overall.  I know you mentioned Python but I'm not a one dimensional programming kind of guy so attached is a VI, and an image of what I'm trying to describe.

Another issue you will likely see is the ECU doesn't respond immediately.  Put another CAN node on the bus and monitor the traffic and you'll see the time it takes from getting the request to sending the response.  Depending on what the request is it may take a while.  I know a command to erase memory for instance will take several seconds with a response about once a second saying "I'm still working and need more time".

[pscheidler]

Thanks a ton for your response! I understand what you're saying about the multi-device latency, but I should have said that this test is just the USB-8502 talking with the device under test. We work directly on the in and out stream of the 8502 because it is just easier for us, and we know we are interested in all messaged.

Does going through the CAN database system speed things up? We use a very specific device, but it has just a few message types, so we would have need to make our own database, but I don't imagine it would be that hard.

I believe we had done a test where we would send out a batch of messages with known timing, and the timing there was good, but waiting for the incoming message and then sending one or not based on the message out is what kills our timing.

[Hooovahh]

---

@pscheidler wrote:

We work directly on the in and out stream of the 8502 because it is just easier for us, and we know we are interested in all messaged.

---

You can have lots of open sessions at the same time.  You can have 1 In Stream, 1 Out Stream like you do now, and then 1 In Queued for all unique IDs you care to look for.  XNet is quite flexible and allows for doing things in alternative ways.  That being said it really shouldn't take that long to just read all frames, and go through a for loop looking for your ID.  Still depending on how your code is written there might be arbitrary waits between reads and a Queued Read would help wait until that comes in.

---

@pscheidler wrote:

Does going through the CAN database system speed things up? We use a very specific device, but it has just a few message types, so we would have need to make our own database, but I don't imagine it would be that hard.

---

Going through the database itself doesn't make it go faster it just helps organize it.  You can make your own database and save it to use later, or you can create one dynamically at runtime.

[pscheidler]

I need to resurrect this because we've tried some pretty big changes and we're still seeing the same issue.

Our system is just the NI USB-8502 communicating with one device, which just responds to our messages. We open the port with streaming in and streaming out sessions, and I'm just sending 2 can packets out, each with a payload of 2. The code is super simple, just:

myFrameOut[0].Identifier = 2;
Status = nxWriteFrame(nxSessionOut, myFrameOut, (u32)sizeof(nxFrameCAN_t), 0); // Timeout=0, fail if we cannot queue the message
myFrameOut[0].Identifier = 1;
Status = nxWriteFrame(nxSessionOut, myFrameOut, (u32)sizeof(nxFrameCAN_t), 0); // Timeout=0, fail if we cannot queue the message

I change the ID so I can be sure what message is going out, even though there are no other messages on the bus at this time.

I have a second NI USB-8502 as a bus monitor, and that shows that the time between these messages is 15 ms.  Why so long? Is there some other setting setting a max output rate? Our baud rate is 1 Mb, so the total 24 byte message should take under 200 us to go out.

Anything I can do to speed this up?

[ZY_Ong]

The 15ms latency you are seeing is mostly contributed by the communication over USB. USB is one of the buses that are high in latency with poor determinism. Perhaps you want to consider PCI or PXI buses with much lower latency.

Instrument Bus Performance

Another suggestion for your use case is to use cyclic frames:

1. Using NI-XNET Database Editor to create cyclic frame with transmit time of 0.01s (or even smaller but be careful it will increase the bus load!). You cannot send cyclic frame using memory database. 

2. Create an output session to send this cyclic frame. Cyclic frames are added to the hardware queue automatically by the XNET firmware at the configured interval. 

3. Once the desired response are received, call nxStop with the Scope set to Session Only to stop the cyclic frame. You can then call nxWriteFrame to send another cyclic frame.

[pscheidler]

Thanks a lot for your reply!

What we really want to do is send a message, wait for a reply within 10 ms, and quickly re-send the message if we don't get the reply. Right now, the re-send will come out 25-30 ms after the original message, because of the 15 ms latency to send a message. But, because we only re-send if we did not get a reply, it seems like cyclic frames is not appropriate. If we cannot send within 15 ms, I worry that the latency to read a message is high as well, and testing that would be difficult. We could run in to a situation where we receive a reply 9 ms after the first message, but the software would not see the message until the transmit had gone out.

We are using USB-3 ports, no external hubs, and the CAN device is the only one using significant bandwidth. I am very surprised that the 15 ms delay is caused by 100 us latency. 

[ZY_Ong]

I found out the latency is actually caused by starting the session when you call XNET Write for the first time. Subsequent XNET Write takes less than 1ms.

You can call XNET Start.vi explicitly to reduce the time taken for the first XNET Write VI call.

[pscheidler]

Thank you, that sounds interesting but I am not 100% sure I understand it.

It sounds like you're saying I can use nxWriteFrame to send the message the first time, and then I can send it again more quickly with nxStart? That would be a great solution to my problem.

Do I have that right?

[ZY_Ong]

---

@pscheidler  已写:

Thank you, that sounds interesting but I am not 100% sure I understand it.

It sounds like you're saying I can use nxWriteFrame to send the message the first time, and then I can send it again more quickly with nxStart? That would be a great solution to my problem.

Do I have that right?

---

If you didn't call nxStart explicitly before nxWrite, the first nxWrite also includes the functionality to switch the session to run state, thus the longer execution time.

You should call nxStart explicitly before first nxWrite, to get an actual execution time of nxWrite.