SPARK MAX CAN Latency Testing

Team 401 conducted some testing of the CAN latency between the roboRIO and the SPARK MAX several weeks ago, and we just finished putting together our report:

TL;DR: There is some significant latency in CAN stack of the SPARK MAX when compared to the Talon SRX (upwards of 70 ms vs. < 5 ms of the Talon).

We’re hoping REV will be able to fix this issue!

7 Likes

This is really interesting. I was actually talking to @Thad_House about general CAN latency improvements for 2020 and was curious as to what that means for this test.

Quoting Thad from Discord:

Can calls now take 7us rather then 150us. Which is what they were before. Also the spikes into the ms range don’t happen anymore either.

Thanks for sharing!

Yup. Pretty much any CAN performance testing can be thrown completely out the window for 2020. The entire CAN subsystem has changed, which is going to change how the vendors write their CAN code as well. So things done in the past to not lag out user code are not necessary anymore, which means performance rewrites at pretty much all levels.

7 Likes

Will this be elaborated on closer to Kickoff, or in the WPILib docs?

There’s not really much to elaborate on, but there will be a node.

Basically, all CAN messages sent and received on the rio went through the netcomm daemon in the past. This meant any send or receive went over an IPC pipe. For 2020, CAN has been moved entirely into the robot process, which means the robot process now has direct access to the CAN subsystem. The IPC is what was causing all the direct call latency, and all of the vendors had chosen different ways to make these latencies less visible to users. For 2020 however, none of these workarounds are necessary, as direct calls are not fast enough to not affect user code.

5 Likes

The Pink line is a DIO getting set, and the Yellow/Blue lines are CAN traffic. 18us between the DIO getting set and the CAN message starting to transmit. Working on testing the 2019 image now.

3 Likes

These are reads off of the 2019 image. Best case is about 75us, worst case is about 200us. Also, something interesting to note is how much longer the DIO line is low. That whole time, your robot code is run running, and is instead just waiting for the CAN IPC call to return. That is the reason many of the vendors were handling workarounds, because a single CAN call would take up almost 10% of your robot loop time.

9 Likes

Music to my ears! It didn’t prevent us from doing good things last year for sure, but we were totally seeing weirdly high loop times when making lots of CAN bus calls to our SRX.

Thanks! I love this level of investigation.

3 Likes

These changes look very good!

However, it looks like even the latency in 2019 WPILib was much lower than the numbers I observed measuring total latency from DIO set to SPARK output, indicating most of this latency is a problem in REV’s firmware (70+ ms!)

1 Like

Thanks for the detailed analysis. We’ve been looking at this issue closely and confirmed that it is on the API side, and not at all related to the firmware. Now that the latency for the CAN calls is so much quicker for 2020 as verified by @Thad_House we will be removing our own workaround for the issue.

3 Likes