FRC 95 The Grasshoppers 2019 Build Thread

[JamesCH95]

We were seeing a similar issue when we were telling a motor controller to move that did not exist, this seems like a change from last year as I do not remember this happening previously.

I’m not sure that I like this change too much if it acts this way if a motor control that was instantiated properly goes out during a match and similar calls get made. It was also not incredibly easy to diagnose as the errors, in theory, should not have been able to affect other motor controller’s output.

I think that some more testing would be in order to see if this is the case and if so include some checks to make sure that a call is never made to a motor controller that does not exist.

Thanks! @golf_cart_john It’s nice to know we’re not the only ones.

[apm4242]

N0NUG8w.jpg1200×1600

Do you stack your motor controllers? I can’t tell in your electronics layout photo. If so, have you seen any downsides to this - access to buttons, heat, etc? We’re always looking for more efficient layouts.

[euph2137]

Is there a reason why you are using 2x 3:1’s and not one 9:1?

[JamesCH95]

Do you stack your motor controllers? I can’t tell in your electronics layout photo. If so, have you seen any downsides to this - access to buttons, heat, etc? We’re always looking for more efficient layouts.

Each set is triple-stacked. We have encounter no issues so far, but haven’t worked it out much yet.

Is there a reason why you are using 2x 3:1’s and not one 9:1?

We didn’t have any 9:1s left!

[euph2137]

OK, that makes sense now.

[JamesCH95]

|HAL: CAN Receive has Timed Out|
||Error at frc.robot.Main.main(Main.java:27): HAL: CAN Receive has Timed Out|
||edu.wpi.first.hal.PDPJNI.getPDPTotalCurrent(Native Method)|
||edu.wpi.first.wpilibj.PowerDistributionPanel.getTotalCurrent(PowerDistributionPanel.java:79)|
||edu.wpi.first.wpilibj.smartdashboard.SendableBuilderImpl.lambda$addDoubleProperty$3(SendableBuilderImpl.java:242)|
||edu.wpi.first.wpilibj.smartdashboard.SendableBuilderImpl.updateTable(SendableBuilderImpl.java:95)|
||edu.wpi.first.wpilibj.livewindow.LiveWindow.updateValues(LiveWindow.java:290)|
||edu.wpi.first.wpilibj.IterativeRobotBase.loopFunc(IterativeRobotBase.java:263)|
||edu.wpi.first.wpilibj.TimedRobot.startCompetition(TimedRobot.java:81)|
||edu.wpi.first.wpilibj.RobotBase.startRobot(RobotBase.java:263)|
||frc.robot.Main.main(Main.java:27)|
||Watchdog not fed within 0.020000s|
||Loop time of 0.02s overrun|
||Warning at edu.wpi.first.wpilibj.IterativeRobotBase.printLoopOverrunMessage(IterativeRobotBase.java:273): Loop time of 0.02s overrun|
|||
||Changes to the code|
||Removed all subsystems (and their associated commands) other than drive system and vision co-processor|
||Removed shifting functionality|
||Commented out LiveWindow|
||Deleted the save.xml file|
|||
||Things checked|
||Can see all 6 talons for the drive base, the PCM, and the PDP; they also all have reasonable firmware and distinct device IDs|
||Re-flashed firmware of PDP|
||Removed drivebase and basically everything else (or start from scratch with simple code that has Athena and PDP) DID BOTH AND NEITHER HAD THIS PDP ERROR BUT NOT SURE IF THEY were just so simple they didn't use the same part of the hardware/firmware|
|||
||================================================|
|||
||Connected a different PDP to CAN network and shorted CAN network so it was only RoboRio and that secondary PDP; secondary PDP powered by its own battery, but primary battery still powering everything else, but they're not part of CAN network (confirmed w/ Phoenix)|
||Used a recent-ish commit (32361fccc094c7a9c87b0b6b2bb2e1dc5a213aee? not 100% sure) and still had that PDP error (in addition to a bunch of others b/c other items not on CAN)|
||Used commit from just after JDW updated codebase for VSCode early this build season (a89e8efdcf726c9181dcd327e06a6f5bf00351a1) and still had that PDP error (in addition to the ones b/c of missing Talons); this commit was from before we switched from IterativeRobot to TimedRobot|
||Used debug_galore first commit (where removed basically everything), and DIDN'T see this issue, but nothing else was really happening so hard to say|
||Used second commit of debug_galore branch where added back in OI and drivebase where drivebase pointing at a single talon that was connected to the secondary PDP, and DID see the PDP issue|

Still having significant code-related or CAN-related issues. Any thoughts on the above error and notes is appreciated.

[AriMB]

Sounds like the same problem they’re having here. There’s no solution yet, but you might want to follow that thread.

[BenjaminA]

We had this issue recently and went though similar debugging steps. We found that we we initializing CAN-based devices that don’t exist. I.e. when our practice PDP’s CAN bus module was fried, we got this error because the RoboRIO couldn’t find a device with the specified ID. I know it sounds a bit silly, but make sure you don’t initialize any CAN based device that does not exist or has a different CAN bus ID. Please let us know what fixes it for you!

[RobotNerd1]

We are also receiving this issue with the watchdog not being fed, we have checked multiple times that we are not initializing any extra CAN devices and that each of our devices has a unique ID. I’ll be interested to see what the root cause of this is.

[rich.varone]

We’ve also been having weird issues that look like CAN or network problems as well as we’ve seen watchdog errors sometimes, we don’t know if it’s related or not but we’ve noticed that for some reason our packet loss has gone way up after flashing our robots to the 2019 RIO and and radio images. We haven’t had enough time yet to try going back to the 2018 images to see if it remedies our connection issues. Any chance your driver station reports packet loss?

[JamesCH95]

We got the CAN/control issue sorted. We were able to intentionally reproduce it by commanding a talon that did not exist after carefully paring down our code to eliminate any unknowns, even commands that ‘shouldn’t’ have been called, but referenced absent CAN devices.

We were also able to reproduce the issue by simply ‘sleeping’ or freezing the main thread for 500ms (a rectal-plucked value), so if the main robot loop is taking too long to run (over 20ms maybe?) this error/bug/stupid/error can be triggered.

You can see the code in the video below, and the errors/symptoms. The test code is in our GitHub repo which is linked elsewhere in this thread. The following commit is our test code to illustrate the problem: 9b2e009266a4d6727e7c0fbca31ed2f2fd898e37

We believe that something in the newer firmware or programming environment triggers these issues when an absent CAN device is referenced, even if it’s not necessarily actively being called. We’re a little fuzzy on that details

Any chance your driver station reports packet loss?

Yes it did. Especially when multiple missing devices were being reported.

[JamesCH95]

SIGNIFICANT DISCOVERY: if you try to call/talk to/access the PDP from more than one line of code this triggers the time-out issue. It is possible that calling other devices from multiple lines can also cause things to spaz out.

[golf_cart_john]

Our fix can be found in this commit. It involves making a singleton class, which is designed such that only one instance will ever exist. This results in only ever instantiating one PowerDistributionPanel object.

[JamesCH95]

Drives and auto shifts nicely. Phew.

Weighs 61.8 lbs.

[JesseK]

SIGNIFICANT DISCOVERY: if you try to call/talk to/access the PDP from more than one line of code this triggers the time-out issue. It is possible that calling other devices from multiple lines can also cause things to spaz out.

Interesting. We also use an (effective) singleton to manage PDP calls as a result of different testing. Last year I wrote my high frequency logging framework and did benchmarks of a few sensors like NavX and a Rio-wired 63R. The PDP took 4ms to read the current of all 16 ports. Oddly enough, with multi-threading and concurrent calls it took 7 ms to read all 16 ports. I haven’t tested again with 2019 code though.

[Thad_House]

This was a known issue with the old PDP class. It was fixed over the summer to only be about 0.45 ms to read all PDP data. The cause of this was each can read takes about 0.15ms, and they were not being cached, so each value read was taking that long. Now the status packets are cached, so only 3 can calls are needed per iteration.

This is also the reason creating multiple new instances doesn’t work in 2019. Although it should be throwing an exception on construction of more then the first one, so I’ll check that out to make sure it happens.

[RobotNerd1]

So, @Thad_House if what you are saying is correct then is that theoretically not the cause of the issue?

[Thad_House]

That is the cause of the issue. I just checked the source code, and it is allowing multiple PDP instances when it shouldn’t be. We will fix it for the next wpilib release. Only once PDP instance is supported, like most other physical devices.

[JamesCH95]

Ended Saturday with the practice bot starting to get assembled:

And the competition robot having some real functionality:

Auto shifting and the hatch mechanism work nicely. Next items on the list are the hatch ground intake, elevator drive, and cargo mechanism.

Here is a video of our first real practice driving.

Precision driving this year will be… challenging… much more so than many other games that I recall. I hope our vision-aided driving works out…

[Cog]

Wow that’s a really eye opening look at cycles for this year. The lineup and precision required at both ends of the process seem finicky, and both of those aspects are for the most part out of the drivers’ convenient line of sight.