2015 RoboRIO/Control System Feedback

Spawning a separate thread for Positive/Negative control system feedback, so it doesn’t get lost in the mass of replies to 2015 Lessons Learned. I figure it’ll be easier for our NI denizens to keep track of things in here.

Our experience was overall positive. We were already using CAN, and the new 2-wire native CAN bus and web config made things so much simpler that setting up our 9 Talon SRXs was laughably easy. I nearly giggled updating the firmware of all the devices via the web config.

My only bad experience was with the analog inputs. We had string pots for feedback on our elevators, and we ran into a baffling issue trying to add any other sensors to the robot. We wanted an IR prox switch, IR distance sensor, or ultrasonic to tell when we had a tote to stack, but adding any one of those screwed with the string pots. I’d plug in the IR distance sensor and the elevator would start twitching as the signal for each pot started bouncing around. I originally figured it was because our 5k pots were outside the 3k impedance NI recommends, but switching to 1k pots at Champs didn’t help. I finally gave up worrying about that and plugged in our navX MXP board just to see how bad that would make things. Except plugging in the navX magically made everything stable, even when I plugged in the previously problematic IR distance sensor. I’m still slightly lost as to what was going on, but I’m hooking up an o-scope to the bot when the crate shows up to see if I can at least isolate the problem to the 5V, GND, or signal lines.

Reposting my comments from other threads:

The new control system rocks. I cannot stress this enough. This was the first time in my 8 years of FRC that my team has fielded a robot without a single major controls problem during competition. The new motor controllers are absolutely fantastic; we didn’t have a single problem with our Talon SRX’s, and the integrated signal wires are a godsend (even using them in PWM mode, being able to easily daisy-chain motor controllers on the same side of the drive train saved us a lot of wiring effort and mess). Just about everything is a huge upgrade from what it was before.

I’ll add one more minor quibble in that the signal cables, when plugged into the RoboRio, seemed mechanically less-secure than they ought to have been, though this is fixable by simply dabbing some hot glue on the connector once it’s plugged in (thanks to team 1678 for showing us this trick at championships last year).

The mDNS SmartDashboard issue cost us a match thursday at champs. I hope they can figure that and maybe a few other mDNS issues out for next year. Having to restart Driver Stations when running back to back matches is sometimes very odd.

Also, what happened to allowing F1 to enable the robot? It doesnt work anymore on the new DS.

I saw lots of teams with lots of control system issues this year. None of them was really the fault of the control system itself. When the instructions are followed by someone who actually pays attention to them, and who is not sloppy with the wiring, things just work. The roboRIO’s web console is great, and the rework of the DS-to-robot communication protocol seemed to make a positive difference in connectivity. I just wish Windows weren’t so unpredictable in how it deals with switching between networks with and without DNS active.

I do have one other mildly negative observation: the Weidmuller connectors seem to be happiest with 20 gauge wire, but teams seem to want to use 18 gauge or thicker. This makes inserting the wires without risking short circuits more difficult than it ought to be.

More than one team accidentally enabled their robot by pressing the F1 key while in another application on the Driver Station, so using F1 to enable was changed to the three-key “]” combination.

Oh thats good to know. I didn’t see that get published anywhere. For demos in the past where WiFi connection was awful, we would sometimes put the computer in the robot, and use a wireless keyboard or controller to enable and disable the robot. Good to know there is still a button combination to allow this.

The smartDashboard introduced an issue to our team. On the second final match on Galileo, the smart dashboard did not populate with the sendable Chooser we use to select our autonomous program. If we hadn’t had ***some of the absolute best volunteers and tech guys I’ve seen in my entire time in FIRST ***helping me figure it out, our robot would have run an incorrect default autonomous and flipped over.

The fact that they took the digital input away from the driver station and put selecting something as undeniably important as selecting autonomous and put it something as shaky and unreliable as the SmartDashboard was, in my opinion, the worst move FIRST made this entire season.

Either bring the digital input on the driver station back or fix the SmartDashboard. And if the smartDashboard is fixed then make it easier to implement. I consider myself an experienced programmer, and if I was not able to figure it out to the extent that I was still having issues, then that means other teams were not afforded the luxury of selecting an autonomous before a match.

That is an issue.

I apologize if this sounds angry and emotional, and definitely biased. It is. As I said earlier I had to work through this issue in the finals on our division so it is quite near and dear to my heart.

P.S. Also shout out to the guy who was finally able to fix it! I’m afraid I didn’t catch your name during it, but not only were you polite during that stressful period, but you were kind to me, told me what was wrong and how to fix it, and then took your time without me even asking to come back to the pits and fix it on our other computer. I cannot thank you enough.

The spec says they’re good to 16 gauge if I remember correctly. If they’re not, the spec should reflect this. Also, there are a couple places where 18 gauge is the minimum, namely Vin for the PCM and compressor out on the PCM. The problem is remedied slightly if you use wire with thinner insulation.

And now, to some of my personal complaints.

There was a windows 8 setting this year which didn’t play nice with the driver station. It disabled controller outputs sometimes.

The Axis camera thing where it would work only on the field, not at home and not in the pits unless you’re a networking whizz with time to figure it out.

Also, those fuses which were really hard to put in.

And the fact that(at least at the beginning of the season, I’ve been too busy recently to check if that has been fixed), you couldn’t use the new driver station to control cRio robots, and the cRio and roboRIO driverstations were recommended not to coexist.

Yeah, there are apparently some recalcitrant bugs in the Java Dashboard, especially around the Sendable Chooser. I have confidence that they will get fixed.

But even before they do, digital inputs are still available on the Driver Station. Just use the Launchpad board that was in the Kit of Parts this year instead of the Cypress board that was in the Kit of Parts previously.

Or use joystick/gamepad buttons to select the desired routine. Or use a physical control on the robot itself. Or any of about a half dozen other things that I can think of quickly.

This has never happened to our team (to my knowledge). Moving it to the three button press seemed necessary, but I wasn’t opposed when it was brought up in Beta. However, I witnessed a much more likely scenario at the shop. Because the old enable (F1) wasn’t there, the drivers just moused over the enable button on the screen to enable and then used the enter key to disable. This meant the mouse was left over the enable key. During driver practice, the students had disabled the robot and were resetting the field, throwing pool noodles back to the other side of the drivers station when, you guessed it, a noodle landed on the mouse button, enabling the robot. Luckily it was just into teleop (not auto), so the robot didn’t do anything.

Long story short, don’t leave the mouse over the enable button or you could enable from across the room.

It seems to me an accidental mouse click is much more likely than an accidental F1.

Overall, the new system was a huge improvement.

**Things I didn’t like:
-MATCHES ARE NEVER THE RIGHT LENGTH OR EVEN THE SAME LENGTH. Seriously, most teleop periods are three seconds too long, but every now and then, we get a 2:15 teleop.

-Timertasks don’t work properly if they are started before connecting to the driver station, but work if the reset button is pushed on the roboRIO. Troubleshooting this behavior is difficult, and it’s never comforting when resetting or redeploying code puts the controller in a different state than a cold boot.

-Java timertask timing is worse than in previous years.

-It takes forever for robots to connect to the field.
-It takes forever for robots to connect to the driver station laptop after a match.
-Many of the workarounds and best practices to make these go faster (changing team number, restarting mDNS service, not having other stuff open…) aren’t published anywhere.

-The log viewer was never released for the power board.

-Default joystick values used when a joystick is disconnected don’t always make sense.

Good things
-WPIlib remained mostly the same
-Getting rid of CommandBase and putting the instances of Subsystems in Robot is a great change
-Toggle when pressed methods on Buttons is really useful.
-Loading code is much faster in Java, which is really nice for debugging
-The debugger works over an ethernet connection, which is awesome.

-The smaller size and weight is a huge benefit to teams.

-Current monitoring is an awesome feature to protect mechanisms from damaging themselves.

See #2. http://wpilib.screenstepslive.com/s/4485/m/24192/l/144976-frc-driver-station-software#OperationTab

I posted before on my feelings about the success of the control system. We need a physically rough game to make a final statement of success (hint hint GDC).

We used the current monitoring as a substitute for limit switches in one scenario.

Some areas for improvement:

. Transition between tethered in the pit and connected to the field was a continual issue.

. Certain laptops had trouble connecting when operating on battery. Theory is that certain network chipsets have some deep power saving mode that interferes with connectivity.

. physical retention for control cables at the roboRIO – maybe someone should design a 3d printed part that sticks to the top/side of the unit, with retention like the Jaguar?

. Time between cold boot of robot and field connectivity was a lot longer this year than last. We actively avoided the power cycle this year, because it took so long. Maybe that was good, though, in the end, because we identified some faster ways of fixing things.

Regarding the complaint about match length, we did a fair amount of checking match length at one regional and didn’t run into issues with running time. Jared, was there an event you noticed the issue at?

I’ve noticed this for several years now. You need to go into the properties of the WiFi adapter on the laptop and disable all power saving options. Otherwise when the battery gets below a certain level the Wifi adapter will actually disappear from the operating system until the power cord is plugged in again.

Team 111 Wildstang did this season: http://wildstang.org/blog/?p=225

I completely agree with both of these quotes.

What I believe would make a huge improvement to the quirkiness of mDNS is to put a “Restart mDNS” button on the DS. It would give all users a really quick way to connect to the field and RR. Restarting the DS, changing the team number and changing it back, disabling and re-enabling the NIC all work, but they are all more difficult processes than what is needed.

The developers got a lot of feedback about the DS connection issues at the field. Based on historic response to feedback, it will likely be a focus for them in the off season.

Agreed. I saw a bunch of this last year as CSA, less this year as FTAA but I suspect a lot of it was dealt with by our crack CSA crews. I tell teams that, if you can’t give the wires a good tug, they’re not in there well enough.

From what I heard from folks at FIRST, NI has an update ready to go that fixes the need to restart the DS (or toggle the team number) between back to back matches.

It wasn’t released during the season due to the complexity involved with trying to ensure that every team got a mid-competition season update, but should be available sometime during the offseason.

Overall, the new control system has been great. I love the web config, and I have dreamt of being able to use more modern versions of Java for years.

The most annoying problems I ran into were also the most common (mDNS being slow on the field and Axis camera configuration), so I have faith that they will improve for 2016. We’ve already seen some improvements, so hopefully another few months will make it even better.

As Alan mentioned, the Weidmuller connectors are easy for teams to wire incorrectly. Too many teams have copper splayed out around the connection. Hopefully this will get better as teams gain more experience with the connectors (and with their robot shorting out on the field).

We ran into the first issue at CVR…the problem is that the scheduleAtFixedRate() method schedules execution relative to the timestamp of the first execution (and the RoboRIO initially assumes it is turned on at the beginning of the epoch), but when the robot connects to the driver station, the local clock is synched to the driver station time. Your code freaks out because it thinks it has ~45 years worth of work to do in order to catch up, and as a result your TimerTask hogs the entire CPU. TL;DR, never use scheduleAtFixedRate(). Thanks to Joe Hershberger for helping us to figure this out.

As for TimerTask timing being bad otherwise, this is because of some incomplete JNI between the HAL and WPIlibJ. Whereas LabView and C++ use the onboard hardware timer to generate interrupts for calling recurring tasks, the current implementation of TimerTask uses Thread.sleep(), which is quite unreliable. We would schedule 5 ms periods and get around 5 ms most of the time, but we would see several 10+ ms periods per second, and occasionally even 200+ ms periods.

We hacked our WPIlib JAR with some additional JNI to get to the FPGA timer and were much, much happier with our timing jitter. I think Tom submitted a patch to WPIlib, so hopefully this will be fixed for next year. It took us a long, long time to figure it all out…

I generally loved the new control systems and the Talon SRX motor controllers this year. Just a few requests:

1 - Please reconfigure the kernel to include POSIX message queues and semaphores. Embedded Linux is incomplete w/o these primitives.

2 - Please make it easier to use static priorities in threads. Many teams roll their own software solutions and we can’t do the same things we did with VxWorks.

3 - Please use latching connectors on the roboRio. It is nearly a crime that we use the old PWM connectors in a high vibration environment.

4 - Please do not use Silverlight. It requires the client be Windoze and install a bunch of crap.


Requiring Silverlight to manage a Linux device bugged me.

If we left the robot on for a long time, maybe 15-30 minutes, the robot would freak out. Planning to debug that later.

This was actually an issue at Worlds, since Chrome dropped NPAPI support the week before. So we spent about 10 minutes trying to connect to the webdash, until I remembered, and we switched to IE. I would prefer to use chrome, which means they should switch to something chrome supports, and not silverlight.