Feedback Thread: Robot Control System

I would like to see a change to the FPGA that will allow teams to read PWM outputted by a sensor.

Also, this thread has gone WAY off-topic. This is a control sytem feedback thread, NOT KoP feeback thread! problems with the PDB, for instance, do not belong here.

I’m just going to echo some of the comments already made about the radio. The connection is not nearly robust enough for the competition, if I need to duck tape a wire into place to prevent communication issues, it’s not designed right.
I’d also like to agree with the reset button issues, and just generally the suitability of all of the parts and pieces for shock and the dual ports for tether, and radio would be very nice.

The comment about making one board to attach on top of the cRio is awesome, that would make everything so much easier, I would love it.

Though I do like the sidecar for being physically seperate from the cRio.

On the last 2 robots my team (399) made, the cRio has been in a some-what hard to reach with cords area.
But, the sidecar could be located in a better area because it was so small, so that cords could easily be added / removed and that made debugging much easier.

cRIO – excellent. The only trouble I’ve had is accidentally bending a pin of the compact DB15 in one of the module slots. This was probably due to sloppy and repeated insertion of a module.

Robot Radio – Hasn’t shown much reliability on-field, and I have no scientific method of troubleshooting.

Camera – It’s a pretty good camera. A shame we can’t use the FPGA to process images at the 30fps 640x480 that the camera is capable of. A method of increasing the view angle would be quite useful. Insulating the case from the frame presents a problem if you use the provided camera gimble. Gimble could use redesign to aid in insulation and robustness.

Solenoid breakout – I’m not sure why this was created. I would prefer screw terminals to those PWM-style connectors. (I believe the screw terminals are a removable phoenix connector, which would make it easier to move the cRIO from 'bot to 'bot)
http://sine.ni.com/images/products/us/040729_crio9472_m.jpg](http://sine.ni.com/nips/cds/view/p/lang/en/nid/208822)

Analog breakout – I appreciate that the noisy regulator was replaced with a linear regulator this year. However, I find it limiting that the only power supply is 5v. What about 10v? -10v? 3.3v? It’s silly to have a 12 bit AD converter, and only use 10 bits of it. Plus, those cables look just like the 3-pin cables we use for most of our other signals. I’ve attached an example pin-out.
In addition to this, I’ve always found the height of the board and that retaining tab on the plastic shield to get in the way.

Digital sidecar – Everything is packed in way too tightly, and I don’t use most of it anyways. I hate having to pull connectors apart with the cable, especially with PWM-style cables, because it tends to pull the connector apart FROM the cable. I would really like the option to use the digital module in slot 6 as 32 DIO, straight-through (even without the resistor pull-ups). This would allow use of sensors that have send and receive on the same line, and it would also allow the use of MORE sensors. (Someone could line their 'bot with limit switches, or have 12 optical encoders, or use the parallax SONAR). Also, it needs much better color-coding. Every year I have to go through and paint it up with nail polish so we don’t put things in backwards. I like the lights on the relay outputs, and the space in between the PWM connectors for the PWM signal. In general, it’s hard to place anywhere, because wires go to it from all directions, and it’s hard to neaten those up.

Power Distribution board – Seems to discourage proper protection. The main breaker should fit in neatly where power is supplied. The camera power supply should have a crowbar at 500mA, not 3000mA. (The camera datasheet specifies 5.0–5.5VDC with a max power draw of 2.5W) The cover of the PD board should not prevent fuses from being used for supplying power to the analog breakout (2A), the solenoid module (10A), and the Digital Sidecar (5A with many servos, 1A without). The power supplies for the camera, the cRIO, and the Robot Radio also throw a lot of noise on the power line. The camera and the Robot Radio should have physical switches to disconnect power from their DC-DC supplies. There should be a large capacitor to buffer noise, because the noise on the main power supply affects the reliability of analog sensors.

Wago Terminals – A little tricky at first, but pretty good. Requires two wago tools if you’re using zip cord. I would prefer the wago terminals that have plastic levers on them instead of requiring a wago tool.
http://www.wago.us/images/PCB_2716_87x87.jpg](http://www.wago.us/products/27816.htm)

Wago Connectors – Need to be color-coded, so wires are not inserted backwards. Split apart when non-official wago tools are used (because most screwdrivers are too wide).

Spike Relay – Push-ons are susceptible to failure. May break off from the board after repeated use. Very expensive item for just an opto-isolator and a three-position double-pole, double-throw relay. No color coding or keying. I would like a CAN-enabled relay to use instead.

Victor 884 – Small package. Nonlinear control. Unprotected fan hurts if you don’t expect it. No color coding or keying.

Jaguar – Lots of features available with CAN. Linear control. Forward and reverse limit switches. No color coding or keying. No reverse-voltage protection. Fork terminals are prone to failure and reversal. A better solution would be to use a pluggable connector on both sides of the Jaguar.

analog bumper.png


analog bumper.png

I would totally agree with this point. Our team (997) has a recurring electrical issue during the seeding rounds at Atlanta that would manifest as the robot completely dying for upwards of 30 seconds after being/receiving a bump. This was traced to a bad ground connection on the cRIO! There has to be a more secure method to connect wires to this critical piece of hardware.

I have heard the complaint regarding having to disconnect the cRIO from the radio multiple times on this and other threads. What our team did to solve this issue (it was WAY too hard to get to the connector on the cRIO) was to unplug the network connection from the radio end and then plug it into a simple (cheap 4 port variety) network switch connected to the classmate, etc. It made things easy in the pits to convert from a competition (radio) configuration to a pit/testing (tether) configuration.

However, I will agree that the time for the new wireless switch to boot is very long and the power connection on the radio is very prone on problems as well.

Enjoy!

Floyd Moore
Mentor Team 997

A few new comments after IRI:

Stop Button Override wait: This is annoying. We broke our USB hub, but since we only have two USB devices (gamepad and Cypress board), we just hooked everything right up to the Classmate and unplugged the stop button. We never use it anyway since it kills the code on the robot, and the space bar works just as well. BUT, now we have to wait 20 seconds in addition to the FMS lock time to tether the robot after a match (this is especially time critical during eliminations). The old IFI system could be run without a competition dongle with no problem (if it needed to be disabled, you could just unplug the tether cable or the power cable when on radio)

Wait for code when downloading code: Sometimes I need to download code FAST. Something like an autonomous change between elim. matches. I already know I have to wait for the robot to boot, and wait for the code to build, but I also have to wait for the existing code on the robot to load for some reason. I have no knowledge of if it is loaded or not. I can tell if the robot has booted by looking at the RSL, but there is no indication of code. I could tether it to the Classmate, but that is busy clearing an FMS lock.

Robot crash when downloading code: I experience this every now and then. For no real reason, the robot crashes and reboots (loss of comm and code on the Classmate, RSL goes out, a minute or so later it is back up). I didn’t notice this much during the season, but had a huge issue with it at MARC (good thing the FMS lost power so I had like 15 minutes of waiting to fix it, the robot crashed about 4 times in a row before working)

A few plusses:

I really liked the use of wireless on the practice field at IRI. They gave us our radio encryption code to program into our existing DS radio, and let us use it without changing the radio on the robot.

A few suggestions:

  1. Don’t re-build and re-download everything every time. I look at what it does, and it re-builds the ENTIRE WPIlib every time I make a change to one file. I already talked to you about this one at the championships. Just reminding you.

  2. Some sort of cRio emulator for LabVIEW (I heard the C++ guys get one) would be really nice. I mean, I know I can write it myself, but then I have to worry about what happens if the WPIlib changes next year.

  3. Dual-booting Linux and XP on the Classmate could provide a locked down environment, plus it could be optimized to boot fast and run Driver Station. The XP portion could run the development environment.
    This causes two problem:
    First, you can’t use WinXP drivers for gamepads and other stuff (the blue DS had this same issue so I wouldn’t worry about it)
    Second you have to build the Dashboard for Linux and Windows.

  4. Give us the patch for the Cypress issue.

Please remind us what you are referring to here.

Thanks,
-Joe

Thank you to Joe and Greg for asking for this feedback.

After working with the FTA’s (Mark Koors and Rob Jenkins) at IRI, I would like to add this to the critique of the Classmate:

  1. Failure for the Classmate to connect to FMS is a bigger deal than Mark McLeod eludes to above. I will try to explain.

If the Classmate fails to connect to FMS (and it is only in Driver Mode), then the FTA has two options: a) run the match without that team participating, b) swap out the Classmate. The first option frustrates the team, of course (and causes a 1-2 minute delay* while making this decision). The second option creates at least a 4 minute delay*.

  1. Driver and Developer mode:
    If a team shows up to the field while logged on to Driver and Developer mode at the same time, the Classmate again does not connect to the field. In this case, a 1-2 minute delay* occurs (and that is only if it is diagnosed correctly).
    • We ran 104 qualification matches at IRI this past weekend, with a top notch field staff. Due mostly to these delays, we ran about 45 minutes behind schedule after running 104 matches.

Sincerely,
Andy Baker

There is a third fix for the Classmate failing to connect to FMS that also takes a couple of minutes. It involves the Restore stick and was documented in the FMS error sheet by FTA Pete Kieselbach after week 1.

I haven’t seen trouble with having both Driver & Developer accts logged in and have run successfully with them during the events I worked. Is there any more detail on that particular problem?
Do you know if the Classmates that exhibited the FMS-failure to link problem had been:

  1. rebooted in the field queue
  2. been awakened from sleep mode
    *]or had been kept constantly running?
    One of the troubleshooters would probably have taken note of this.

If someone has anything, I’d like additional symptoms or triggers on the inability to connect to the FMS. The code reads UDP from the port with a one second timeout, processes anything it received, and loops around for more.

Any idea if a laptop that fails to connect has ever connected? Is it the laptop settings or other issues?

Any idea if it tends to happen after a suspend, a reboot, or ???

To try and reproduce this, was this FMS lite or the real deal?

Greg McKaskle

Here’s what I know…
I worked on about a dozen of these particular FMS-fail to connect cases at official events with the full FMS. That’s why I’d like to hear more reports from other areas of the FRC world to see what patterns others saw.

Under competition circumstances we’re just trying to get them operating again, so we don’t get much opportunity to study the problem. We just try to make it go away as fast as possible. I never had the time to try it against FMS-lite. I meant to do so in Atlanta, but my laptops were all tied up when the opportunity arose.

  • It seemed to manifest when the Classmate woke from sleep, but only a small percentage of Classmates suffered.
  • Once it started, rebooting did not improve the problem.
  • It can be fixed by restoring the Classmate to KOP as-delivered (~2 hours).
  • It can be fixed by a bizarre use of the restore stick (~2 minutes).
    I can’t say that restoring prevented the problem from coming back. Standard advice quickly became “don’t ever let it go to sleep.”
    When some of them did go to sleep again the problem did return, but that’s a very subjective observation and I wouldn’t put any faith in it without collecting less-subjective samples.

There were a lot of cases at the week 1 NJ Regional, but I didn’t see a single one at the 4th week SBPLI Regional. One big difference is that most teams at SBPLI had them on constant power or shut them down completely while waiting in the queue. By week 4 the sleep problems had been widely reported and teams weren’t letting them sleep.

You saw one of these in Atlanta with team 192 in the Curie pits, I think it was.
Taking 192 as an example, they reportedly worked fine with FMS at their regional, but failed to connect to FMS during the Championship Thursday practices. It was fixed at the field and tested fine with FMS. We rebooted their Classmate from scratch and it still connected fine.
Unfortunately, they did not have an inverter to keep the Classmate powered and instead put it to sleep for the long walk and wait in the queue. When it woke up it no longer recognized FMS.

What interests me most about Andy’s report is the attribution to conflicts between the Developer and Driver user accounts. I often had teams running both accounts so some of the sleep problems could be quickly fixed. The only FMS problem with having both accounts open that I knew about was early on when users could accidentally run two copies of the Driver Station software, but you guys prevented that with one of your updates.

Thanks for the info. Anyone else with observations?

Greg McKaskle

I have never seen this issue on our DS. We were at 2 regionals (FLR and philly). At both regionals I was always sleeping the classmate before each match to save battery. At FLR we had the cypress plugged into the usb hub along with 2 other joysticks (KOP I believe), the hub being plugged in with only the black cable. For philly, the cypress was on the 2nd classmate port and it was being removed before each sleep. In software, the only changes from the default setup were using a custom dashboard, but no data was being sent by the robot (was used for the camera only, no tracking data). I had also bypassed the lockout on task manager in Driver mode for debugging early in the season and never bothered to re-enable it. We did lose connection once or twice on the field, but I believe that was related to our broken ethernet port, since it never happened again after we duct-taped an extension cable to our port.

At FLR though, our two alliance partners in quarterfinals could not connect to the field, but seemed to be tethering fine. One of them got a new classmate after the first match and was able to play in the second. I’m not 100% sure this is true, but I think I saw the default dashboard on both of their robots, and both were using the cypress, and I think they may have slept too.

Has anyone checked to see what happens if a classmate is waked up when the ethernet cord is plugged in? I was having our drivers make sure the classmate was fully running before plugging into the field.

On another note, would it be possible for NI to release an optional update to the DS that includes the cypress fix so that it can be gone for the remaining offseason events?

On another note, would it be possible …

To recap the conversation about the Cypress issue, I was personally able to provoke what looked like an issue with the Cypress device driver by sleeping the laptop. It didn’t happen each time, but in ten sleep/awake cycles, I saw it once and figured that was good enough to write a bug report. Yes, unfortunately this test was run after the season was already underway. My preference would be to let FIRST determine how I/O will be done for the DS in upcoming years, and include sleep/awaken tests to verify it works as intended and document any workarounds or configuration issues that will improve the issue.

If the same HW is to be used and it is not possible to get the sleep issue fixed at the driver level, it should be possible to update the DS to restart the driver when it appears to be hung. Code to diagnose and restart the driver has not been written, or I’d gladly let encourage FIRST to release it. In general, I’d prefer to fix the root problem instead of bonking other SW over the head when they don’t seem to be responding.

Greg McKaskle

I may reiterate what others have already said, but I want to be thorough:

I. Classmate
A. Watchdog problems are still an issue. We fixed the majority by changing powersave and bios options, but it would be nice to see this as part of the formal instructions for rookies AND veteran teams.
B. Transition times are bad between the driving account and the developer. Even worse is the boot time, and the time required to log back out then back in after being FMS locked.
C. When IN the driver station mode, the USB can be problematic. We’ve had multiple instances this year of it refusing to acknowledge the logitech gamepads we are using.
D. The USB is not consistent in determining the joystick order. It had a bad habit of swapping the way they are installed (when they do install).
E. The E-stop wait. Dump it. A warning sign is enough.
F. PLEASE allow us to use any laptop instead of the classmate. I can buy a $200 ebay laptop that costs less than the classmate, has a longer battery life and more processor power, can be used to program AND as the driver station, and a whole host of other things as well. We used a normal laptop during most of the beta test and never had an issue.

II. PD board.
A. It would be nice if could change to standard-size nuts rather than metric on the positve and negative leads.
B. It would be nice if the wagos could be turned to point upward, so that it isn’t nearly impossible to work with them once the PD board is installed on the robot.

III. Crio
A. Power port does not have a positive enough connection to the wires.
B. A short ethernet cable and female-female connector should be mandatory as part of the robot wiring. This is what our team has been doing without issue to prevent the over-use of the crio ethernet port.

IV. Can
A. It would be nice if a can-manual or a “Can for Dummies” came along with next years instructions. This year, unless you’re fairly electrically savvy, most teams shied away from Can. It opens some great possibilities…

V. The oft-maligned game adapter.
A. We hot-glued the reset button and the power button. Our robot still reset once from an extremely hard hit. Would it really be that hard to take a tiny form-factor bridge and mount it inside a cradle that fits in the crio itself? Then it could pull power from the Crio, be easily removable, etc.

B. The backup game adapter is horrid. Connection times are ridiculous. We bought one as a backup, but never used it except on the practice bot because of the start-up time. It would have made it a crunch to even set our autonomous mode on the field.

VI. Joysticks
A. I think it would be smart to substitute the commonly used logitech gamepads for the attack joysticks. The gamepads are simply more flexible (requiring only one pad for tank style drive) and are easier to transport. They also allow your drive team more flexibility in where they stand and how they move around.

Even though the joysticks come in the KOP does not mean that they have to be used. The gamepads are cheap enough that if any team wants to use them, then they can (about $20).

Our team had a problem connecting to FMS during Bayou. We troubleshooted, and we found out that whenever we tethered the robot, the subnet mask would always reset to 255.0.0.0. I had to manually go the tcp/ip settings before every match and change the subnet back to 255.255.255.0. This fixed everything.

We also had the all-too-common loose ethernet port problem.

During this match at the 1:14 mark you can see 3389 ramming into the alliance station wall causing the ethernet cord to fall out. I plugged it back in fast enough for it to reconnect.