Hibner: Software Testing Via Simulation

[Chris Hibner]

At the FIRST Robotics Conference I will be showing how to test your control software using simulation. The presentation will be Thursday at 8:00 a.m.

Here's the abstract:

In industry, modern control system software is thoroughly tested in simulation before it ever sees the actual hardware. With the tools provided by National Instruments, testing FIRST robotics software in simulation is easily within reach. This presentation will cover modeling techniques and how to use the models to test control strategies and software.

The presentation aims to cover the following topics:
1) why use simulation?
2) How to create a simulation environment to test your software.
3) How to create simple dynamic models of your robot system in order to test more complicated feedback control.
4) Taking your software from the simulation to the cRIO (and vice versa).

The presentation will be highly interactive and audience participation will be highly encouraged (Pavlov would be proud). The presentation will start with a clean slate, and a simple simulation example will be created. As the presentation progresses, the simulation will be expanded to cover more complex systems. As with my other presentation, all of the examples will be developed live. That should hopefully result in a good, fun learning experience for everyone (and perhaps some fun at my expense).

If time allows, the final example will be simulation of autonomous software using a simple robot drive system model.

If there is anything you would like to see addressed, feel free to post it in this thread.

[JamesBrown]

This is a topic that is of great interest to me and that I would like to share with the teams I work with. Is there any chance that this could be taped and made available online? Any slides that could be posted would be useful but with the way you plan on creating your examples live I think video would be the best way to experience the presentation for those of us who can't be in STL.

[JesseK]

What language will the simulation be created in?

[Chris Hibner]

All of my material from the conference as well as the example code will be posted to chiefdelphi.com after I return from St. Louis.

Collen Shaver will have to answer questions regarding video.

The examples will be done in LabVIEW, but I will also touch on how to do simulations in C++ and Java.

[Chris Hibner]

I forgot to add one thing to the original post:

If your team had a software mishap, such as your robot tried to destroy itself, you had a software bug that was difficult to solve, a broken sensor caused your robot to go haywire, or you tried to do something that never quite worked out, I would love to hear about it. If the story is interesting enough or makes a good example, I'll try to include it as part of the presentation. I'm always looking for good, real-world examples to use.

[JesseK]

I'm not sure about how the timing of this presentation will fit into my own schedule, so I'll post a story here.

Thursday, April 7, 2011: VCU Day 1
We put the bot on the field after readjusting the chain runs for reliability after a demoralizing performance in DC. In our first practice match, the robot did a very dangerous robot dance -- the lift was up to 10', the bot would go-turn-stop-go-turn-stop and do so very violently while doing a sweeping left turn (tough to describe...). It only progressed worse as autonomous continued due to the side-to-side rocking. Finally the bot stopped due to an internal timer (9s) and the lift came back down. We were pretty sure the welds on the lift would pop since the entire base of the lift was twisting as this went on. We were lucky. On the plus side the chains didn't pop off during that entire episode, reassuring us that they were fine for the rest of VCU, and we're now 100% sure our c.g. is "low enough".

It turns out that the double-throw pneumatic valve wiring to the shifting transmission cylinders went bad while we were fiddling with something else. One side of the drive train was stuck in high gear while the other side was stuck in low gear. The 'drive straight' 'P' on the gyro PID wasn't set nearly aggressively enough, so the software couldn't compensate for the 2.56:1 ratio difference in mechanical speed that resulted between the left side and the right side. 100% PWM signal to both sides resulted in very different speeds. Later we tuned the gyro's PID and ran 11 consecutive perfect autonomous modes.

It's a good case for simulations that test error conditions or mechanical failures on the bot.

[vhcook]

Thank you for doing this presentation. It gave me a lot of ideas for next year.

[Pmaxm00]

If we weren't able to attend for certain reasons (Busy driving the robot or at a driver meeting), is there a way we can get information about this? I am very interested but I wasn't allowed to go because there was a drivers meeting.

[vhcook]

Chris indicated he'd have all the presentation materials and examples from both his sessions posted in CD-Media possibly around the middle of next week or so.

[Chris Hibner]

I have posted the presentation material and code samples here:

http://www.chiefdelphi.com/media/papers/2494

Sorry that it took so long.


The material and examples for the autonomous presentation should be posted within the next two days.

[ajlapp]

This simulation technique is awesome! We were doing very simple simulations to test logic and various code samples but I never fully thought through how to add some virtual models to our testing scheme.

I've already sorted out a few pesky bugs with our PID scheme.

A few questions...

What is supposed to be on page 6 of the presentation slides? Its empty.

On page 21 you discuss adding some limits to the model. Did you add these live during the presentation? I don't see this limiting in the sample code.

Page 29 shows some sensor noise modelling. It appears to always add 0 noise. What is a good noise value for different sensors? I'd like to use "D" in my scheme, so realistic noise would be great.

Thanks in advance.