Go to Post Thou Shalt Not Leave Crap In The Walking Path. - Billfred [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rate Thread Display Modes
  #16   Spotlight this post!  
Unread 04-03-2016, 03:04
Peter Mitrano Peter Mitrano is offline
Registered User
no team
 
Join Date: Jan 2016
Location: Worcester MA
Posts: 21
Peter Mitrano is an unknown quantity at this point
Re: Potential Git and CI integration

Quote:
Originally Posted by Jaci View Post
For a robotics program, this seems like the wrong way to do things. JUnit has it's place, but I don't think FRC is it. Unless your code (and your framework) are specifically designed for isolated testing, unit testing is not really suited for a robotics / embedded software environment.
I'm inclined to agree on this one.

Quote:
Originally Posted by Jaci View Post
The main reason I say this is because of things like Motion Profiling, PID and any kind of Feedback. While, yes, it is possible to test, the tests often don't reflect actual operation, or do a very poor job of it. That being said, that's probably just my opinion weighing in.
I'll push back on this one. I see no reason why you can't tune a good robot model such that the same PID parameters work in simulation. It's a bit of work, but people have done it with the Atlas robot, Ardupilot, and Baxter, which are all far more complicated and dynamic robots.

Quote:
Originally Posted by Jaci View Post
This seems more viable, but it's usefulness would depend a lot on what gazebo sends back. If you're sending back things like what speed the motors are, or if a port is on or off, that's often quite useless from a testing perspective. Something more useful would be things like displacement of the Robot. Things like that can be compared against 'what it should be', and could practically test things like PID loops and motion profiles.
This I think is gazebo's strong point. You can get literally anything. You want to test that your auto drives forward 5 meters? Great, just get the world pose before and after. You want to test that your LaunchCatapult command works? Cool, put a ball in and check where it lands, or even just get the pose of the joint. You can also piggy back on any sensors your robot has, and check something things like driveUntilWall, by comparing absolute distance between the robot and a wall, versus what your rangefinder says and where your robot stops.
That said, I still think even testing "is my motor spinning" is a huge improvement over what most teams do (no testing at all).
  #17   Spotlight this post!  
Unread 04-03-2016, 03:28
SoftwareBug2.0's Avatar
SoftwareBug2.0 SoftwareBug2.0 is offline
Registered User
AKA: Eric
FRC #1425 (Error Code Xero)
Team Role: Mentor
 
Join Date: Aug 2004
Rookie Year: 2004
Location: Tigard, Oregon
Posts: 488
SoftwareBug2.0 has a brilliant futureSoftwareBug2.0 has a brilliant futureSoftwareBug2.0 has a brilliant futureSoftwareBug2.0 has a brilliant futureSoftwareBug2.0 has a brilliant futureSoftwareBug2.0 has a brilliant futureSoftwareBug2.0 has a brilliant futureSoftwareBug2.0 has a brilliant futureSoftwareBug2.0 has a brilliant futureSoftwareBug2.0 has a brilliant futureSoftwareBug2.0 has a brilliant future
Re: Potential Git and CI integration

I think there is real value in testing with a simulator. I don't think it's the right tool for typical unit tests, and I probably wouldn't use it to tune PID constants either; the real value in the simulator is in being able to see all the pieces work together and interact with each other and the environment. Off the top of my head, I might put together an interface that looked something like this:

Code:
struct Object_status{
	//Put position, orientation, speed, here
	std::map<std::string,double> etc; //extra in case the object has extra configuration like part that move relative to the others
};

struct Object_setup{
	Object_status status;
	std::string path_to_model;
};

struct Robot_setup{
	Object_setup object;
	std::string path_to_executable;
};

using Name=std::string;

struct Initial_setup{
	std::map<Name,Object_setup> objects;
	std::map<Name,Robot_setup> robots;
};

using Status=std::map<Name,Object_status>;

typedef bool (*Done_callback)(Status);

Status run_sim(Initial_setup,Done_callback);
  #18   Spotlight this post!  
Unread 04-03-2016, 10:15
virtuald's Avatar
virtuald virtuald is offline
RobotPy Guy
AKA: Dustin Spicuzza
FRC #1418 (), FRC #1973, FRC #4796, FRC #6367 ()
Team Role: Mentor
 
Join Date: Dec 2008
Rookie Year: 2003
Location: Boston, MA
Posts: 1,114
virtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant future
Re: Potential Git and CI integration

Quote:
Originally Posted by Peter Mitrano View Post
That said, I still think even testing "is my motor spinning" is a huge improvement over what most teams do (no testing at all).
Absolutely. This is the huge value of a easy to use simulator, and it allows our programming team to reliably create code before the robot is done and after it's in the bag.

In python, it's far more important to do off-robot testing because it's a dynamic language -- there's a large risk of crashing due to misspelled variables, etc. Because of this we've had excellent test/simulation support for several years now. In the course of development for the pyfrc simulator/test harness, we have a set of 'default' tests that just do the following things:
  • Run autonomous mode (with support for running multiple modes)
  • Run teleop mode
  • Run a 'full match'
  • Run a 'full match' with a monkey bashing on the controls randomly

And this meets the needs for most of our robot code -- eg, make sure it doesn't crash due to misspelling of a variable name.

However, we've found there is *some* value in junit-style tests -- but usually we only go to the trouble of writing those when you've got a complex state machine that you want to make sure deals with the edge cases correctly.
__________________
Maintainer of RobotPy - Python for FRC
Creator of pyfrc (Robot Simulator + utilities for Python) and pynetworktables/pynetworktables2js (NetworkTables for Python & Javascript)

2017 Season: Teams #1973, #4796, #6369
Team #1418 (remote mentor): Newton Quarterfinalists, 2016 Chesapeake District Champion, 2x Innovation in Control award, 2x district event winner
Team #1418: 2015 DC Regional Innovation In Control Award, #2 seed; 2014 VA Industrial Design Award; 2014 Finalists in DC & VA
Team #2423: 2012 & 2013 Boston Regional Innovation in Control Award


Resources: FIRSTWiki (relaunched!) | My Software Stuff
  #19   Spotlight this post!  
Unread 05-03-2016, 00:30
AustinSchuh AustinSchuh is offline
Registered User
FRC #0971 (Spartan Robotics) #254 (The Cheesy Poofs)
Team Role: Engineer
 
Join Date: Feb 2005
Rookie Year: 1999
Location: Los Altos, CA
Posts: 803
AustinSchuh has a reputation beyond reputeAustinSchuh has a reputation beyond reputeAustinSchuh has a reputation beyond reputeAustinSchuh has a reputation beyond reputeAustinSchuh has a reputation beyond reputeAustinSchuh has a reputation beyond reputeAustinSchuh has a reputation beyond reputeAustinSchuh has a reputation beyond reputeAustinSchuh has a reputation beyond reputeAustinSchuh has a reputation beyond reputeAustinSchuh has a reputation beyond repute
Re: Potential Git and CI integration

Quote:
Originally Posted by Jaci View Post
For a robotics program, this seems like the wrong way to do things. JUnit has it's place, but I don't think FRC is it. Unless your code (and your framework) are specifically designed for isolated testing, unit testing is not really suited for a robotics / embedded software environment. The main reason I say this is because of things like Motion Profiling, PID and any kind of Feedback. While, yes, it is possible to test, the tests often don't reflect actual operation, or do a very poor job of it. That being said, that's probably just my opinion weighing in.
JUnit != Unit Testing. JUnit is a testing framework. It does a great job of giving you tools to check that the expected results are correct, organizing test cases and reporting failures. You are welcome to use it only for unit tests, but it can be used for any type of testing where there is a need for it's features. WPILib uses JUnit running on a roboRIO to make sure PID loops and sensors are all working.

Testing all depends on what you put in it. If you are going to try to mock out a robot subsystem to do unit testing, you are going to have a bad day. Figuring out by hand what a system should return is hard. If you write a simulated subsystem with simulated sensors to support more of an integration test, you can get some very powerful results. You can model almost all FRC subsystems as a motor attached to a mass.

We do pretty exhaustive automated testing of all our control loops and motion profiles, all inside google-test in C++. Last time I counted, we had 20+ tests for our superstructure, and many more tests for things like motion profiles. If you can't control a model of your robot pretty well, you have no hope of controlling the real thing. Unit tests are also very helpful at helping you understand when smaller units of code are performing as expected. This enabled us to have all the software we wrote before the hardware was done this year up and running on the robot in 24 hours, with no modifications after calibrating pots and measuring limit locations. Our arm and flywheel controllers all worked pretty well and were stable, just from the quality of our simulations. That's pretty powerful and well worth the investment.

Motion profiles are actually much easier to test than controllers. You can compute ideal trajectories and see if your generator successfully generates and follows them. You can also write tests making sure they have the bounded acceleration characteristics you expect and bounded velocity characteristics.

I believe that FRC should try to teach some of the best practices in industry to our students. Testing is a key part of developing and maintaining production software. I write both unit tests and integration tests in both industry (safety critical vehicle automation) and at robotics, and it drastically helps the code quality in both locations.
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 00:17.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi