Potential Git and CI integration
 

Many teams in the FIRST Robotics Competition use Git to manage their code.. Some teams even test their code on Continuous Integration services such as Travis CI or Jenkins. With the growing popularity of Git in FRC and the good coding practices that come with it, I was wondering if there was any interest in a System that would work with the RoboRIO to manage code versions and make deployment a lot smoother.

The idea is simple, whenever code it committed, the CI service builds the artifacts and tests them. If all tests are successful, the artifacts are uploaded to an external server where, upon update or boot, the RoboRIO will fetch the artifact, load it and restart the program. Alternatively, the required development tools could be loaded onto the RoboRIO system (javac/gradle or the FRC Toolchain) and the RoboRIO acting as a Git Remote to push to. This could potentially streamline the build process and make management much easier to achieve.

For RobotPy, I believe no build system would be required as the sources could be loaded really easily, whereas for Java teams, the FRCUserProgram (or module if running Toast) would be built and copied to a temporary cache to be reloaded on code reboot. C++ teams would require the toolchain to be ported to the RIO.

Before any developments are made, I'd like to ask if there is any interest in the project and if so, what features or concerns you have. A repo has been setup over at OpenRIO and is free for anyone to contribute to, however, feel free to post your thoughts here.

I look forward to the development of this project,
~ Jaci R

Re: Potential Git and CI integration
 

For RobotPy, no build system is required. We experimented with it this off season, one of the tests was if we could put it under TravisCI easily (and thus get testing status before merging automatically) The code is available https://github.com/FRC125/NUPY and the tests are available at https://travis-ci.org/FRC125/NUPY

The big change to the .travis.yml file was telling it to use the right version of python and install dependencies. The config file is https://github.com/FRC125/NUPY/blob/master/.travis.yml

Re: Potential Git and CI integration
 

For an interpreted language such as python, it's super important to test your code before you upload it to the robot, as even simple syntax errors or referencing a variable that doesn't exist can put your robot dead in the water. That's why for years RobotPy has had first-class support for running your robot code without the robot -- which makes it super simple to integrate with a online CI system like Travis-CI.

It would be nice if off-robot testing were a first-class concept for the other robot languages, and I think that improving that situation would have a bigger impact on teams. After all, not everyone has an extra RoboRIO lying around to run a CI system on... and really, you shouldn't be running your robot on internet-connected networks!

Re: Potential Git and CI integration
 

Maybe next year after the season we can build up a proposal on how to get this into the official WPILib. I've seen firsthand how much a good simulator base helps test code and makes sure it always runs properly. It would be nice to potentially have this be a native library replacing the HAL, like we currently do in the unofficial languages, but I don't really think FIRST would go for that solution.

Re: Potential Git and CI integration
 

I know 254 put together a FakeWPILib they were using last year, I'm currently working on getting that up and running locally. Hopefully after that it'll be simple enough to get it running under CI. I'll let you know if I make any progress. Obviously this still leaves C++ and LV teams.

Re: Potential Git and CI integration
 

I think LV has its own testing stuff you can use... but obviously, there are a lot of challenges running LV + CI on a commercial CI provider.

Re: Potential Git and CI integration
 

The Toast project I've been working on is designed to do just that, by replacing WPILib's networking and HAL when run in a simulation environment and formatting it into a GUI for the user to read instead, with hooks for modules to do what they like with the data if they want custom testing support.

In my signature you can see the Toast project is fully supported under CI and metrics regarding documentation and code quality are generated as well. Modules also have full CI testing support thanks to GradleRIO

Re: Potential Git and CI integration
 

For C++, 971 puts the hardware interface code in a separate process and uses custom message passing to communicate the hardware state to the control loops, etc. We can then write the hardware interface code once, test it on the hardware, and then move on. We are able to then write unit tests which simulate the hardware and generate messages with simulated sensors and physics. This allows us to do all our controls development before the hardware has been finished.

Re: Potential Git and CI integration
 

That sounds overly complicated for the majority of teams. Cool... but it seems most teams are just slightly above the "can reliably hit the compile button" threshold.

But any chance this is published anywhere?

Re: Potential Git and CI integration
 

http://robotics.mvla.net/files/frc97...14_code.tar.xz

The good stuff is in frc971/control_loops/. It currently only builds on debian wheezy, amd64. One of these days, we'll finish open-sourcing the 2015 code...

It is probably beyond what the majority of teams can do, but is useful for showing what is possible and helping inspire students to do better. I'll never claim that our code is 'simple' or 'easy'. :p

Re: Potential Git and CI integration
 

If it wasn't overly complicated for most teams, 971 wouldn't do it ;)

Re: Potential Git and CI integration
 

For anyone who's interested in trying to compile this on Debian Jessie or recent Ubuntu releases the first thing you'll run into is that the libcloog version it's looking for isn't available in the repositories.

Re: Potential Git and CI integration
 

Probably. We made a conscious decision to support only one OS version to keep it easier to support. We are revamping our build system in the next ~month, and hope to release this year's code building with the new build system under Jessie. This should also reduce our dependency on the host OS and make the build more repeatable. (If you like to geek out about build systems, check out bazel.io.)

Re: Potential Git and CI integration
 

Digging up this thread because I think it's phenomenal.

am the current FRCSim developer. As FRCSim develops, this should hopefully become much easier to do. Gazebo can be run headless on a server, and since all the communication between WPILib and gazebo is done over Gazebo Transport, a message passing system built off protobuf, it should be way easier to write tests.

How would you guys approach testing with frcsim? Would you write junit tests that calls your robot code? Or would you run your robot code as-is, and write a seperate "supervisor" program that messages the the robot code and listens for the appropriate effect in gazebo? Both seem viable to me.

Re: Potential Git and CI integration
 

Testing code in the Robotics realm is a two-way street. Personally, for testing, I think just running the code yourself in a simulator is good enough for FRC (but that's up to personal opinion).

For a robotics program, this seems like the wrong way to do things. JUnit has it's place, but I don't think FRC is it. Unless your code (and your framework) are specifically designed for isolated testing, unit testing is not really suited for a robotics / embedded software environment. The main reason I say this is because of things like Motion Profiling, PID and any kind of Feedback. While, yes, it is possible to test, the tests often don't reflect actual operation, or do a very poor job of it. That being said, that's probably just my opinion weighing in.

This seems more viable, but it's usefulness would depend a lot on what gazebo sends back. If you're sending back things like what speed the motors are, or if a port is on or off, that's often quite useless from a testing perspective. Something more useful would be things like displacement of the Robot. Things like that can be compared against 'what it should be', and could practically test things like PID loops and motion profiles.