2018 Beta Testing and 2017 Usage Reporting
Written by Kevin O’Connor, FIRST Robotics Competition, Robotics Engineer
Beta Testing
This year we are once again looking for 90 teams, 30 for each software language, to help us test the 2018 FIRST Robotics Competition software. A small number of teams have been invited to bypass the application process based on past performance in Beta testing. Most of the spots are still up for grabs, though, so make sure to fill out the application found here by Sunday, September 24th, if you are interested. Beta teams will be asked to upgrade their 2017 robots to the 2018 software, perform some specific tests with any significant changes or new tools, and help educate other teams about the changes for 2018. Beta Testing will begin approximately the 2nd – 3rd week of October and each task will have specific deadlines through approximately the end of November. Teams may continue to test through Kickoff, but software starts to lock in early December so testing done in December may not be done in time to fix any bugs discovered.
2017 Usage Reporting
One of the under-the-hood features of the FRC Control System software is a system called Usage Reporting. This system tracks what WPILib objects are created in each team’s code and reports the data back to FMS when the robot is connected to the field. This data helps FIRST and our Suppliers understand how teams are using the Control System which can give us insight into things such as adoption of new features or usage of legacy classes.
Here are a few numbers from the data that teams may find interesting:
25,151
The total number of motor controllers used on 2017 robots
6,296
The total number of USB input devices used to control those motors
2,0451
Total Encoders used to get feedback from those motors
1,4122
The number of teams that used pneumatics (with the PCM) on their 2017 robot
762
The number of Digital Outputs used to control non-actuators (lights, sensors, etc.) on 2017 robots
33
Number of teams programming in Python
15
The number of Analog Outputs used to control non-actuators (sounds, lights, etc.) on 2017 robots
The complete processed data set can be found here.
A few notes on this data:
The data has been semi-anonymized. Team numbers have been replaced with rookie years and the data has been resorted so teams are not in order by team number.
We can only track the objects teams create in code. If you create extra motor controllers that aren’t on your robot3, they will still be captured by this system. If you create motor controller objects of the wrong type, that wrong type will be captured by this system.
Counted objects and TRUE/FALSE show the largest number of any given object used in any one match (i.e. if a Robot had 2 Encoders in match 1 and 3 Encoders in match 7, 3 will be reported). Language and Framework report what was used in the last recorded match.
Some objects naturally result in double counting (Encoders use Digital Inputs)
1 Only includes encoders plugged directly into the roboRIO
2 Compressor may not show for C++ and Java teams that didn’t use the Compressor object. Counting the # of teams that had > 1 Solenoids is a more accurate count of teams using pneumatics.
3 Example: A handful of teams report more than 16 motor controllers which would be awfully tough with only 16 channels on the PDP.
When they uploaded this document, they put it at the same url as the one they linked to in the blog last year. Is the 2016 report still available anywhere?
Edit: 3946 is row 1553 this year, based on rookie year, language, infrastructure, 4+ CANTalons, 1+Talon, 1+Analog Input, 1+Encoder (direct). But…we didn’t have three joysticks or use arcade this year??
Edit2: Three joysticks is because we used two flight sticks as a primary and a single xBox as a secondary, and put all three in the code. I couldn’t find arcade drive code anywhere on our 2017 competition GitHub; it must have been something we used in the last match but never synced to the repository.
I’d be interested to see this as “% of market” as well, since the raw values are going to scale with # of teams and average # of motors per robot (I think average number of motors per robot in 2017 was higher than 2016 just based on the game)
I’m interested in the two Unknown programming languages, rows 359 and 580. Anyone who used a nonstandard language care to come forward? Kotlin reports as Java.
Did you actually use the compressor object in your code? Instantiating a Solenoid will run the compressor automatically, but will not report a compressor usage.
Is this data from a log file somewhere on the RoboRio or DriverStations? Or is it simply captured when the robot executes its code when plugged into the FMS? I’m no software engineer but at time my curiosity gets me to wonder and explore deeper into the how it works.
I assume the boot order looks something like:
Driverstation Launch >
Contention to Robot using [USB, Wifi, Ethernet, FMS] >
Match Team Numbers >
Launch Robot user code [Log file created or FMS listens for objects created]
If there are log files and we knew what date, time, match#, we could use that for a number of fun or useless reasons. Seems our Diverstations are logging A LOT of useful data. I even heard of teams that had hard drives overwhelmed with log files. RoboRio resources are limited so if there are Logs they might get overwritten.
We have had a few time after a code update the TalaonSRX’s don’t either get created or set to a mode. We use them all as Velocity closed loop but they don’t respond to any user commands. Using the Rio’s Web dashboard the CAN bus does see all drives and the test function show each drive as no mode. I have chatted about this in the past. I can’t 100% reproduce the conditions but for some reason it feels like it tied to us using the Driverstation TEST mode before putting the robot into Teliop or Auto. If there was a log file of objects created that would eliminate one possibility.
All the data is captured through the UsageReporting class in WPILib. That data is reported back to the driver station that then pushes it to the FMS. You can find most data yourself in the DriverStation logs.
It should be noted that at no point is the match number communicated to the driver station or robot.
