[FRC Blog] Beta Testing and Usage Reporting

https://www.firstinspires.org/robotics/frc/blog/2020-beta-testing-usage-report

Beta Testing and Usage Reporting

Sep 10th, 2019 Written by Kevin O’Connor, Robotics Engineer, FIRST Robotics Competition.

Beta Testing

The calendar has rolled over to September which, for the FRC Control System team, means it’s time to start thinking about beta testing! We’re looking for 20-25 teams per software language to participate in our annual FIRST Robotics Competition Control System Beta test to help shake out any bugs in our updates to the software before the season. This year these changes include a new software documentation site, a re-written Command Based programming structure (don’t worry, the old one will stick around for now!), some new kinematics and control classes, and more. Beta Testing will begin around the second week of October and have specific task deadlines through the end of November. Teams may continue to test through Kickoff, but software starts to lock in early December so testing done in December may not be done in time to fix any bugs discovered. Interested teams should fill out the form here by Sunday, September 22, 2019. Teams should receive notification of selection status no later than September 27, 2019.

Usage Reporting

The FIRST Robotics Competition software has a feature called ‘usage reporting’ that tracks what WPILib objects are created in each team’s code and reports that data back to the field when the robot is connected. Data from the 2019 season can be found here. Some notes about the data:

  • The data has been semi-anonymized. Team numbers have been replaced with rookie years and the data has been re-sorted so teams are not in order by team number.
  • We can only track the objects teams create in code. If a team creates extra motor controllers that aren’t on the robot, they will still be captured by this system. If a team creates motor controller objects of the wrong type, that wrong type will be captured by this system.
  • Counted objects and TRUE/FALSE show the largest number of any given object used in any one match (i.e. if a Robot had 2 Encoders in match 1 and 3 Encoders in match 7, 3 will be reported). This means that if a team that switched motor controller types, or IMUs, or anything else during the season, the sheet will show numbers for both devices. Language and Framework report what was used in the last recorded match.
  • Some objects naturally result in double counting (e.g. Encoders use Digital Inputs).
1 Like

I wonder who that could be… though the LabVIEW thing is questionable, we gave that up years ago.

1 Like

The fact we’re up to 77 Python teams is super interesting. Additionally, ~18% of teams used Spark Maxes over CAN, which is super neat to me.

3 Likes

Some brief number crunching/looking at things compared to 2018, and singling out what rookies are doing.

13 Likes

Wait - only 45 teams used Command-based Java?

To be fair, there’s a reason we rewrote it from scratch for 2020. :man_shrugging:

1 Like

I don’t think it’s counting all of the command-based as command-based for some reason… I’m pretty sure 1885 is my team, and we used command-based, but it shows up as Timed. Maybe we had to do something specific.

It won’t report correctly if you called Scheduler.getInstance() before the TimedRobot constructor ran. The FMS stores the last framework value sent.

I’m not super involved with the controls side of things, but why is that interesting? Isn’t that the standard way to use a Spark Max?

Yes, using Spark Maxes over CAN is the “correct” way to use them, but not the only way. Some teams opted to control them over PWM due to the CAN interface being lackluster earlier in the season. My main point of observation was that 18% of teams took the plunge on brand new hardware never used before in FRC.

4 Likes

You’re right Tim, that is interesting. Putting that into perspective, that’s roughly equivalent to the percentage of teams using LabVIEW.

The language data has been surprisingly linear the last few years. At this rate, LabVIEW will have no users in 5 years (they’ll probably hover around whatever rookies decide to use it per year). Python will overtake LabVIEW in four years and overtake C++ in six years. C++ will overtake LabVIEW in 3 years. That means after three years, LabVIEW will be the least used officially supported language.

5 Likes

A bit more on the changes coming this season.

  • ScreenSteps replacement, ScreenSteps will be redirecting to https://docs.wpilib.org/ shortly before kickoff. This is the part of the WPILib project that I and @AustinShalit are primarily responsible for. These docs have examples built against WPILib to ensure validity, have links checked automatically, and have a sane contribution process via Github.
  • Kinematics classes for Swerve, Mecanum, and DifferentialDrive. These ease the burden on closed-loop driving for these drive types.
  • Command-Based replacement, this is a complete rewrite of the Command-Based system with the previous system being deprecated. Info on this system can be found here. @Oblarg is the primary individual to reach out in regards to the command-based system.
  • LQR for being able to finely characterize different robot mechanisms. This will be moved into the WPILib organization as soon as distribution gets worked out.
  • Units (for C++) and helper methods (for Java) have been added to reduce the amount of issues teams can have between different systems and their units.
  • PIDController no longer has an async feature (this was a footgun for many teams), and returns a value that teams can use.
  • H.264 support is currently being worked on for CScore API, but this is not guaranteed for 2020 due to some internal implementation issues.
  • RobotBuilder will/is in the process of being updated to support the new Command-Based framework.
  • IterativeRobot will not be removed this year, as we see a potential for packet based robot control to be a feature. This may be rewritten/rethought in the future.
  • Various bugs and issues have been fixed throughout the library
  • We are using an internal maven host now for artifact distribution. This shouldn’t affect teams for the most part if they are using the all-in-one installer.

Note that this does not constitute an official change log, and I probably have missed a few things. Things are still being changed and pull requests merged. We’ll be happy to respond to any questions. An official change log is currently being worked on.

6 Likes

There’s trapezoidal motion profile support and the ProfiledPIDController class for 2020 as well (you give it an unprofiled goal and it constrains the change in setpoint over time to the max velocity and acceleration). It’s motion magic/smart motion but on a roboRIO and with the actual industry-standard term for it. :stuck_out_tongue:

Ramsete drivetrain control and quintic hermite spline 2 DOF trajectory generation are in PRs at the moment and are expected to be merged after we kick the tires a bit and write more tests.

Note: LQR is for controller design, not system identification. The robot characterization tool gives you Kv and Ka feedforwards. We were able to use those to back out an accurate state-space model and design an optimal PID controller via LQR (theoretically no online controller tuning required).

5 Likes

I would very much like to have Python as a supported language. For a lot of us, that’s what we’re used to using at work, and can also find more mentors who regularly use Python.

5 Likes

If these make it in, expect there to be matching commands available, as well :wink:

2 Likes

The main barrier to Python becoming official is finding maintainers to port features. We have like two people who regularly contribute to wpilibc and wpilibj as it is. We’ve had a lot of people contributing docs and tests though, which is super nice. A high school student also contributed the kinematics and traj gen impls.

2 Likes

Often I find myself not feeling up to contributing to WPILib due to me not wanting to implement an idea in both Java and C++, for whichever reason that may be. Feature parity is important and that kind of rule is entirely valid, however I do think it’s possible to open up WPILib to be more welcoming to new contributors. I don’t entirely know the solution for it, but I imagine I’m not the only dev who feels that way. Perhaps a stronger use of the Projects tab on GitHub could help? Not sure, though. Is there a Slack that prospective contributors can join? The TBA slack is incredibly helpful for TBA contributors.

Introducing Python is, yes, a lot of work. However, it’s something that I think would drastically, noticeably, measurably - however you want to frame it - positively impact the program. It is the single thing I want next for the program now that bag is gone. More than NY districts. And that’s saying something.

8 Likes

Feature branches. Solved. I won’t even charge consulting fees for this suggestion.

3 Likes

I’m curious to see the performance implications of the new controls features in python though. Python struggled to avoid loop overruns last year even with nothing running due to condition variable overhead in the watchdog code. It’s kind of silly.

2 Likes