The Great Drive Base Performance Characterization Data Sharing Thread

A few months ago, we published a whitepaper on a method for characterizing FRC drives. If you haven’t read it, please do so (it can be found in my signature) - we think many teams will find it extremely helpful.

The paper, among other things, provides a convenient framework for the sharing of FRC drive performance data; the constants Vintercept, Kv, and Ka (as described in the paper) jointly describe the majority of the non-turning behavior of a FRC drive to a high degree of accuracy. Combined with information on drive motor choice, gearing, and wheel size, they also give detailed insight into the degree and nature of frictional losses in the drive.

In this thread, I invite teams to share the results of the characterization described in our paper, in the hopes of compiling a reasonable impression of the “typical” behavior of an FRC drive - something which the community currently lacks, and (I think) which would be of significant value. The tests are easy to run for any team with encoders, and the data not difficult to analyze.

This endeavor will only work, however, if participants a standard test procedure. I will describe the procedure, which consists of only two tests below. The tests I outline are designed around differential drives, but can easily be adapted to most other types of drives with a bit of imagination.

For all tests, the following data must be recorded: robot velocity, voltage seen at motors (for teams without TalonSRX motor controllers who cannot measure this directly, battery voltage will suffice - but please indicate that you have done this in your post), and time. If you do not have a telemetry logging implementation, I am pretty sure it is possible to do this with Shuffleboard; I will edit this post with instructions after looking into it (or if someone replies with instructions before I do so, I will add them).

Four tests must be performed to fully-characterize a single-speed drive - two for each direction, forwards and backwards (for shifting drives, each gear must be characterized separately, so eight tests would be needed):

First, participating teams should run quasi-static tests. This involves, simply, slowly ramping the robot velocity at a rate low enough that the robot’s acceleration is small. The exact rate of ramping is not crucial, so long as it is sufficiently-slow; we currently use .15 volts per second. It is not necessary that the robot reach maximum velocity, so do not worry if you cannot do so in the space available.

Secondly, participating teams should run step-voltage acceleration tests. In these tests, the robot is simply given a constant voltage. Again, the exact voltage is not crucial (we use 6v), so long as it is low enough to avoid slipping the wheels.

Tests should be run on carpet similar to the FRC field carpet, if possible. If it was not possible, please note the surface tests were run on in your post.

Data processing is not difficult. Firstly, data should be “trimmed” to exclude any data points at which the robot was not being commanded to do anything. Secondly, robot acceleration should be calculated from robot velocity and time. We have found it effective to do this by taking the slope of the secant line of velocity over a 60ms (3 standard loop iterations) window. Thirdly, data from the quasi-static test should be trimmed to exclude the initial period in which the robot is not moving due to static friction. Fourthly, data from the step-voltage acceleration tests must be trimmed to remove the initial “ramp-up” period that exists due to motor inductance; this can be done by simply removing all data points before maximum acceleration is reached.

Finally, the data can be analyzed: pool your trimmed data into four data sets - one for each side of the robot (left or right) and each direction (forwards or backwards). For each set, run a linear regression of voltage seen at the motor (or battery voltage if you do not have Talon SRXs) versus velocity and acceleration. Voltage should be in units of volts, velocity in units of feet per second, and acceleration in units of feet per second squared.

Each data pool will then yield three parameters - intercept, Kv (the regression coefficient of velocity), and Ka (the regression coefficient of acceleration). Please report those here, along with robot specifications.

I realize this has been wordy, and may be a lot to ask of teams that are undoubtedly busy during build season; but I think sharing of this data would be of great benefit to the community as a whole.

Accordingly, I will include the data from our test bed. The test bed is driven by a 6CIM drive using VexPro Ball Shifter gearboxes, geared 4.16:1 in high-gear and 9.01:1 in low-gear. The gearboxes direct-drive the front wheel shafts, with motion transferred to the other wheels via. 15mm HTD belts on 30T pulleys. The wheels are 4’’ diameter, 1.5’’-width Colson wheels. The tests were run on FRC-field-like carpet. The test bed weighed 135 lbs (including battery) during tests.

It is worth noting that due to an oversight in test bed design, the wire runs between the motor controllers and the motors were unusually long on this test bed, and so the motors may have seen lower voltages than those that were recorded; this could decrease the effective efficiency of the drive.

The resulting characterization parameters can be found here.

I have also included “theoretical” parameters (computed as described in the whitepaper), and “efficiency” values calculated from them (again as described in the whitepaper). Please include these values in your analysis.

Well, that turned out extremely long-winded. I do hope some other teams have time to run these tests and share their results. If there are any other values you’d like to see included, let me know.

1 Like

Recently, we characterized our drivetrain using the same quasi-static and step-voltage tests detailed in 449’s drive characterization paper.

We used 6.6v for our step voltage test, which ran for approximately 4 seconds (before almost colliding with school trophy case). Our implementation of the quasi-static test didn’t take a volt/sec input, just a generic “ramp-percent-voltage-per-loop” variable which allowed us to play around with the ramp speed until there was no slippage during acceleration. The quasi-static test had a timeout of 10 seconds.

Here is our implementation of a characterization command, which makes characterizing easy when coupled with Shuffleboard (uploads characterization .csvs to a flash drive, to be processed in R): https://gist.github.com/18melan/96d9bd43eacb6f8f5132b377dc26d37a

Here is what the processed data (after going through R) looked like: https://drive.google.com/open?id=1DDBQiZP8jYNTvpz1bfltgUDW2x1Q4WMr

At the moment, our test bed consists of a 4 CIM 2 mini-CIM 6 wheel 4+2 drive, driven from the rear wheels with custom single-speed gearboxes, geared at 7.69:1 for a top speed of 15.10FPS adjusted. The drivetrain uses Vex 6" traction wheels, and Talon SRX speed controllers with slaved Victor SPXs. Voltage compensation was enabled, and current limiting was set to 45A.

Surface was close to FRC field carpet, maybe a little bit harder. Test bed weighs around 60-70lbs with battery and extra add-on weights.

Here are our resulting characterization parameters.

At the moment, we have not characterized our drivetrain in reverse. The forwards parameters we determined have a reasonably low steady-state error from our initial testing.

We went through most of the procedure a week and some ago. Velocity worked out after a few mathematical pitfalls, acceleration was a bit more convoluted, and I didn’t complete that to the point I can say it was done correctly (close, but I’m not 100%).

I don’t claim to be the best mathematician, but I followed along with your paper and still had trouble reproducing your final answers, even with some outside consultation. I have a MSEE degree, and I’m guessing most HS level students would not find this abundantly clear.

I had planned on sharing the spreadsheet we used that shows our collected data, and step by step calculations. As I didn’t complete acceleration, I wasn’t quite ready to do so. My suggestion is ultimately that if you’d like to expand your supply of data, a more hands on guide from data collection to processing of that data in final form would be helpful. At the moment my team is focused on design tweaks and building, so it will be a while before I get back to reviewing our own data and publishing it here.

It’s a great idea and very helpful for anyone tuning a drivetrain though, and served us well.

You may find the R script we currently use to process our logs helpful.

I should probably get around to editing the drive characterization paper to describe the multi-variable regression approach in more than the footnote that’s currently there, as it is superior to (or at least simpler than) the “two single-variable regressions” approach.

That said, the linked script was written entirely by our head of programming, who is a high-school junior, and I have had success explaining the math behind the process to others. I’ll agree it’s certainly not trivial for a high-schooler, but it’s definitely not prohibitively difficult (2 or 3 hours of instruction ought to be enough for someone with knowledge of basic linear regression, which is a high-school level topic).

While there are other tasks that have priority during build season, we fully intend to release simple code (and a matching processing script) along with a step-by-step implementation guide at some point after build season to help teams through the process.

1 Like

For the multivariate linear regression, what kind of test would you run, or how do you combine the data from the quasi-static and step voltage tests?

You just trim the data as described and lump it all into one data set. You should record voltage, velocity, and acceleration (this can be calculated from velocity and time as described) throughout each test.

Are you only getting one acceleration data point for each step voltage test (just using the max acceleration)? Or are you including all the acceleration data after the max acceleration?

Nevermind, I get what you mean now

1 Like

Here are the results from a 6 cim, 6 wheel, west coast drivetrain:

https://gist.github.com/JayShower/eeda672aa3033b457da113aa625c85e0

Here is the command used to record the data:

https://gist.github.com/JayShower/dd62c4c71fbf644957566f2a0906b96e

The encoder velocity is measured through an encoder connected to the TalonSRXs with default velocity measurement window (64 measurements in the rolling average) and default velocity measurement period (measures the change in position over change in time between now and 100 milliseconds ago).

The motor voltage is measured through the TalonSRXs that the encoders are connected to with default voltage measurement window (32 measurements in the rolling average).

I am pretty sure the TalonSRXs take a measurement every 1 ms (for both voltage and velocity).

Acceleration is calculated by taking the change in velocity over change in time over a period of 60 milliseconds

Here is the data (I appended the step voltage test results onto the end of the quasi-static test results in order to do the multivariate regression).

Left Forward:
https://gist.github.com/JayShower/d281c5417c4061e9716cf05fa9d35bd5

Left Backward:
https://gist.github.com/JayShower/f7087b48a65c2ec5d6b04b2be51e7e6f

Right Forward:
https://gist.github.com/JayShower/de2d8ec8ab8d1df97d60a4f1e0c69fa2

Right Backward:
https://gist.github.com/JayShower/4c17a34b9748d1e6e401d3aff2898829

Here is the octave/matlab script used to calculate the results (I ran it in octave):
https://gist.github.com/JayShower/b8b14e12b6d3e0c3cac904c08d26f560

Thanks so much for sharing your data! Could you, perchance, share your drive’s weight, gearing, and wheel diameter so that theoretical values and efficiencies can be calculated?

Additionally, if you could share more details of the drive design (gearbox type/model, wheel type, chains/belts, etc) it might help to put the data in context better.

Additionally, I looked through your data a bit and it seems you didn’t quite trim the quasi-static data enough; there are still a number of points at which the robot wasn’t moving (visible as a “flat” portion at the start of the velocity plot). I’ve re-calculated your values with a more-appropriate trimming (I removed all velocities under 2 inches per second), and taken the liberty of putting them in a spreadsheet of a form similar to my own - I’ll fill in the theoretical values when you report the drive specs:

Thank you, I thought I only had to remove data where velocity is zero, but it makes sense that data where the robot is still getting up to speed needs to be removed too.

From a team member, here is the information on that robot. The tests were done in high gear.

weight: approx. 60 lbs (that’s a guess)
gearing: 5.67:1 (assuming this was in high gear)
wheel diameter: 4"
gearbox type: VexPro 3-CIM Ball Shifter
wheel type: Colson
Chains/Belts: Belts

Great! I’ve updated the spreadsheet, accordingly.

Your acceleration efficiency is quite low; you appear to have a lot of friction in your drive, or else your weight guess is lower than your actual robot weight. Your fairly-high intercept voltage suggests large frictional losses as well. Something to look into!

Thanks for letting us know!

Sorry for being sense, To be explicit, if I supply a non slipping, step-voltage of 6 volts to our drivetrain what am I exactly measuring? Finally how do I get to ka

Is there a thread that I can read up to learn about the step-voltage method of determining Ka?

On my list of things to do in the offseason, when we can actually unbag the robot.

In the mean time, we have detailed telemetry logs from both regional so far. PM me if you are just looking for more data to work with!

TL;DR: I’ve created a program that teams can use to easily run the test procedure here

For the better part of the last few weeks I’ve been working on an upgrade to the RobotPy simulator’s drivetrain physics based on the model in the drivetrain paper. The sim provides a way to create a drivetrain model based on theoretical parameters derived from your robot’s size/mass/etc, as well as measured parameters for kv and ka using the test methodology given here.

Most importantly for the purposes of this thread, I’ve created a NetworkTables based test method for implementing the test methodology here. There are two pieces: a test controller implemented as a python script you run on your laptop, and your robot program. For convenience, I’ve provided a python implementation of the needed robot code.

The test controller prompts you to set your robot up, you put the robot into autonomous mode, and the test controller commands the robot to do things and gathers the data. You disable your robot before it hits something, and then the test controller prompts you again. Once all the data has been collected, it writes the data out to a single JSON file, processes the data (including the trimming steps above), computes kv/ka, and shows you pretty graphs using matplotlib.

The great thing about this is that because all the core logic for the testing is on the laptop, implementing robot code that sends reliable telemetry back is trivial and requires no tricky logic implementation. You just need to implement an autonomous mode – the python version is ~20 lines of code.

Unfortunately, I rarely have a real robot or real carpet to test things on so I haven’t tested this on a real robot yet – but that’s one of the reasons that I built the robot simulator in RobotPy! The drivetrain characterization program has been tested extensively against my simulated robot, and it does a pretty good job of measuring kv. Unfortunately, the simulation of the model doesn’t account for current drop, so while I assume that the test script measures ka accurately, it’s possible there are bugs.

Drivetrain characterization test program

I welcome any feedback and fixes, and hope that this will make it easier for others to run this test procedure on their robot.

PS: If you’re in the Boston area and have real carpet and robot(s) to characterize, I’d love to stop by and help you experiment with this.

PS II: I tried combining the data and performing the multivariate linear regression as stated earlier in this thread, but it didn’t seem to work for me. If anyone knows how to do it properly in numpy I’d love to hear about it.

Awesome! Thanks so much for doing this. We’ve been planning to write a java version of this, ourselves, and haven’t gotten around to it yet. The easier we can make this to do for teams, the more data we can get (and the more teams can reap the benefits of having a well-characterized drive).

Thanks to Oblarg’s assistance, I added multivariate linear regression to the data analysis program. I also changed the plotting a bit and added better error handling.

Something that occurred to me is perhaps it would make sense to create a separate repo / project for this. The data logger / analyzer could be distributed as a pypi package (so it’d be really easy for anyone to install and run), and the repo would contain robot code for the various languages and motor controller combinations.