Data Logging Through Driverstation

[OwenDotCode]

So my team has been wanting to collect more data than the driver station lets us. Ideally we don't want to be streaming this data over the network. We also need the data to not be lost if we lose power or brownout. Anyone have any ideas.

So far we have thought of:
USB drive in roborio
Raspberry Pi over Ethernet
Raspberry Pi over serial(done similarly to this (https://files.slack.com/files-pri/T34QRB68K-F3FL6BL57/a_portable_serial_data_logger__2_.pdf))

[kenfox]

You have plenty of capacity for logging to a file on the roboRIO. Just scp the match data off the robot when you get back to the pit.

The only thing you have to take care of is use a background thread for sync'ing data to disk. If you try to do that in the main robot thread, it will occasionally pause on you for much longer than a driver station event cycle.

[phurley67]

What language? You can find plenty of good examples. We have borrowed from a number of teams (thanks 254) with our base code for next year (java). See https://github.com/frc-862/valkyrie/ Check out the DataLogger, and, LogWriter, for a reasonable setup.

We log to a USB key, so as to be sure we never fill up the RIO. But if you are careful there is no risk of filling up the RIO in one or two matches.

[OwenDotCode]

We have considered both of those, but our issue if will those still work if the robot loses power?

[OwenDotCode]

Also we are using Java

[Dwight_2]

What language? You can find plenty of good examples. We have borrowed from a number of teams (thanks 254) with our base code for next year (java). See https://github.com/frc-862/valkyrie/ Check out the DataLogger, and, LogWriter, for a reasonable setup.

We log to a USB key, so as to be sure we never fill up the RIO. But if you are careful there is no risk of filling up the RIO in one or two matches.

Your link is not working I don't know if you posted it wrong or some but my team is looking for a similar code to log the total distance our T-shirt cannon has traveled using encoders.

[bobbysq]

We have considered both of those, but our issue if will those still work if the robot loses power?

If you log to something like a CSV as the data is collected, power loss shouldn't affect it since it'll only add one line at a time and it's a plain text file.

[phurley67]

Try it now (sorry about that, the repo was not supposed to be private).

[OwenDotCode]

Quote:
Originally Posted by OwenDotCode
We have considered both of those, but our issue if will those still work if the robot loses power?
If you log to something like a CSV as the data is collected, power loss shouldn't affect it since it'll only add one line at a time and it's a plain text file.
So would the CSV file be saved every time another line is added?

Thanks for the like to the Valkyrie, I will talk to my team about it at our next meeting.

[phurley67]

Our code will create a new file (with a new header record) every time the robot code restarts -- if it due to power loss, redeploy, or restart. It is pretty trivial to join a couple csv files either using bash script or in an editor.

Right now we are using sequential numbering, as we did all our off season testing using the qdriverstation and the date/time on the rio was not getting set. Probably going to change that, but it works as it (plus if the date is set the file's timestamps will be accurate).

[Greg McKaskle]

There are a number of good suggestions in this thread, but let me add a few points.

The OS and other utilities log things to the roboRIO flash all the time. A team is running with user privileges It should be pretty easy to look at your log file design, estimate usage, and determine whether a USB stick or RIO flash is the right approach. Also, if needed, you may want to disable buffering to on file open to better deal with power loss.

If you find that your CSV file is too large, binary files are often much smaller, typically a factor of five to ten.

Depending on the rate you want to log at, you may also want to consider network table logging. This logs the data at more like 10 Hz, but it automatically logs every variable update and it is built into the default DB. It is possible to build a custom DB and log faster if you wish.

I highly encourage you to log data, though. I love it when teams have data to understand what their robot is doing.

Greg McKaskle