Data Logging

FIRST General Forum
#1

I’ve been interested in data logging on the competition robot for a long time. It’s been one of those things where we’ve repeatedly “wouldn’t it be nice if…” but we’ve never gone back and spent the time to make it happen.

The one or two times we’ve looked at doing something the kids implemented something ‘quick and dirty’ that caused us problems down the road, due to memory consumption, or slow loops. So they were quickly put on the shelf.

Are there some best practices we should be looking at? For instance:

On one occasion, the kids tried logging by storing information in arrays to write to a file at the end of a match. However, we watched the memory usage and cpu usage increase throughout the match and it began to effect the performance of the Rio - so that effort was set aside.

On another occasion, the kids tried to send all the global variables over network tables and then log them on the driver station computer. But the network table default update rate meant they didn’t get the data all the data they wanted.

So - what’s the best practice? I’d imagine that writing the data to a file on the Rio is optimal. But should the data be written every loop? Should it be stored and periodically written? How much of an impact would that have the Rio performance?

I’m sure other people have already gone way down this rabbit hole, but I’d like to give the team members some direction on what they are doing. Their ‘end’ goal is to do all this in LabVIEW, and then write a playback program that they can ‘watch’ the telemtry during a match to determine what was going on and where things went wrong.

#2

The paint is still wet, but you should definitely take a look at WPILIB’s On-Robot Telemetry Recording Into Data Logs

1 Like
#3

Right, thank you. But we’re a LabVIEW team, which is why I asked specifically about strategies we could apply. I’m neither proficient nor competent at digging through Javacode (indeed, the high school doesn’t even offer what I’d consider competent programming classes).

With strategies, we can apply it to the code we already know.

I’ve started going through that document you linked and it does have some interesting things I think we can apply.

Thanks!

#4

The design constraints for LabView are going to be similar to those for the Java code - you’re going to want to write logs in a binary format (string manipulation is sloooooooooooow) and you’ll want to offload the logging to a separate thread with a lower priority so that it doesn’t compromise any timing-critical code. This means dealing with thread-safety, which might be easier in LabView than in Java/C++ due to how inherently asynchronous it is, though I don’t know for sure.

Even if you don’t have great programming classes, you might benefit from switching to Java. LabView poses a lot of logistical challenges to teams; you can’t version control it effectively and sharing/viewing code requires a full workstation with proprietary software installed. There’s also a lot less community support available, simply because so few teams use LabView these days…

#5

Sorry, I missed the reference to LabVIEW in your last paragraph. (Consider using categories when posting.) I don’t think I can be much help to you here.

1 Like
#6

Oblarg, can you explain a little more about what you mean when you say that string manipulation is slow? My first thought would be to write everything to a .csv file. But are you suggesting that’s a bad idea?

#7

Yes, I’m suggesting that’s a bad idea, at least in a real-time environment. Strings are a very inefficient way to store data (CSV even moreso!), and string concatenation is typically extremely slow (it’s a massive array op).

#8

Thanks! Good point. I added a LabVIEW tag to the title. I specifically didn’t want it binned to the labview category though because I really wanted thoughts from people in all languages.

#9

What would the best strategy here be? Store the data in arrays every xx ms, and then write chunks of that array to disk at longer time frames, like once a second, and remove that portion of the array from memory?

#10

Yeah, that’s about the gist of it.

#11

You can do this even easier with LabVIEW, since it has so many nice built-in features for parallelization. I would use a queue, or a real-time queue, where any parts of your code can write to the queue, and you have a single while loop that simply waits for data from the queue, and writes it to a file. You can then set the real-time priority of that loop so it never runs ahead of your robot subsystems.

Now on top of that, if someone is willing to implement the DataLog format in LabVIEW, you can write to that format which would allow you to use whatever tools end up being created for next season.

4 Likes
#12

I’m definitely going to fall short on the “best practice in Labview” stuff. But, confirming, have you looked into the Read/Write from measurement file vi’s?

I’d imagine a workflow where the robot code writes data to an NI-defined measurement file on the RIO side. Then, you’d transfer the files to a local PC (somehow) and have a separate labview program to read and display the contents?

There’s lots of other valid architectures here, but I’d suspect/hope labview’s built-in logging functionality is efficient enough for robot use. If it’s not… I’d say it’s unlikely you’d be able to improve on it yourself through pure labview. That’s just a guess though.

A sorta-nuclear option: Build the C++ code associated with some of WPILib’s logging, and invoke it from labview: https://www.ni.com/pdf/manuals/370109a.pdf

1 Like
#13

The cheapest option is to add the occasional message (print statement) to the code. We used this to output the shooting setpoints and actual values for each shot. This is reasonable to identifying that a system was working or not, but not much else.

You might also look at TDMS file logging. Something like
examples\File IO\TDMS\Advanced Read and Write\Synchronous Read and Write\TDMS Advanced Synchronous Write.vi

Many of the asynchronous and memory feature are not available on RT, so the API is not as complicated as it seems. You can also try the Express version examples\File IO\TDMS\Express Read and Write\TDMS Express Write Data (Time Domain).vi

The viewer for the data is built in and it is pretty efficient.

Another option that I’ve played with in the past is to speed up the Network Tables rate to the dashboard. You can then use the record feature of the dashboard to journal the NT updates to file. The good portion of this is that you can view it synchronized to video. The downside is that the NT updates are somewhat difficult to pull apart and get the data for other viewers. And changes to the dashboard tends to invalidate the ability to review log files. But it is slick to see things synchronized

#14

I have a question for you about this. We tried continuously logging to a TDMS file using an NI reference, and found that often times the tooling to read the file would say that the file was corrupt. Is the TDMS file format immune from sudden power loss or similar that we see in the robotics environment? Are there other common reasons that the file may end up corrupted?

Admittedly this was 2016, so it is entirely possible that there were issues with the thumb drive, or maybe libraries have been updated since then etc.

#15

I don’t have first-hand experience, but haven’t heard stories like that from customers. I hope to experiment with it some in off-season, but haven’t gotten there yet.

1 Like
#16

I’d have to say that’s been the assumption for years, but it all depends on how you implement it.

Is the string form more bytes in length? Is there more work converting native types to string?

A lot of the world’s APIs just use json these days.

Is there an embedded TSDB that could be used?

#17

Yes and yes.

Take for example an 8 bit integer with value 255. You can log that value to a file with a single byte, while a string conversion would take up 3 bytes. When you start doing that with floating point values it grows incredibly large.

JSON works fine for most things but it really sucks for real time stuff.

1 Like
#18

ya. that was the point i was trying to make. Exactly how “real time” do you think you need to get with a FRC robot? I mean considering you might randomly hit a GC pause. Its probably ok to be “soft” real-time.

#19

It isn’t just the real time portion. The higher the CPU the less deterministic it gets. I know in our case, the difference between storing large groups of values in arrays in preparation for logging versus NOT storing them resulted in a measurable CPU hit. That plus running a camera through the RIO was a non starter. Anything that reduces the CPU usage for us is a good thing.

1 Like
#20

Floating point text output is several orders of magnitude slower than just copying 4 or 8 bytes, as you need a loop and division at each step, plus a lot of conditionals to handle various output cases like sign, exponents, etc. This really starts to matter (eg adds up to measurable delay and missing loop time requirements) if you’re logging hundreds of values.

Also, JSON is a really bad choice for logging. The biggest issue with it (other than being text) is that it requires closing delimiters for a valid file construction. This makes it not very friendly to the robot use case where you can have unexpected power-offs and there’s no good trigger point to “close” the log file.

2 Likes