For a while, we’ve been facing the occasional crash due to an out-of-memory error. Previously, it happened once in a blue moon, no big deal. However, in the past 24 hours, it’s gone from crashing after 30 minutes, to crashing after 5, to now crashing after less than 5 seconds after the robot code starts.
Here’s a link to our code.
In order to help debug, I looked at all our periodically running functions. All of them are either logging or SmartDashboard calls. Removing them changes nothing. No heap objects (new) are created in any periodic functions.
Checking with top, the RoboRIO’s free memory steadily climbs from ~30MB free, decreasing by 4MB every 2-3 seconds, until it reaches 0 and it crashes. Per another thread we disabled the web server to no avail.
We also tried to run VisualVM on it, but were unsuccessful as it crashed before we could connect. Instead, I tried running VisualVM on a simulation, and what I found is that HashMap$Node is being allocated at a crazy high rate.
Just curious. Do you have a Rio 1 or Rio 2? We had a memory issue with the Rio 1 crashing continuously and updated to the Rio 2 and have had zero problems since.
We had a similar issue and had to update to the Rio 2. Everything is working great now. This issue didn’t start happening until this year. I don’t know exactly what changed, but our memory would slowly get eaten up until a crash occurred.
The default this year is serial GC. I think it was the sysctls? But clarity on which one(s) helped would be helpful so we can document it (and even do a patch release which does it for teams for this year, and fix it in the image next year).