Old process not killed

[truher]

Our deploy process now often results in two copies of the java process on the RoboRIO. I dimly recall other folks complaining about this same issue awhile ago, but I can’t find those threads. Does anyone remember those cases?

[truher]

One weird clue, the FRC_UserProgram.pid file is missing, so the frcKillRobot.sh script never kills it. I would expect subsequent deploys to somehow recover from that state, e.g. the “kill -9” route, but it seems not to.

We’re running the dev version, if that matters.

[Joe_Ross]

@truher

We’re running the dev version, if that matters.

Can you clarify what you mean by “running the dev version”? I can think of a few different things you might mean.

WPILib is in the process of making breaking changes for the new season and things aren’t always in sync across all repositories and tools. In addition, vendors haven’t made compatible updates. I very much expect things to be broken right now, but not sure that I’d expect it to exhibit what you’re seeing.

[truher]

@Joe_Ross

Can you clarify what you mean by “running the dev version”? I can think of a few different things you might mean.

sorry, i meant this:

https://github.com/wpilibsuite/allwpilib/blob/main/DevelopmentBuilds.md

is there another kind of “dev version”?

We use this version to avoid the disruption of library porting at the start of the build season.

@Joe_Ross

…not sure that I’d expect it to exhibit what you’re seeing.

Right, I’m not sure how the contents of the jar would somehow break the roboRIO end of the deploy process in this way. A student noticed today that the new process seems unable to “kill -9” the old one (the output from this line never appears); maybe that’s a clue for folks more knowledgable about how all this works than i am. Interesting to me, the cpp referenced here uses a different “pid file” than the frcKillRobot.sh script, which is kinda weird to me, but whatever.

I found a few threads that mentioned replacing lvrt with a shell script. we’ll try that, since our actual requirements seem a lot simpler than the code here seems to support.

https://gist.github.com/PeterJohnson/2702b331ee3c236188ec76ecf499c333

Lvrt process causing CPU spikes

Title says most of it… we’ve lately seen the lvrt process eating about 30-40% of our CPU and it causes quite a bit of latency and lag on the robot, resulting in control issues. Not sure what is causing it but I’m curious if others have seen this and what might be causing it.

Fatal Robot Code Crash

Just out of curiosity, do you know how much memory the JVM should have available on a RIO 1. I don’t know how accurate this command is Logger.getInstance().recordOutput("Memory Usage", (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) / 1024.0 / 1024.0); But I saw the code crash when it reached around 9 mb. EDIT: I’ve found it apparently to 64 mb from this line, # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this scr…

[anon94582938]

Our team was experiencing those issues, we followed these steps to temporarily fix it. It wasn’t too much of an issue because it only happened on deploy.

1. SSH into the RoboRIO (ssh admin@10.[first 2 digits of team number].[last 2 digits of team number].2)
2. perform the command ps -ef to list the processes running
3. Find the java process that was stalling out
4. kill -9 the process

[truher]

@anon94582938

Our team was experiencing those issues, …

Yeah, we did that too. Did the problem go away for you? Or are you still doing this on every deploy?

[anon94582938]

Still doing it on every deploy, it’s been an issue for the past 6 months or so and we haven’t found a fix online yet.

[truher]

ohh noooooo!

[Thad_House]

Generally, issues like this are caused by issues in team code. If you’re using custom threading anywhere, or not handling InterruptedExceptions correctly, it can cause this behavior. If you’re not using either of them, check to see if your vendors are.

[truher]

@Thad_House

Generally, issues like this are caused by issues in team code.

Could you say more about that? We’re not doing anything with threads, etc, and not using any vendor code that everybody else doesn’t also use. How would a threading issue result in the erasure of the pid file so that the process killer can’t find the process to kill?