Python code runtime


#1

My team has been working on a command-based Python project for our 2018 robot this offseason. Our project has four subsystems, about 20 commands, five sensors, and five PID Controllers. Recently, we realized that our code is actually not running fast enough on the RIO, dipping below 50 Hz to ~45 Hz. We were initially getting even slower runtimes (~15 Hz), but were able to reduce sensor calls and Smart Dashboard updates. Is this normal for a Python project of this size? What about other languages? Are there any specific practices we can do to reduce our runtime? Thanks in advance!


#2

I gotta say, the thread title initially confused me; “runtime” usually refers to a some software that supports the execution of a program.

A 45Hz control loop is probably not a very large performance loss (although ~15Hz certainly is!).

My team also had some pretty severe performance issues during last build season (although it only occurred to the team to investigate performance being the source of our problems after Dustin packaged up the loop timing code I had originally written to time TimedRobot).

We had 11 Talon SRXs on our robot last season, and managed to trace down the cause to some inefficient code in the robotpy-ctre WPI_TalonSRX.set() method (which we since fixed during the season, luckily).

I suspect most performance issues that RobotPy teams face are due to how threading works in CPython (not going to go into the technical details here, but read about the GIL if you’re interested). Reading from a sensor in itself shouldn’t be expensive.

Now, as it turns out, the WPILib PIDController of seasons past runs in a thread. NetworkTables also uses threading. So in your specific scenario, your robot code would be using at least 7 threads (counting the main thread). Python on the roboRIO is probably having quite a fun time switching between your different threads, especially if your PIDControllers have a low period…

You could try utilising the WPILib PIDController currently aimed to land for the 2020 season. I’ve backported that here: https://pypi.org/project/wpilib-controller/.

If you’re up for getting a bit dirty, you might consider profiling your robot code using a profiler such as py-spy: https://github.com/benfred/py-spy. Without more concrete data, almost everything in this thread is basically conjecture.


#3

I experienced the exact same issue. Running my code through cProfile showed that the major culprits were a) PDP status calls, b) Talon SRX read calls, and c) LiveWindow calls. I believe © has been fixed. I haven’t had the time to investigate more thoroughly though.

https://gitter.im/robotpy/robotpy-wpilib?at=5afbaf15e1cf621dba221eac


#4

This should also be fixed in the 2019 season. (Currently in the process of updating RobotPy for the 2019 changes.)


#5

I’d echo everything that auscompgeek said.

The PDP in particular causes lots of delays (in all languages, not just RobotPy), but that should be solved for 2019.

One thing to look at is the CPU usage on the RoboRIO. If it’s at 100%, you’re definitely going to run into problems.

One thing that isn’t obvious is that running the sample cscore processing code (that doesn’t do anything!) at 30fps will cause the CPU usage to go through the roof. Don’t do that. However, doing processing at a lower frame rate is fine – but doing image processing on the RoboRIO always requires a fair bit of optimization, especially in Python.

Running CameraServer without processing is very efficient and shouldn’t cause more than 5% overhead, but this is true in all languages.

The loop timer is really useful for measuring execution time. Getting some profile data would be great… I think the answer is that historically, teams have been able to do some fairly complex things with RobotPy. However, it’s totally possible that there’s just something weird we’re doing somewhere that would cause things to be really slow. If you can isolate those things via profiling or whatever, that would be extremely useful.

I don’t really like using the WPILib PIDController in python (because of the extra thread), but try to use TalonSRX instead.