Exactly what it says on the tin. I’d like to get some context on how heavily-used this feature is, because it’s old and not very well-documented.
0 voters
If you answer “yes,” please respond below with details on how you used it (which features were helpful? which weren’t?).
In 6 years I don’t think I’ve ever used it.
This year we wanted to use test mode as a way to test robot functionality in the pits, sort of like an alternate teleop mode. Unfortunately, we were unsuccessful because of limitations with WPILib. Specifically, the CommandScheduler will never schedule commands or run subsystem periodic()s if test mode is selected, even when you are enabled on the FRC driver station.
Perhaps the question should be rephrased to include an option for “tried to use test mode, but couldn’t get it to work”?
For more context, see: this conversation on the FRC Discord.
We used it for “unit testing” - aka initial testing to ensure wiring, motors, and pneumatics were working. Worked quite well for that, but the Rev Hardware Client would work just as well, to be honest.
Two things are being conflated here: test mode and LiveWindow control:
Test mode is the DS mode, and doesn’t have any functionality of its own.
LiveWindow control is an old functionality in WPILib that is intended to allow controlling actuators from the dashboard regardless of code.
Historically, LW control is active in the DS test mode. It could be disabled for a few years, but there wasn’t away to completely prevent it from starting, until 2023 (call enableLiveWindowInTest(false) from robotInit). To avoid conflicts, the command scheduler is disabled while LW control is enabled.
So, in short, I assume this is asking about LiveWindow, and not test mode itself.
We used it for small bits of debugging, when we purposely only wanted to run one subsystem without invoking any of the commands that usually run it. It took a few iterations to get the hang of it (specifically that you have to manually call periodic() on any subsystem you do want to test), but we did find it useful.
One specific use case was tuning a PID on a pivoting arm. We were able to directly just run the pivot motor with an ArbFeedforward calculated at each iteration, so we could tune the PID by itself. I don’t think this would have really been doable in REV client.
We use it for subsystem level testing. We do not use LiveWindow. Instead we add something like this to each subsystem:
void testModePeriodic() {
if (subsystemTestEnable) {
periodic();
} else {
safe();
}
}
Then add some hooks to the subsystem in network tables. This would be similar to using LiveWindow and Sendable interface, except it calls the periodic function of the whole subsystem since the command scheduler is disabled in live window mode.
Edit: Calling enableLiveWindowInTest(bool) would allow the command scheduler to stay running in test mode. Might investigate, though it would mean no per-subsystem enable without extra logic in the periodic() function.
We use it to test single mechanisms to isolate problems and use easier control schemes. It’s a great feature for initial testing so teleop doesn’t get polluted.
We use it and it’s massively important to us! We use it with Shuffleboard from C++ and depend on it for testing, bringup, and tuning of all of our H/W. We have a whole lot of code that runs only in test mode, allowing us to streamline regular auto/teleop code. This code needs test mode as a hook, although we could just as well add some sort of logic to recreate it as a special case of another enabled mode (via some sort of switch) if it came to it.
Yes test mode.
No to LiveWindow.
LiveWindow is swapped with custom code to inject test waveforms into our closed loop controllers to facilitate manual tuning.
At some point in the future, test mode will inject whatever sysid needs to make pretty plots and good guesses at gains. But today is not that day.
Personal opinion: I like the concept of LiveWindow, but robots today are too complex for a one-size-fits-all-solution to be applicable.
I think a broader question is “what should test mode be designed to do in WPILib?”
As mentioned in other posts, historically the WPILib approach for test mode was LiveWindow–a dashboard-based mechanism to enable teams to test motors and sensors (e.g. verify wiring) without the normal teleop running. The utility of this feature has declined in recent years with the wide use of CAN motor controllers and their vendor-provided utilities to set up and test motor controllers. Sensors generally can be observed/checked in teleop without the need for a separate test mode.
So if we remove LiveWindow entirely, what should WPILib do in test mode instead (at least as a default)? As gerthworm suggests, maybe an integrated sysid (if we have the tools/implementation to make that viable). Or do we just have it as an alternate teleop mode (which feels kind of unsatisfying to me)? Other ideas–maybe some kind of framework for automated hardware-in-the-loop functional testing (e.g. an alternate auto)?
Many use cases for test mode shouldn’t have the scheduler running, so I think that should continue to be the default, maybe with a commented-out line in the template showing how to enable it in test mode if that’s desired by teams that want to change the default operation (whatever that ends up being).
Last year we disabled Live Window and created a method in Robot Container called getTestCommand(), we did this similarly to how Autos work.
Just used it to zero our shooter hood and climber which came in handy during practice and tuning. Also an easy way to charge air without having to use teleop which had other code that automatically ran that we didn’t want to run just to charge air.
One implementation I hoped for in the past was providing a mode to test hardware in the pits. IE check for broken sensors/wires and other issues. Would be nice to be able to bind some sort of test method or command to each subsystem to do sanity checks on sensors and motors. There has been more than a number of times where our team did not notice we ripped out a solenoid wire to find out we could not control a critical mechanism the following match.
That’s exactly what LiveWindow was designed to do (e.g. manual controls to test individual sensors/motors/solenoids independent of other software automation you might be running in normal teleop). So maybe we just need to beef up what LiveWindow is and how to use it so it’s more discoverable?
Maybe… If there was a way to setup “testing checklist” (Inspired by the labview dashboard checklist thing) in addition to the current live view capabilities to have the ability to automate some basic tests as well as wait/prompt for Manuel ones.
Probably a pipe dream but this is kind of what I am thinking. Default behavior with no custom code. provide a simple dashboard that highlights sensor values and motors. (IE highlight a sensor green and stay green if a state change was detected)
While enabling users to add a lists of tests to go though and prompt. IE prompt the user to press limits 1 and so on. Or allow them to write custom tests IE make sure encoder 1 and 2 move in the same direction (Connected gearbox)
Think a framework that would easily allow teams to do that would be great for a pre match checklist type thing. Teams could even add tests to check the battery voltage and remind them to switch out batteries before a match.
I think one of the problems with this kind of a tool, while very convenient in concept, is that maintaining it is nearly impossible given the breadth of sensors/motors that need to be debugged.
The WPILib Sendable implementation has been troublesome for years largely because it has been pulling double-duty as a telemetry API and the backing implementation for LiveWindow (which attempts to do what you’ve described). The latter requirement drove a lot of “magic” global state dependencies and other stuff going on in the background that eventually became a performance liability, in addition to being somewhat brittle and hard-to-use.
For another immediate example, look at how many problems teams have had with SysId this year. The vendor integrations have been an ongoing nightmare since the frc-characterization days.
No-user-code solutions are very hard to do unless you’re in a very constrained problem-space, and we might be better off delegating individual motor/sensor debugging to vendor hardware clients and refocusing test mode towards integration testing of user code.
Yeah I agree with the no user code part. The bit that would be most interesting to me though would be user written tests that could be run though/prompted via test mode. Heck maybe still run telop but still have tests run that check for state. That being said as nice as that would be to have built it I don’t know how much it would help newer teams where as some of the bigger teams if they wanted something like that they could implement it relatively easily.
In my mind there are two different potential use cases for the test mode. Using it for software development/testing or using it as a tool to test a robot is ready for a match/hardware is working.
These both seems like things that we could leverage parts of the existing Command-based framework for, I think. Perhaps the move here is to have an alternative test-mode scheduler that offers more testing hooks?
That would be nice, that would enable you to run your telop code if you wanted at the same time as your test code. Then for subsystems you could add test commands to be run. Although are commands able to return any type of status or success state?? I will say one downside of using the command framework is depending on how you have your code structured. (Talking from java) Most of the time individual motor controllers are private inside of a subsystem so unless you defined the test commands inside of the subsystem you would not have access to the motors without changing the visibility or adding a way interact with the subsystem in the way you desire for the test. (Thing I am thinking of is running a single motor in multi motor gearbox)
There’s nothing fundamentally wrong with making your hardware dependencies package-private or public to aid testability - this is a reasonable informed decision in a lot of cases. Just make sure your students know the dangers of abusing it.