I would like to use the simulation facilities in WPILib to help my team develop largely without the use of the robot. The first goal is to allow code developed to go essentially unchanged onto the robot itself. I see many quick examples of simulation and read the docs. Either I am having trouble orienting myself or there is a misalignment with my intent and the present state of WPILib.
I would expect some configuration to cause the HAL to direct calls to synthetic elements instead of actual device drivers. There would be some central simulated robot state so these synthetic elements could give coherent responses. It would be nice if the simulation state also captured physics and environment as well. I would anticipate this requiring explicit HAL use for device access. This would extend to team-developed drivers for devices through MXP and CAN.
Goals (in order of increasing ambition):
Whole system development without the robot
Expectation unit testing where mock/simulated elements can be queried. e.g. The robot transitioned from State1 to State2, did the motors run?
Vision system development where the synthetic robot orientation drives the camera/viewport of a 3d model of the field.
Pre-integration collaboration where subteams exchange mocks of their system before developing the actual versions.
Analysis-of-alternatives investigation where different hardware can be evaluated in simulation
I’m also interested in this topic, and like you, I expected that the HAL would be the key element: unmodified robot-domain code on one side, physics-domain simulations on the other side. It seems like the most natural thing to do with WPILib is not that, though: instead both the “robot” and “physics” domains are mixed together on the “robot” side. I found it most natural for simulation code to wind up everywhere (example: my swerve sim thing.
I think there are good reasons why it works this way – beginner programmers have just one thing to learn, it works the same in java and c++ – but i think it could also work the way you and i expect. If anyone is working in that direction, I’d be interested in learning more about it, maybe helping out if that would be useful.
Splitting at the HAL is the approach I have taken in the past on professional robotics projects. I also noticed that is the approach in the Autodesk Synthesis code but I have not had the time to get that up and running yet.
The WPILib HAL comes in two forms: there’s a HAL for the RoboRIO that maps HAL function calls to RoboRIO hardware, and there’s a simulation HAL that has data structures for everything in the HAL and provides a complete set of simulator-facing functions. The simulation HAL is used any time you run robot code on the desktop.
In addition to the RoboRIO actual hardware functions, the HAL also supports simulation-focused virtual devices (SimDevice) that provide a way to access higher level functionality in simulation–for example, rather than needing to mock the SPI layer of gyros, a Gyro class in WPILib creates a SimDevice in simulation that just provides direct access to the gyro rate and angle.
The simulation HAL also supports plugins, so e.g. the simulation GUI is a plugin that displays a GUI and uses the simulation HAL functions to change HAL values seen or set by the robot program. There’s also a WebSockets plugin so you can use any language to talk to the simulated HAL.
The simulation HAL functionality is exposed via both the low level HAL functions and higher-level WPILib “Sim” classes.
Code construction wise, yes, the typical approach is to write code into simulationPeriodic() functions that is the sim analogue of the “normal” robotPeriodic(), but that’s certainly not the only way to do it. You could write separate classes, or even run a completely separate application and talk to the simulation HAL via websockets instead.
You can find more information in the WPILib docs under “Robot Simulation” regarding both physics simulation and unit testing.
I’m also interested in this. I’m new to mentoring FRC coding this year and less experienced with robotics-specific coding and with WPIlib, but this would be similar to other setups I’ve done on projects with third-party services that perform a portion of the work, and makes sense to me from a design perspective. Our team is working in Java, and I’d be happy to help with an effort in this direction if someone can provide support and review on the nuts and bolts in the robotics libraries.
The real hardware vs simulation distinction is done by what platform the code is being run on / compiled for. Hardware calls go to the real HAL on the RIO, and to the simulation HAL (ie HALSIM) on other platforms. The library handles all this under the hood, and exposes functions such as RobotBase.isSimulation() to check and methods that are run only in sim such as simulationPeriodic() for simulation-specific user code.
Physics simulation classes such as ElevatorSim use physical constants to simulate mechanism responses to applied voltage. These constants can be derived from CAD, determined using SysID (if the robot is already built), or set arbitrarily to reasonable values (the latter will require retuning of other constants once running on the real robot, as the simulation physics model wouldn’t be accurate).
The responses from the physics simulation can be fed into hardware simulation classes that provide a way to set sensor values etc.
Hardware simulation objects can also be used in unit tests to query hardware state without adding a ton of getters to the tested code.
The rationale for having simulation-specific code in the same source set as “regular” code is that often said simulation-specific code needs access to subsystem internals and such.
As Peter said above, the simulation split is at the HAL layer. If you wanted to handle the HAL simulation layer completely separately, that is something that would work perfectly fine today. Its how the sim gui plugins work, but you don’t have to write it as a plugin to properly work. You could always just fire up another thread that only talks sim calls.
In general, the choice for the documentation and examples to combine the sim side and robot code side is out of convenience. Completely separating everything is complex, and while there are cases where it could be better, generally for most teams it seems like mixing the systems works well enough.
Ah, Peter reminded me of another factor that I think dominates my thinking on this topic, which is that the students I work with are very much beginner programmers, so everything I do with them is optimized to be simple, making use of the plentiful examples in WPILib. So the “typical approach” is good for that reason.
So I think if we were going to do anything else (e.g . the websockets approach), it would end up to be “mentor-ware”. I’ve ruminated on that too: the field elements are mentor-built, so maybe the physics model could also be mentor-built, and deliver the value of simulation to the students without embroiling them in the complexities of supporting it. But I dunno, I also really like the students to be able to understand everything that they touch, all the way to the metal. And i think there are real issues there – how does closed-source CAN-based stuff work with the websockets approach?
Anyway, still interested in exploring a more segmented approach, but i don’t think it’s easy. Maybe an off-season activity for summer 2023.
CAN motor controllers use SimDevice to expose higher level functionality in simulation, and SimDevice is exposed via WebSockets as well. Different motor controller vendors have different levels of simulation support, however. You can easily see what’s exposed in SimDevice in the “Other Devices” window of the simulation GUI.
I think this is part of what has confused me so far. I entirely understand providing an escape hatch for the robot code to change function when running in simulation but the docs/examples would lead me to believe that it is the primary mode of operation.
It would help me a lot if there was a simple, but not empty, example that includes regular robot code that is largely unaware that it is running on real or simulated code and then a separate section of code that handles simulation. For my purposes, I would like that in C++ but I can handle Java as long as it aligns with the C++ calls. I appreciate that the examples are focused on single topics for when I am looking for a specific topic. However, it feels like there are no comprehensive examples that demonstrate making all these things work together.
I’ll throw out another approach to simulation that 6328 has found to work well for us. We structure our subsystems to isolate hardware interaction to an “IO layer” with multiple implementations. This allows us to swap out real hardware for the WPILib physics sim classes while bypassing vendor libraries (or write multiple “real” implementations for robots with different hardware). The main control logic doesn’t need to know which IO implementation is being used. This structure also interfaces with our AdvantageKit logging framework to enable more advanced features, but we started using IO layers for simulation before AdvantageKit existed. More details here:
I’ll also say that our team has also switched to AdvantageKit and found that it really does make setting up simulation a lot easier than before, since it’s as easy as just adding another IO layer. Another advantage of using AdvantageKit is that 6328 has made an amazing viewer application (Advantage Scope, although it doesn’t require the use of AdvantageKit).
Yes, fundamentally the HAL and our IO layers have similar goals. However, we’ve found it easier to deal with small interfaces for each subsystem that we have full control over, and it clearly enforces a separation in our code between control logic and “hardware” (though this isn’t impossible with the HAL approach). The IO layers are also more flexible when we need more than one “real” or “sim” implementation. Ultimately either approach is reasonable, so it mostly comes down to personal preference.
(Also we prefer not relying on vendors to provide robust simulation support.)
The main idea is the same as @jonahb55 talked about in their post, although I think we did the implementation a bit different. We ended up using multiple gradle modules. One module had code that ONLY ran on the robot (wpi), which imported WPILib code, one module had the core of our code that ran in both the simulation and on the robot (core), and one module was only for the simulation (gdx). We used lots of dependency inversion since the wpi module imported the core module and the gdx module also imported the core module.
In my opinion, the end result was a lot cleaner than global simulation state and none of the simulation code had references to things like MotorControllers. However, I was the only person on the team that understood what the code was doing and it’s really hard to teach new students to create an interface (inside core), then create two separate implementations of said interface (in wpi and gdx). It was an elegant design, but teaching it was difficult.
I personally don’t really like the simulation approach that WPILib takes, because it seems to combine simulation code and non-simulation code side by side with if statements, but I don’t have a better solution to recommend that’s easy to teach. Since I couldn’t teach it and there wasn’t documentation for it, we didn’t use my simulation stuff in 2022 since college was keeping me busy.
I think in 2020 our code hit on some of your points. We simulated vision data in a very similar manner that the robot reported data to us and since there wasn’t global state everywhere, many unit tests were possible.
One thing I find useful is to break down “Simulation Facilities” into its components. This should in turn help understand the best way to achieve the goals.
Starting with the idea that “Simulation” equals “get robot-like functionality without a robot”, there’s a few steps involved:
Cross-compiling
Since the robot isn’t present, presumably, you need the code you wrote to be built so it can run on a different CPU architecture.
WPILib fully supports this.
IO Stubbing
Presumably, the robot-specific IO is not present. This means the code, at some point, will reference hardware which doesn’t exist, and break in one way or another.
A minimum viable product involves “stubbing out” all IO related functionality, so it simply returns “0” or “happy”. This will allow the code to at least start to execute on the alternate target.
This step is often referred to as choosing your “boundary diagram” for simulation. That is to say - choosing what’s in scope to be “same code as robot”, and what’s “sim-specific functionality”. The end usage goals are the primary input to determining what the appropriate boundary diagram(s) are.
WPILib supports this at the HAL layer, and at some other “engineering units” sim layers, as Peter mentioned. Other teams have picked slightly different layers. Again, the correct answer here is highly dependent on end use case.
IO Simulation - Open Loop
Rather than simply stubbing IO, the ones that you care about might need to be manipulated. This is purely for functionality checkout - IE, get certain lines of code to run. Inputs are manually set, outputs are manually checked.
A simple unit test infrastructure could provide the input and check the outputs. However, all testcases would still be “hand-maintained” to ensure they represent realistic-enough IO to test the right lines of code. And, again, depending on the goals, this may be enough to achieve them.
WPILib supports this through the same layers mentioned above, along with build infrastructure to define and run unit test cases.
IO Simulation - “Representative” Closed Loop
Rather than manually maintaining inputs and outputs which are just “good enough to hit the right line of code”, you could instead write down simple relationships between how you expect your inputs to change when your outputs have certain values. This isn’t yet modeling exact physics, but rather reducing the manual developer burden to maintain true “open-loop” test cases.
It could be as simple as “when voltage is > 0, speed should increase”. Again, the goal here is simply to get the right lines of code activated, while minimizing the developer burden.
WPILib supports sim-specific code entrypoints where you can write code to achieve this functionality. It also supports some device simulation classes that add in these behaviors.
Physics-Accurate Closed Loop
A further, orthogonal step is to make that sim-specific code not just “good enough to get the right lines of code activated”, but also to ensure the code output → code input relationships are physics accurate. That is to say - they accurately model your specific robot’s actual behavior.
This is a big step forward - it has very little to do with anything done previously, and is much more dependent on your ability to model a physical system accurately. It can add a lot in terms of checking alternatives in designs, but (in my experience) does fairly little for your ability to test your software well.
WPILib’s mechanism simulation classes, when paired with proper data capture and analysis, can help achieve this.
My Punchline
WPILib “supports simulation” to the extent that most of the key building blocks to physically model a robot+software system in a desktop environment are present. However, the onus is still on end teams to stitch them together in a way that achieves their goals.
Goals Analysis
Here’s some of my thoughts on the goals you provided, at least as I understand them:
“Unit testing” is actually your easiest goal to achieve. Pre-integration collaboration would be the next easiest. They require cross-compiling and poking at certain IO points, but neither requires closed-loop behavior to start.
Whole-system development (implying robot+software) and vision system testing would be the next hardest. In addition to some potentially-custom bits for the vision side, both seem to need some level of closed-loop modeling of robot behavior.
Analysis-of-alternatives is the most challenging. Not only do you have to have closed-loop models, but you have to trust their accuracy enough to drive design decisions.
RobotPy has supported robot simulation in some form since 2014, so it’s a topic I care about a lot.
I think the way that HAL and SimDevices et al is implemented is a pretty reasonable approach, it’s just not documented very well. Once you figure it out, it makes a lot of sense.
However, I really don’t like having simulationPeriodic and mixing your simulation/real code – and even after moving to wrapping WPILibC I’ve hidden it away from users. Instead, in RobotPy we have a physics.py which is only loaded in simulation. A very simple example of this sort of thing is in our examples repo: examples/physics-4wheel/src at main · robotpy/examples · GitHub . A more involved example would be my team’s 2022 code (2022-robot/robot at main · frc6367/2022-robot · GitHub).
I believe I advocated for this type of split instead when simulationPeriodic et al was introduced, but there’s a number of factors that make it a lot harder to do in C++/Java (particularly C++).