Unity WebSockets Hardware Simulation for 2024

Hello,
I have been searching for almost 6 months to find my team a simulation software that actually replicates the hardware on the robot. There seems to be incredibly sparse resources in this area—the built-in WPILib simulator is very limited, Synthesis doesn’t appear to work anymore, AdvantageScope is focused on replaying inputs, etc. So far, the most promising option I have found is the work of Bill Kinahan, who, in 2021, set up his WPILib robot project to run a WebSockets server and made a Unity project that interfaced with it and ran fully simulated hardware. Being able to do this would be a very powerful tool, and I am experienced with Unity so I am willing to expand on his prototype and add support for more hardware, etc. However, support for this project seems to have died out in 2021, and, although I can get the code working if I use the 2021 version of WPILib and run his exact robot project, I can’t figure out how to get it working with actual 2024 robot code. I have found no way to configure the 2024 build.gradle file to get the WebSockets server running—or maybe I have but it is the wrong version and the Unity won’t accept it, or maybe there is another issue entirely. I am very much in the dark when it comes to WebSockets, and WPILib provides almost no documentation. Does anyone know if there was more work done on Bill Kinahan’s project? Or does anyone know more about WebSockets simulation with WPILib? Thanks so much for your help. If uploading my build.gradle files would be useful, let me know.

Unfortunately, you are in a pretty sparse area. Bill’s effort is the only substantial effort I’ve seen on this. You may be able to reach out via email, but the WPILib work is standard websockets and plumbing unity shouldn’t be too bad.

There isn’t really a canned thing for this, which is also what makes it an exciting project! Excited to see what you come up with.

1 Like

You can use AdvantageKit to log your simulation IO layer and use AdvantageScope to visualize it. If you are worried about physics simulation, WPILib has support for it.

This did change between 2021 and 2024. Romi projects use the WS client. The template build.gradle for them is found here and contains the WS enabling code: vscode-wpilib/vscode-wpilib/resources/gradle/javaromi/build.gradle at main · wpilibsuite/vscode-wpilib · GitHub

Have you seen the documentation in GitHub? The environment variables are documented in the client and server plugin readmes, eg allwpilib/simulation/halsim_ws_server at main · wpilibsuite/allwpilib · GitHub

The protocol hasn’t changed, and is documented here: allwpilib/simulation/halsim_ws_core/doc/hardware_ws_api.md at main · wpilibsuite/allwpilib · GitHub