I have a Pololu Romi with the latest WPILibPi installed on it. On a PC (localhost) connected by wifi to the Romi (10.0.0.2), I am able to build robot java code within the 2023 WPILib vscode and control the Romi using an XBox controller through the WPILib Robot Simulator on the PC. This part works great
If I connect a webcam to the host PC, I am able to see the input video feed on Shuffleboard (connected to localhost) if my robot code contains the following lines:
UsbCamera camera = CameraServer.startAutomaticCapture(); camera.setResolution(640, 480); CameraServer.getVideo();
I am also able to run some OpenCV processing code on the input images and use
CvSource outputStream = CameraServer.putVideo(...) and
outputStream.putFrame(...), which shows the processed result on a Shuffleboard camera widget.
However, what I want is for the webcam to be connected to the mobile Romi, not tethered to the bulky PC. In this case, there are two main options for image processing:
A) Analyze the Romi webcam image with code running on the Romi raspi, and then transfer the results to the Robot code on the PC (via network tables).
B) Somehow (but I don’t know how) transfer the image from the Romi webcam to the PC and have the Robot code on the PC perform the image analysis
Are there any major advantages / disadvantages of A or B?
I think option A (using the Romi raspi as an image co-processor as it was probably designed for) is preferable, but it seems difficult to debug. Specifically, I like the idea of creating different
CvSource outputStream(s) and being able to see the intermediate stages of the analysis pipeline on Shuffleboard. That works when the webcam is connected to the PC, but not when it is connected to the Romi, i.e. so far, when I run (python code)
output_stream.putFrame(output_img) on the raspi, I cannot see the image streams on Shuffleboard, which makes it very difficult to debug.
For option B (analysis code running on the PC robot code), I can easily see the outputStreams on Shuffleboard when the webcam is connected to the PC. But when the webcam is connected to the Romi (which is what I want), I would need to transfer the raw image data from the Romi raspi to the PC robot code.
To summarize, my questions with the webcam connected to the Romi are:
- if the analysis takes place on the raspi, how could I see intermediate stages in the image processing pipeline on Shuffleboard (with Shuffleboard connected to localhost so I can see other SmartDashboard values at the same time)?
- if the analysis instead takes place in the robot code running on the PC, how can I transfer the raw webcam image from the Romi to the PC ?
Sorry if I am missing something obvious, and if I am, could you please guide me?