Communicating Cameras over Wi-Fi from Raspberry Pi

Technical
#1

Hi all! Alright so I’d like some assistance if possible. I am currently working on a robot utilizing a roboRIO as our main computer and a Raspberry Pi functioning as our wireless connection to a Linksys AC1200 router which is connected over ethernet to a computer which we use to control the robot. We’ve been utilizing this method for about a year now thanks to marshall who helped us set up our raspberry pi as an ethernet bridge, however now our next step is communicating our two cameras across this network.

Ideally, we’d like to communicate the cameras to both the RoboRIO and the controlling computer, however I’d settle for communicating them to the computer so we can see through the robot as a first step then move on to the RoboRIO at a later date. We’ve been able to see through the cameras on the RPi, but the communication and receiving aspect is what I’d like some help on.

Any help is appreciated! If any one is confused about our network, below is a diagram.

LUN_COM_ComputerNetworkDiagram
LUN_COM_ComputerNetworkDiagram915×644 5.86 KB

#2

Are you doing any video processing local to the rPi? The rPi is going to have a lot more horsepower (and lower latency) to process the camera video than the RoboRIO will.

The most common way in FRC to do camera streaming at present is using CameraServer, which opens the camera using /dev/videoX and sends the video stream using a MJPEG over HTTP transport (which can be viewed in a standard web browser or via most of the dashboards). CameraServer also supports doing local video processing by exposing the video stream via OpenCV.

Alternatively, you can look into custom transports such as gstreamer to send data more efficiently (e.g. h264 over RTSP). Or if you want to get really fancy, ROS, which has a much steeper learning curve, but has greater framework support for Realsense depth cameras.

1 Like
#3

Are you using the WiFi interface on the Raspberry Pi, which is onboard the robot?

#4

The plan is to eventually utilize the RPi for all video processing, and actually have a google coral on hand once we get into more video processing. We do not plan as of now to use the RoboRIO to process camera video, when I say I’d like eventually to transmit data from the camera to the RoboRIO, I’m mainly thinking of target data and the IMU in the cameras, but that’s a problem for later.

Could you point me in the direction of a guide to use CameraServer on the Raspberry Pi so I can begin digging into it? I haven’t had much experience with video processing in FRC, my team (when it existed) just plugged in cameras to the roboRIO and prayed they worked.

We eventually plan to utilize ROS, but that is a plan for the future when we have the time to learn it (I’m planning on taking a class in the fall!)

#5

I believe so, yes, the Raspberry Pi is what we are using to connect to the Router, there is no hardware between the two.

#6

A lot of teams use the WPILibPi image, which is a custom Raspbian-based image for the Pi with all the CameraServer libraries already built and a web dashboard for configuring cameras. It does have wireless disabled by default for FRC competition legality, but it’s pretty easy to re-enable it. Docs here: Vision with WPILibPi — FIRST Robotics Competition documentation

In FRC use cases, most teams use NetworkTables to transmit target information from the Pi or other coprocessor to the RoboRIO.

1 Like
#7

Is this only in a lab-testing environment? Or for competition too?

#8

Thank you, I’ll begin digging through this and see what issues I may run into. I might need some assistance re-enabling the wireless as it has been a second since I have worked with the RPis wireless.

#9

To re-enable wireless, you essentially need to run the below (as root):

#10

So this is for the NASA RMC Lunabotics competition which my collegiate team is a part of and will be going to in about 2.5 weeks.

1 Like
#11

Hi Peter,

Please excuse my lack of understanding on this subject. I have imaged my RPi with WPILibPi, gotten to the web dashboard, and even connected a basic camera to it and been able to see through it. However, when I connect the intelrealsense d435i camera to it, when I pull up the camera stream it doesn’t show anything.

This I assume is because this camera is special and I will need to do some setup for it on the RPi. I’ll work on that part when I get to it.

As I am unfamiliar with WPILibPi, I’m wondering how I get to some form of command window/prompt on the RPi now to run commands such as shown above to re-enable wireless?

#12

Use an application such as PuTTY to connect to it via SSH. Username “pi”, password “raspberry”.