We are working on an offseason vision system that uses an intel Realsense depth camera for more accurate object detection. We know that the camera is great at detecting the distances objects are away from it, and think that it can be great for detecting game pieces or even other objects on the field. We need advice on how to integrate everything with Networktables, and need programming help with object detection. If anyone has any insight on how the process would work, and if any preexisting solutions exist, we would be grateful.
We looked at using these a year or two ago (even got one of the cameras to experiment with). We ultimately abandoned the idea due to limitations on the framerate (only 30FPS on the RGB camera, which is still needed for effective object tracking), and the processing requirements. The other issue is that the LIDAR/IR grid used for depth tracking has a lot of problems dealing with reflections, which are very common between the diamond plate and polycarbonate of the field perimeter.
More power to you if you can make this work, but before you get too far into developing the networking and programming for a FRC vision solution, I would strongly recommend playing around one of the camera modules using the stock tools and SDK that are available on a regular PC to see if it’s even viable.
Hi there! Funny enough we have done something very similar. We wanted to be able to take advantage of steroscopoc object loclization, so we ended up using a ZED 2 depth camera with a custom developed CUDA accelerated backend. We have had great success so far and will be deploying it on our robot at madtown throwdown this offseasion (most likely anyway)
I used Intel realsense d435 using python on a coprocessor.
I answered some questions about it in this thread:
Edit:
The code repo uses basic color/shape filter for cone/cubes in 2023 season, network tables to communicate to rio, and uses pyrealsense2 library in a python3.8 virtualenv for using the realsense API.
My mentor helped build the pyrealsense2 library from source on my coprocessor
