Implementing Grip Pipeline on command-based C++

[Team4470]

Hello,

This is our first year using a command-based project and using GRIP vision processing. We have made a grip project and have worked with the command-based project with success. Now we are faced with the problem of how we are supposed to implement the GRIP code that we have generated. We are able to send a camera feed to the driver station using the StartAutomaticCapture() function in our camera subsystem. We tried to make the grip pipeline a command with the .cpp and the .h file provided in the generated code. We also tried to make a grip pipeline object and call the .Process() method by passing in a matrix generated by GrabFrame(). This compiles but causes the error: “device or resource busy” and sometimes we would lose robot code altogether. What is a method we can use to implement our grip pipeline into our code?

Here is a link to our code:

Tigears4470/2020RobotCode

Thanks

[Peter_Johnson]

You are trying to access the UsbCamera on port 0 in two places: VisionCommand.cpp line 24 and CameraSubsystem.cpp line 11. Whatever is doing it the second time will give a resource busy error.

My recommendation would be to move the entirety of your VisionCommand to the subsystem. It also needs to be in a separate thread; right now your VisionCommand will block the entire robot program. See VisionRunner for an easy way to do this, or this documentation page.

You can use cs::UsbCamera by value everywhere, it keeps an internal reference count.

[Team4470]

I apologize, that was our code before grip, our latest code is in the git hub branch “grip_vision”.

I can see what you are saying but does that mean we should put the whole grip pipeline in our camera subsystem? Also how would we multi thread that because we had plans to use either command groups (we are still not sure if a parallel command group is multi threading) by making the grip pipeline a command (as it is in the latest code) or we would try creating a grip pipeline object and running that in a std::thread although, again we are still not sure if that will cause problems because unlike java, C++ does not have a frc vision thread class.

Our team was also considering using a raspberry pi to lessen the burden on the roborio processing our vision, would that be the better idea?

Thanks

[Peter_Johnson]

Yes, putting it on a Raspberry Pi is a good approach, especially if you’re uncomfortable with adding threading to your robot code. I recommend you take a look at the FRCVision image and its examples.

[Team4470]

I apologize for not reading up on this yet, but will I still need to figure out how to incorporate the grip pipeline in my current code?

edit: To clarify, does that mean I put the grip pipeline code in the raspberry pi or do I keep it in the roborio just like how I have been doing it.

[Peter_Johnson]

You would put the GRIP pipeline code on the Raspberry Pi. You can use Java, C++, or Python on the Pi (the FRCVision contains a full copy of WPILib).

[Team4470]

Ok so I looked into the Raspberry Pi and I have setup the FRC vision image on the pi and I can access the frcvision.local web server and I’m able to see the camera stream. I read the wpi instructions on how to upload code and it says to put an executable cpp file and it should run simply like that. Do I put the .cpp file that grip generates in there because it also comes with a header file that is necessary with all of the method and variable declarations? Or do I just put all of the variable deceleration in the .cpp file and upload that? We are a bit lost at this point. We looked at the example program and that created a grip pipeline object but where do we put the grip pipeline class itself?

[Peter_Johnson]

You have to build the .cpp file using make to create an executable, which is what you then upload to the Pi. See the instructions in the example; for C++ you need to install a cross-compiler and put it on your PATH, then run “make” to build the executable. You’ll need to edit the Makefile to add the generated .cpp file so it also gets built, and modify the example .cpp file to use the GripPipeline instead of the example pipeline.

[Team4470]

When running the “make” command on the example raspberry pi code we get an error:

PS C:\Users\newton\Downloads\cpp-multiCameraServer\cpp-multiCameraServer> make
arm-raspbian10-linux-gnueabihf-g++ -pthread -g -o main main.o -Llib -lwpilibc -lwpiHal -lcameraserver -lntcore -lcscore -lopencv_dnn
-lopencv_highgui -lopencv_ml -lopencv_objdetect -lopencv_shape -lopencv_stitching -lopencv_superres -lopencv_videostab -lopencv_calib3d -lopencv_videoio -lopencv_imgcodecs -lopencv_features2d -lopencv_video -lopencv_photo -lopencv_imgproc -lopencv_flann -lopencv_core -lwpiutil -latomic -Wl,–unresolved-symbols=ignore-in-shared-libs
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lwpilibc
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lwpiHal
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lcameraserver
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lntcore
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lcscore
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_dnn
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_highgui
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_ml
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_objdetect
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_shape
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_stitching
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_superres
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_videostab
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_calib3d
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_videoio
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_imgcodecs
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_features2d
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_video
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_photo
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_imgproc
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_flann
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lopencv_core
c:/raspbian/raspbian10/bin/…/sys-root/usr/lib/gcc/arm-linux-gnueabihf/8/…/…/…/…/…/…/arm-linux-gnueabihf/bin/ld.exe: cannot find -lwpiutil
collect2.exe: error: ld returned 1 exit status
make: *** [main] Error 1

[Peter_Johnson]

Do you have a “lib” directory in C:\Users\newton\Downloads\cpp-multiCameraServer\cpp-multiCameraServer that contains a bunch of .so files? There should be one, as it’s included in the zip file.