Grip & Intermediate Vision Examples Errors

[SLAB-Mr.Thomas]

Our team wants to evaluate embedded C++ Grip code on the roboRIO. We can't find a complete C++ example on how to do it. Comments here on Chief Delphi point out some errors in the Grip generated C++ code and to use the check box when generating the code from Grip but no explanation of why to use the check box or not.

We thought the Intermediate Vision example available in Eclipse new WPILib example project might give some clues on what to do but that example fails to run correctly.

We can get the "raw" camera image from the roboRIO USB 0 camera to display on either of the Driver Station dashboards (default or C++) (essentially the Simple Vision example) but attempting the use the additional video stream for the "Rectangle" results in no output except the error message:

CS: ERROR: serve_Rectangle: Too many simultaneous client streams (MjpegServerImpl.cpp.403)

Selecting the Rectangle instead of the USB 0 camera device on the Dashboard does not change the error.

Any suggestions on how to get the embedded Grip C++ code to work and also the Intermediate Vision example?

Thanks.

[beijing_strbow]

In the Java Intermediate Vision example, we had to declare the thread in a different way, because there was an issue with the syntax. I have no idea if that issue is also present in the C++ code, but it might be worth checking. Perhaps try running the code in the main thread, and see if that works.

And on the error, I believe we were also getting that on our vision processing. Another one of our programmers fixed that - I'll see if he remembers how.

[beijing_strbow]

He said that error was also a result of doing processing in a separate thread. So right now we're just doing everything in the main thread, and maybe we'll add concurrency later.

[Peter Johnson]

I'm not sure why the intermediate vision example is not working out of the box for you. What camera are you using? Are you connecting to the roboRIO via USB or Ethernet? What does Outline Viewer show under /CameraPublisher/USB Camera 0 and /CameraPublisher/Rectangle for description and mode?

The error message is due to too many simultaneous streaming connections to a given server (it's limited to 10 as a self protection measure to prevent the roborio from getting clobbered by a badly behaving dashboard). It should have nothing to do with how the frames are being fed on the robot side (it should make no difference whether PutFrame is called from a separate thread or not).

If you shut down all of the dashboards, can you connect to http://roborio-<team>-frc.local:1181/ and http://roborio-<team>-frc.local:1182/ with a web browser and see anything (in the intermediate vision example, the first should be the USB camera, the second the rectangle stream)?

[SLAB-Mr.Thomas]

Thanks for all the good thoughts. My Eclipse WPILib plugins updated automatically last night and the vision samples now work perfectly without any code changes. (Not sure the 2 events had anything to do with each other.)

The only time I get that error message now is if the camera is unplugged (unusual message but probably reflects what's really happening in the background to attempt to restart the camera which it does do just fine.)

Was able to get Grip embedded running on roboRIO, albeit slowly (2 frames per second for our 5 steps) and only after fixing all the coding errors in the generated code.

Only question left is what's VisionRunner do for me? I'm not knowingly using it and processing is working.

Repeating what others have found but with my actual code posted herein:
Generate C++ code WITHOUT the checkmark for VisionPipeline.
Generated code has a missing cv:: in the SimpleBlobDetector code.
Generated code has a spurious include for contrib.
Generated code has an extra setsource0 method definition or a missing method declaration. Removing the method seems to be fine.

In robot.cpp of the Intermediate Vision Example:
add my Pipeline class name:
#include "Pipeline.h"  // Base Name matches generated file name. NO Checkmark on the Grip generate C++ code!  (creates reference to parent class VisionPipeline that doesn't exist in C++)

add to VisionThread:
		grip::Pipeline Gripper;  // Class Name matches generated file name.

add after rectangle(....):
			Gripper.process(mat);  //  run the Grip pipeline

			// Give the output stream a new image to display
//			outputStream.PutFrame(mat);
			outputStream.PutFrame(*Gripper.getblurOutput());

Remove from generated code Pipeline.cpp:
//void Pipeline::setsource0(cv::Mat &source0){   // removed. method doesn't have a declaration and its function seems to be redundant with the parameter
//	source0.copyTo(this->source0); 				 // removed. on the constructor so remove this method instead of fixing it
//} 											 // removed.

Fix generated code Pipeline.cpp:
		cv::Ptr<cv::SimpleBlobDetector> detector = cv::SimpleBlobDetector::create(params); // Add cv:: in front of 2nd SimpleBlobDetector


Remove from generated code Pipeline.h:
//#include <opencv2/contrib/contrib.hpp>	   // removed.  file doesn't exist.  Possibly an OpenCV artifact that WPILib has no need for now

[SamCarlberg]

Originally Posted by SLAB-Mr.Thomas

Thanks for all the good thoughts. My Eclipse WPILib plugins updated automatically last night and the vision samples now work perfectly without any code changes. (Not sure the 2 events had anything to do with each other.)

The only time I get that error message now is if the camera is unplugged (unusual message but probably reflects what's really happening in the background to attempt to restart the camera which it does do just fine.)

Was able to get Grip embedded running on roboRIO, albeit slowly (2 frames per second for our 5 steps) and only after fixing all the coding errors in the generated code.

Only question left is what's VisionRunner do for me? I'm not knowingly using it and processing is working.

Repeating what others have found but with my actual code posted herein:
Generate C++ code WITHOUT the checkmark for VisionPipeline.
Generated code has a missing cv:: in the SimpleBlobDetector code.
Generated code has a spurious include for contrib.
Generated code has an extra setsource0 method definition or a missing method declaration. Removing the method seems to be fine.

In robot.cpp of the Intermediate Vision Example:
add my Pipeline class name:
#include "Pipeline.h"  // Base Name matches generated file name. NO Checkmark on the Grip generate C++ code!  (creates reference to parent class VisionPipeline that doesn't exist in C++)

add to VisionThread:
		grip::Pipeline Gripper;  // Class Name matches generated file name.

add after rectangle(....):
			Gripper.process(mat);  //  run the Grip pipeline

			// Give the output stream a new image to display
//			outputStream.PutFrame(mat);
			outputStream.PutFrame(*Gripper.getblurOutput());

Remove from generated code Pipeline.cpp:
//void Pipeline::setsource0(cv::Mat &source0){   // removed. method doesn't have a declaration and its function seems to be redundant with the parameter
//	source0.copyTo(this->source0); 				 // removed. on the constructor so remove this method instead of fixing it
//} 											 // removed.

Fix generated code Pipeline.cpp:
		cv::Ptr<cv::SimpleBlobDetector> detector = cv::SimpleBlobDetector::create(params); // Add cv:: in front of 2nd SimpleBlobDetector


Remove from generated code Pipeline.h:
//#include <opencv2/contrib/contrib.hpp>	   // removed.  file doesn't exist.  Possibly an OpenCV artifact that WPILib has no need for now

The VisionPipeline/Runner/Thread stuff is to make it as simple as possible to run vision processing in a separate thread. It's mostly intended for less experienced teams so they don't kill the main robot program by running vision on the main thread. You don't need to use it if you don't want to. The VisionPipeline include should actually be "vision/VisionPipeline.h"... I'll fix that for the next release

The setsourceN functions weren't removed when I changed the process() method to take all the sources. They can be safely deleted (and they'll be gone in the next release of GRIP).

The include for the OpenCV contrib module can also be safely deleted. I'm working on fixing imports/includes to only use what's needed, much like RobotBuilder. That's going to take a while due to a major internal rework of the code generation module, so for the time being you'll just need to remember to remove the unneeded includes.