How to run vison processing code on classmate during matches?

[ganchara]

My team is going to be using vision processing for our hybrid mode. From what I understand, rather than running the processing code on the CRIO, it is run on the classmate or other laptop used for driving by my team. How do I make this separate code run on the robot while the robot is running?

[fb39ca4]

You would have to send the video frames to the computer, run your image processing program, and have it send relevant commands to the cRIO, all over the wifi link. IMO it would be better to have an on board laptop that does the same scheme, but now, it can be connected through Ethernet, which is much faster.

[sjspry]

The forums have various other threads discussing this scheme. Basically, connect your axis camera to your router and grab the stream of of it. But note it is possible to run NIVision on the cRIO (although I think someone mentioned they were attempting to offload it due to it being so CPU intensive).

[ganchara]

Are we allowed to have a laptop right on our robot to do the image processing? I was definitely planning to offload the images from the camera, rather than using the CRIO because there is just so much more processing power on a laptop. What I am asking really is how to execute the processing code that both finds the rectangle, and then figures out where the robot needs to go, on the laptop, rather than any of the other parts of the process, such as how to process the images.

[byteit101]

Originally Posted by ganchara

From what I understand, rather than running the processing code on the CRIO, it is run on the classmate or other laptop used for driving by my team.

yes and no:
Kinect sensor->data: yes, but its already built
axis camera processing: no (though you can, as you can see from the rest of this thread)
kinect data-> drive commands: no, on robot

If you are doing insane calculations on images, you should offload it to the classmate. If you are doing more simple image processing (rectangle tracking, etc...), the robot will suffice, as offloading it will take more work than it is worth

[Greg McKaskle]

If you plug the camera directly into the DLink switch on the robot, then programs that know the IP address and are on the subnet can request images. This is how the dashboard works. If you wish to change the dashboard, it is written in LV and source is provided, or you can choose one of the other DB tools.

So, getting an image from the camera is as simple as getting the laptop program to ask for it.

As for processing it, you can process it on the cRIO. The target isn't moving, and as long as the robot isn't moving very much, you don't need 30fps. Calculate the speed you need. For processing the image, you have a number of choices. NIVision works on the cRIO and on the desktop and has entry points for C and LV. OpenCV is another pretty accessible choice.

Greg McKaskle

[ganchara]

Originally Posted by Greg McKaskle

NIVision works on the cRIO and on the desktop and has entry points for C and LV.

Well, thats part of the problem we are having, my team is programming in java, and none of the other programmers on the team are willing to switch to C, which is what I am the most familiar with.

[Greg McKaskle]

I'm not sure what to do about that. OpenCV is definitely an option, and if someone is willing to do wrappers, NI-Vision is too.

Greg McKaskle

[ganchara]

Originally Posted by Greg McKaskle

and if someone is willing to do wrappers, NI-Vision is too.

What would have to be done to create the wrappers? None of the programmers on the team are very experienced with writing robotics java code.
Also, what I am really trying to ask in this thread is once the code is written, how do I execute it.

Originally Posted by Greg McKaskle

As for processing it, you can process it on the cRIO. The target isn't moving, and as long as the robot isn't moving very much, you don't need 30fps.

As an aside, the plan my team has is to use the rectangle as a way of positioning the robot, as we plan to have an extremely short range shooter. Would you still suggest processing at less than 30 FPS and on the cRIO?
Today 08:13 PM

[Joe Johnson]

I don't want to hijack this thread, but I have heard usually reliable sources say that sending pictures back to the Classmate (or whatever Win7 Machine you use on your driver station) is a bad idea.

The gist of their argument is that nearby the wifi is up to the task but from across the field with other robots all talking too, the frame rate drops to the low single digits (3-4 fps).

This is the kind of thing that scares me to death because I can't know that it is a problem until we get to an actual competition.

So... ...should I worry about the 10,000 other problems I have to worry about or should I continue to worry about this one (and perhaps decide that I don't really want to send the camera data to the remote Win7 Machine after all)?

Do tell.

Joe J

[~Cory~]

Originally Posted by Joe Johnson

The gist of their argument is that nearby the wifi is up to the task but from across the field with other robots all talking too, the frame rate drops to the low single digits (3-4 fps).

Last year we did such a scheme (no data sent back to robot, however) and were running at about ~21 fps

However we did have some small issues with the robot hiccuping. We believe the problem was the classmate's DS commands due to high latency (we graphed some data obtained from wireshark). We have not ruled out the camera directly attached to the wifi with 100% certainty...

[ganchara]

Originally Posted by Joe Johnson

I have heard usually reliable sources say that sending pictures back to the Classmate (or whatever Win7 Machine you use on your driver station) is a bad idea.

The gist of their argument is that nearby the wifi is up to the task but from across the field with other robots all talking too, the frame rate drops to the low single digits (3-4 fps).

I have heard similar things, however, the counter-argument that I have heard is that running image processing on the cRIO, if ran simultaneously with enough code (no idea how much though) causes noticeable lag in the processing speed. One solution to this that I have heard is to have a second laptop mounted on the robot, and directly wire that to the camera and cRIO, however, I am not sure if the game design rules allow this or not.

[siggy2xc]

You can't communicate between the robot and the laptop in hybrid/autonomous can you? or are things like the keyboard and joysticks the only things disabled?

[DjMaddius]

If allowed to put another machine on the robot for image processing, you may as well use the kinect on it instead of the axis! I've heard its very easy to get good data, much more than the axis could ever give you and with a computer with usb on the robot it is very possible and legal to do it. (The kinect anyways, I know nothing of the computer on the robot)

[sjspry]

I can vouch that grabbing the image stream from the Axis camera works fine. Our team got ~29 fps using the smaller resolution. The 3-4 fps I would imagine comes from someone trying to use the cRIO to relay the images.

As for communication during hybrid, you can, the only thing disallowed is human input (well, excepting the kinect).