Chief Delphi - Live Video on Dashboard

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)

- Programming (http://www.chiefdelphi.com/forums/forumdisplay.php?f=51)

- - Live Video on Dashboard (http://www.chiefdelphi.com/forums/showthread.php?t=75872)

Re: Live Video on Dashboard

I would really like to get a look at the code when you get time to post it. I do not really get what everything means but I want to learn, and our programmers are going to start on a code to try to get this to work and we are prolly gonna need all the help we can get.

Re: Live Video on Dashboard

Everybody seems to be doing great on this thread. I only have a few things to add. I've seen the camera timing get flaky when compression is set too high. I'm not sure where the threshold is or if it involves images size or is simply affected by compression factor. By flaky I mean that the image framerate will drop because some frames will take two periods to be sent.

Also, modifying the image and recompressing on the cRIO doesn't seem like a good use of CPU time. If you can find a single camera setting so that the image can be piped to the dashboard, that will have minimal impact on other tasks and will get the highest framerate to the dashboard.

Greg McKaskle

Re: Live Video on Dashboard

Quote:

Originally Posted by writchie (Post 837255)

Depending on how good the jpg encoder is, i.e. speed and file size at high compression settings, it looks like it might be possible to handle higher quality 320x240 images (for better tracking), scale them to 160x120, compress them with a high setting, and ship them off to the dashboard while maintaining 25 fps.

Since the jpg decoding speed is pretty slow, I would expect the encoding speed to have similar issues. Have you measured how much processor time is required to encode at high compression settings?

Since our entire set-up was implemented on the Thursday night/Friday morning of the competition, I didn't have an opportunity to do extensive testing. As soon as I get access to the code I'll try to do some benchmarking.

Quote:

Originally Posted by Greg McKaskle (Post 837523)

In what we were doing, the video didn't necessarily have to be 25 fps, we settled for 3-5 fps (also probably because of time constraints. along the lines of "does it work?" "yes" "ok awesome don't break it"); all of our images were being split across multiple frames. We had a some problem with dropped UDP packets (probably somewhere along the line of 1 out of every 50 or so), which means we would drop a frame every 2-3 seconds, but it didn't seem to be a problem.

It should be noted that our application was essentially a monitor to tell our operator if he should push the "dump" button, since our robot was tall and opaque; we weren't actually trying to drive the robot real time with this. It's my opinion that basically no matter how good you get the video framerate/quality, it's not going to be able to beat just watching the field for driving ability, due to the camera's relatively narrow field of view. I don't say this to discourage anyone, just to make sure teams are being reasonable about what this can accomplish.

Quote:

Originally Posted by Greg McKaskle (Post 837523)

Also, modifying the image and recompressing on the cRIO doesn't seem like a good use of CPU time. If you can find a single camera setting so that the image can be piped to the dashboard, that will have minimal impact on other tasks and will get the highest framerate to the dashboard.

I agree, I'll have to do some experimenting with the camera compression settings.

However, I was finding that by using parallel while loops, I could keep the bottle neck as the network transfer speed (granted we were still using 10 frames to transfer one image). Again, I'll have to do more rigorous testing, but it seems like most teams won't be doing much else with CPU time than running a basic driver control loop, and using parallel whiles with appropriate considerations for timing should keep this running smoothly.

--Ryan

Re: Live Video on Dashboard

I'm extremely new to LabVIEW programming, and I'm wondering if I can have a copy of your Dashboard code if you actually got it working. Thanks.

Re: Live Video on Dashboard

I am new to programming things to send over a network. How do you compress files and images in the C++ code. How would i uncompress the files and images in VB.NET. I would love to see the hard code of the camera feed functions.

Re: Live Video on Dashboard

Quote:

Originally Posted by RyanCahoon (Post 837162)

Unfortunately I won't have access to the code for another week or so, but I can post the code then if anyone is interested.

I still haven't had access to a computer with LabVIEW on it (I was short sighted and didn't get it on my personal laptop), but I got our coach to send the code to me. This is hack-quality code, so it's not the smoothest, but regard it as a proof of concept. When I get some time to work on it, I'll try to clean it up, but I thought I'd post it so people could experiment if they want to.

Basic Robot Main.vi

Computer Dashboard

Note that the critical parts of the robot code are the two while loops at the top of the program. The rest is just drive code etc. For people who want to experiment, I'd recommend copying those two loops into their own code. Some changing of IP addresses might be required, I'm not sure if I hard coded any in. Again apologies for the sloppy code.

--Ryan

Re: Live Video on Dashboard

Quote:

Originally Posted by xtreampb (Post 840735)

You can use the imaqFlatten(...) method from nivision.h to compress images in C++, but this puts it into what's probably best called a proprietary format. It supposedly can be set to use JPEG to compress the image, so you could probably extract a JPEG from it somehow, but it's most likely going to be padded with extra data.

There's also a series of imaqWriteTYPEFile functions, but these write to the filesystem, so you would then have to read in the file again in order to send it over the network. Unless of course someone knows if there are/how to use pipes in VxWorks.

Assuming you've gotten your JPEG over the network unconvoluted

Code:

Dim im = New Bitmap(stream)

where stream is the stream coming from your network connection, or if you had to reconstruct the data because it was packetized to fit in the 984 byte constraint, you can use

Code:

Dim im = New Bitmap(New IO.MemoryStream(byteArray))

--Ryan

Re: Live Video on Dashboard

Quote:

Originally Posted by RyanCahoon (Post 841006)

Thanks for the code. I see where you are enqueueing the data, but where does in get dequeued and sent to the dashboard? the question may be because I'm not real familiar with the basic robot project.
Thanks for your help!

Re: Live Video on Dashboard

Quote:

Originally Posted by dwodrich (Post 841641)

Thanks for the code. I see where you are enqueueing the data, but where does in get dequeued and sent to the dashboard? the question may be because I'm not real familiar with the basic robot project.

I'm actually not sure where this happens either. I suspect somewhere deep in the code (similar to the code that's responsible for transferring back camera images to the computer that doesn't work in competition, as you don't have to enable that anywhere that I know of either) there's a loop running that takes whatever is stored in the queue called "Dashboard Data" and passes it over the network.

This part of the code isn't mine; if you open up the default Build DashBoard Data.vi, you'll see a similar thing happening, which is where I got the code from.

Let me know if you have any other questions.

--Ryan