Re: Speed of the camera
 

What was it?


Also, what think is at least as important, if not more important, is the effective range. Using the FRC sample tracking program had a range of 10-15 feet. As it increased over about 10 ft, it had more and more trouble tracking a moving target although stationary was fine for a few more feet. This is probably a result of the low res (160x120) we're using, although thats the only res with any usable framerate - around 18 fps.

-jonathan

Re: Speed of the camera
 

I only tested two resolutions, but I remember large being 4fps, medium is around 5.8, and small is around 7.5.

The ability to detect the target reliably will be greatly affected by the lighting. It sounds like your lighting is good enough to get stuff to work. But how does it compare to the event. And when it gets flaky, is it purely due to distance, or is it because you are going into a dark spot or an area with different lighting. Use the debug HSL button to sample the images if that helps. See if some flood lights like they have at events help -- note that depending on how they are mounted, they can cause lots of glare, especially when in the same horizontal plane as the camera. Also note whether you are processing all of the 160x120, or are you decimating it further?

Finally, you may need to switch resolution to see further. Depending on strategy, I could imaging that when there are no close targets, you switch to 320 for awhile, then when the size goes over a threshold, you go back to 160 for faster frames.

Greg McKaksle

Re: Speed of the camera
 

Has anyone that is willing to share found a way to speed the processing up yet?

Just curious...

Thanks

Re: Speed of the camera
 

I am running the two-color tracking code on a 160x120 image at 5Hz and it consumes nearly 50% of the CPU bandwidth (measured using 'spy').

Re: Speed of the camera
 

How smooth does that run the servos?

I am going to be running a motor, not servo's... I would feel safer if it were 10-15Hz...

Re: Speed of the camera
 

Is this with LV or C? What framerate did you ask the camera to run at? Did you add printfs or anything?

Greg McKaskle

Re: Speed of the camera
 

"Is this with LV or C? What framerate did you ask the camera to run at? Did you add printfs or anything?"

That is in C/C++. I passed 5 as the framerate parameter to StartCameraTask. And I have pulled out the code from the demo robot task (minus the wheels stuff) and run it every 10th ds message or about 5Hz. The DPRINTF code is there but Target_debugPrint is 0 so no messages.

Re: Speed of the camera
 

If I remember correctly. A couple of teams greatly improved the overall performance of the old camera by puting a polerized lenz in front of it. Has anybody tried this with the new camera?

Re: Speed of the camera
 

Have you tried running it faster? The CPU will ideally be below 100% to keep odd things from happening to your scheduling, but you have lots more bandwidth.

Greg McKaskle

Re: Speed of the camera
 

If I use a 320x240 image, the camera consumes close to 75% of the bandwidth. If I keep it at 160x120 and run it at 10Hz I go just over 80%. So I'll probably run it faster after all the rest of my code is running.

This is an embedded system where we desire soft real-time performance. Consuming the CPU bandwidth is not the correct measure or goal. We want it run reliably in a fixed amount of time each iteration.

The dynamic memory allocations are a real killer. Walking the free list (while locking everyone else out) is a time consuming and non-deterministic algorithm. The HitNodes are all fixed size so I'll probably convert that code to use a set of pre-allocated buffers. I'll bet that will speed it up quite a bit.

Re: Speed of the camera
 

I am currently running some tests of decoding speed for other JPEG libraries. I'll also look a little bit more closely at the image structure used in the JPEGs from the camera to see if there is much shared data - apart from width, height, etc., I'm wondering whether the MJPEG chip on the camera is making similar huffman tables and other things between each frame.

Currently I have ported one other JPEG library over to the system (mainly just changing a bunch of x86-specific stuff, endianness, etc.). I'll get figures on speeds back as soon as I can (likely tomorrow) and am hoping I can see some sort of speedup.

I have also noticed the large amounts of allocations/deallocations used in the current code, and actually rewrote just about everything I could (camera streaming code, decoding functions, analysis, etc.) to remove unnecessary dynamic memory allocations. Sharing buffers between functions, only calling new when a buffer needs to be expanded and keeping track of its growing size, etc. The effects are not really noticeable in standard usage, but in debugging I'm able to get a live feed with no slowdown on the system or streaming.

Looking at the GFLOPS on the processor, I'm fairly confident in finding a way to get 8-12FPS on a 640x480 image, which is my goal. It's just all about optimizing properly.

*edit*
Oh, and just a random musing... At 1000 images per second of 640x480, (that is _assuming_ that the entire image is fully decoded) the processor would have to be approximately capable of decoding one pixel every instruction... Meaning, each instruction would have to decode the Red, Green, and Blue components of a pixel at once. This leaves a number of thousand instructions (about 20% of the time) over to actually analyzing the said image, and presumably, converting the RGB to HSL.

On my Core 2 Duo 2.53GHz, utilizing one core at 100%, I can decode images from the camera in around 29ms (640x480). Based on what I could find, the processor in the cRio is about 1/4th as fast as mine here, in terms of GFLOPs (JPEG decoding is very math heavy, usually with floating points, sometimes fixed point integer math). I am unsure of the validity of these sources, however... That puts an upper limit on JPEG decoding speed of around 120ms for a 640x480. When you consider that most of this speed comes from architectural differences rather than clock speed, you realize that the actual speed will be even slower after accounting for the many non-math portions of the code (traversing trees, function overhead, etc.). So I believe the maximum speed for decoding a 640x480 on the cRio will be around 180ms if you use code approximately as optimized as the JPEG library used on my mac. I have been unable to find any PowerPC optimized decoder, backing up what Greg mentioned earlier.

So I imagine the wall-blocks are around here (theoretically, at least, based on the logic pattern I've gone through):
640x480: 180ms, around 5.6fps. Funnily enough, this isn't too much more than what I was getting in my original tests.
320x240: 45ms, around 22.2fps. Seems reasonable, and a good place to aim for a fair trade-off.
160x120: 11.25ms, around 89fps. I somehow doubt this. Besides, it's not really useful anyways above 30fps, even if it was.

Does anyone have code that beats these marks? (That is, that actually fully decodes the image)

Re: Speed of the camera
 

The times you mention are about double what I've measured on the cRIO. The measured times are 100ms for large, 22 for medium, and 8 for small. What I'm getting at is that your estimates are relatively close to the current times on the cRIO. With enough time and effort, I'm sure we can trim and shave and reclaim some time. In the off season I hope to spend a bit of time there, but not now.

As for the allocations and stuff that are in the demo code, it is certainly true that the more allocations you have, the less deterministic the execution timing of the code. On the other hand, I wouldn't expect the time cost to change much. The memory managers these days have the right algorithms and enough memory to get good timings on most operations.

Greg McKaskle

Re: Speed of the camera
 

Ah, yes, that makes sense. I looked up the GFLOPs performance again, and I was using the total for both cores - divide that by two, and it appears it works out perfectly. Thanks for the reinforcement. Although I imagine that the total time works out for processing as well, so it may still be reasonable as an estimate of the overall tracking code?

Re: Speed of the camera
 

For fun I made someone reproduce my camera timings, and here are a few things we learned. He wasn't able to get decent camera timings, he was getting around 4fps for 320x240.

1. He had set his JPG compression to 0 when messing around with the camera. When set to very small numbers and to very big numbers near 100, the camera takes much longer to produce the jpgs. This increased the time to get an image to several hundred millisecs.

This improved the numbers, but they were still not what I'd been seeing.

2. He had never set his camera up using the setup utility. He didn't have an account of FRC/FRC. This means that the first authentication to the camera always fails, and the camera is always using the second session attempted. This will pretty much cut the frame rate in half. This issue is probably LV specific, but if you have not done so, run the utility or log into the camera using a web browser. If you cannot log in using FRC for account and FRC for password, you have this problem. Run the utility, or log in and manually create an account.

Hope this helps someone out there.

Greg McKaskle

Re: Speed of the camera
 

I also realized the issue with authentication - in the end, I decided it was smartest just to leave authentication out entirely. There is an option on the camera to allow viewing anonymously. However, you say this cuts the frame rate in half. Does the default code request a new image each time, or does it tap into the MJPEG stream of the camera? In my code I have a task running that constantly gets the MJPEG stream at top speed without any decoding, then a second task that grabs whatever is the latest image and processes it. This is partly how I managed to get a live feed without much slow-down.... It simply makes yet another copy of the raw JPEG data and pushes it to the computer without any processing (using a web server I wrote for it. I have an AJAX application I uploaded to the cRio that just sits and refreshes the address of the image as fast as possible.). Luckily the form of MJPEG the camera uses also pushes headers for each and every image, so it works out.

The information about the compression time, however, is interesting - On this camera, the JPEG compression is handled by a dedicated ASIC, so I wouldn't expect such dramatic changes. I have simply been using the default, whatever that is - I imagine it to be 85.