Re: Speed of the camera
 

The tracking software our team is using was written by me. The WPI tracking software spends the majority of its time on processing the image. I'll let you figure out what happens once you figure out how to process an image extremely fast.

-TheDominis

Re: Speed of the camera
 

Feel free to share this, if you care to. The community benefit would be rather large.

Either way it will get patched eventually.

In other news, I am also running the default tracking C++ code (160x120) at ~20 Hz. I experimented with cropping, and the gains were negligible. (This could be due to a lighting issue that I did not realize at the time.) This backs up what Greg has been saying. Make sure your target is well lit.

Re: Speed of the camera
 

We've been having some problems relating to the TCP connection, and was wondering if anyone could help. Although our image processing was going fast enough, it was taking up to ~1.5 seconds for image acquisition through the TCP connection because it was being opened, read, closed, etc every time in the loop. When we took the TCP open/close out of the LabView loop, we discovered that it wouldn't work for more than 1 image (probably because the camera's server was closing the connection). When we looked on the camera's online interface, we could see near-seamless near-realtime video. Is there any way to get around this (keep the connection open somehow?)? Our processing is working fine, just the images are coming too slowly for any useful processing.

Thanks for any help anyone can give.

Re: Speed of the camera
 

or maybe the system is limiting you to simulate what sort of transfer you get at the competition?

IDK just a thought...

Re: Speed of the camera
 

The overhead of the TCP stuff is not much. It would certainly not cause 1.5 secs of delay. My guess is that you have lots of panels open. Close all windows, open only the top panel and do your timings again. Another reason that you camera may be slow is because you didn't do the camera setup to create the FRC/FRC account.

Greg McKaskle

Re: Speed of the camera
 

I don't know specifically what this individual is doing, but I don't doubt his numbers. I considered writing the custom image handling code, but I ended up working on the rest of the bot instead. We found that at 15 FPS non-threaded we were reasonably good at tracking, though we still have some tweaking. So far, I've only put in about an hour to the camera tracking code, but it seems solid. I'm considering switching it to a separate thread though, since it has slowed down the unit significantly.

The one thing I have noticed looking through the image handling code, is that it is quite inefficient. There are multiple techniques that could be used to speed up the processing speed. Unfortunately, the "Image" struct wasn't all that well documented (that I could see), so I decided to just use the Target.cpp stuff.

I think that next pre-season I might write up a better library and share it here, hopefully we'll be able to get much better image processing performance than the WPI stuff.

Re: Speed of the camera
 

Ok I have a question then. I am not well versed at all in programming so I am asking the community. Is it possible to get adequate real time tracking using custom code on the cRio or is a coprocessor necessary? By adequate I am talking about being able to find a moving target while we are moving and directing our turret to follow the target without a delay. If this is possible to do on the cRio I would love to hear an answer.

Re: Speed of the camera
 

With custom code, there would be no problem with that. The cRio processor is MORE than capable of real-time-ish(obviously 30 FPS max) tracking of a moving object.

We are somewhat able to do it using the twoColorTrackingDemo code, though it isn't as smooth or as fast as we would necessarily want. One of the limitations is the efficiency of the search functions, which aren't terribly fast as implemented in the demo/included code.

I have done much more complex tracking code using a much worse processor in the past, with full 30 FPS.

Re: Speed of the camera
 

So more specifically, we have a turreted shooter capable of firing 4+balls per second. Would we be able to write a tracking code using the cRio that could follow a moving target very accurately allowing us to not miss many if any shots within say 7ft? It would also have to give us distance value at the same time.

Do you see any reason at all to use a coprocessor running an integrated Nvidia 9300 plus 2 much higher quality webcams? Or can the cRio do everything we are asking of it? Mechanically our shooter and turret can follow anything and we are trying to not handicap it in any way aka we want to be shooting at the 4 balls/ second rate constantly and be shooting on the fly not having to wait for our opponent to stop to fire.

Re: Speed of the camera
 

Here's my $0.02 coming from a control systems perspective...

Most people who want to use the camera for tracking are going to want to use the camera data as an input to a feedback loop (generally a PID controller). Analog (continuous) PID controllers are generally great at controlling systems with low-order dynamics. However, when we digitize the controller, we start to degrade this performance. You really want your sample rate to be 10-20x the highest frequency you want to track. With 15FPS from the camera, this means that a lot of high speed maneuvers can cause you to lose track of the target.

Obviously, getting the FPS up is going to help. But there are other things we can do.

Consider a PID loop where the input is the offset in the camera frame of the target centroid, and the output is the speed of the turret motor. This is the most direct way to do tracking, but it suffers from the drawbacks above. But what if rather than commanding motor speed, we command motor position?

Use an encoder or pot for a position control feedback loop on your turret (i.e. make it into a servo). Then use the camera to tell you where you need to point. Example: The centroid is 5 pixels left of center. You can do some trig to determine how many degrees this corresponds to. Now use your high-speed turret PID loop to go to that position. When another processed frame comes through, you can send a refined position. The only real challenge here is getting the turret sensor reading at the precise moment the image is captured, not when the processing results are ready. With a perfect mapping from pixel offset to degrees, as little as a single frame should be enough to get your shooter/dumper pointed in the right direction. Even if it isn't perfect, the refined commands as the camera captures more images will get you on target.

PID control for turret position. P control mapping camera data to turret position commands. Nested control systems are beautiful things.

Re: Speed of the camera
 

We have been experimenting with something just like this. I'm glad to see my ideas aren't too "out there".

PS: We have 3 nested control loops controlling the shooter's speed ;)

Re: Speed of the camera
 

Assuming a moving turret, it might be helpful to know when the image was grabbed on the camera, rather than when the image processing was completed. The Get icon for the image processing sleeps waiting for notification of an image arrival. The Get then decompresses the image. Noting the time at that point, before processing will give a timestamp a fixed amount away from the camera capture time. The image processing will not be very deterministic, so I wouldn't wait until after the processing.

The rough time estimates on the decoding of the jpeg are 8ms for small, 22 for medium and 100 for large. You can obviously measure this for more detail. The TCP transmission will be just a few additional ms, and then you are at the end of image capture time.

Another approach is to dig into the vision lib and write the timestamp into a global each time an image arrives. It would be great if the jpegs were timestamped. I'm pretty sure they aren't.

This timing probably doesn't matter much unless you want to calculate the lead amount for the turret. Then, knowing the image frame time for each position setpoint will help with the estimate where the next will be or correct for the time elapsed since the image get and perhaps take into account the response of the turret.

Anyway, those are hopefully ways of reducing the error caused by timing jitter caused by image processing.

Greg McKaskle

Re: Speed of the camera
 

Greg,

This is helpful advice.

Thanks

Re: Speed of the camera
 

I've uploaded the C++ and C# versions of Team 2152's image processing software. The C++ code compiles in our magical project, but has not been tested.

The C# program was made in Microsoft Visual Studio 2008. I'm not sure what framework this requires (3.5 perhaps). It performs all the actions that the C++ code does and contains the beta ball tracking software.

-TheDominis

Re: Speed of the camera
 

Thanks for sharing.