After a long period of debugging for the Dashboard camera images, I found a solution to the slow images. The graphs on the far right side of the Dashboard slow down the camera images dramatically. I removed those graphs from the Dashboard exe and it now runs at near live speed. The copy of the Dashboard currently running on our Classmate is attached
Dashboard.zip (132 KB)
Dashboard.zip (132 KB)
Interesting. I’ll load it up on ours today and see what happens.
I must be missing the obvious, but how does one replace the original Dashboard functionality with the dashboard application in the attached file?
After doing some profiling and testing the other day, I also have recommendations for video back to the dashboard.
If there is a big lag, ~5 seconds of lag, that is caused by a TCP backlog somewhere in the network from camera to cRIO to bridge to router to laptop. Clearly it could be happening anywhere a processor is bogged down, but the cases I was most easily able to reproduce was to set the camera to high frame rate, then run everything on one classmate so that it was fully bogged down. Using probes in LabVIEW I was able to see that the cRIO was sending images every 33ms over TCP, but the classmate was only displaying them every 100 ms or so. That means that the laptop TCP cache quickly fills with images and the lag builds to around five seconds with occasional jumps. To correct this BIG lag, do less on the cRIO. An upcoming driver station will assist in this as I found some UI things that could be sped up, but if you have other tools on the classmate and you look at the Task Manager and see the CPU fully loaded, that is the likely explanation for the lag.
As for the frame rate, it is highly affected by the jpeg image size being sent by the camera. Specifically, images larger than 16k in size are causing slow memory allocations, ~100ms, at various places in the WPI and development libraries. Below 16K, the allocations take much less time, ~5ms. By setting image compression on the camera you can see the effect.
To reproduce these results, turn off the continuous circle detection. For LV this means turning of the button on Robot Main. C and Java sample code do not do the detection until the button is pressed.
With the image size set to small 160x120, most any compression will achieve high frame rates, 20-30 Hz. The default of 5% is fine.
With medium image size, 320x240, the 5% image compression will result in a frame rate of around 8Hz at the classmate. Raising the compression to 20% should raise the frame rate dramatically as the image size falls below the 16K mark.
For large, 640x480, the compression number necessary to get the image size small enough is around 75%.
Similar numbers were seen to work for C++. Note that high compression numbers affect image quality, and this is not a linear affect. Allocations above 16K are all pretty slow, and below are all pretty fast. Plus, the camera will not and cannot produce images faster than 33ms, so super high compression will do nothing but distort the image. If you plan to use image processing to find ellipses, this will perform more reliably with low compression images. The numbers listed above should be a reasonable tradeoff of quality and speed, but feel free to experiment.
It may also be useful to enable and disable ellipse detection depending on the task being accomplished or on the mode of the match.
Greg McKaskle
One way is to copy the zipped files to C:\Program Files\FRC Dashboard
Another way is to put the files wherever you want and edit the file: C:\Documents and Settings\All Users\Documents\FRC DS Data Storage.ini
The line to edit will jump right out at you:
DashboardCmdLine=""C:\\Program Files\\FRC Dashboard\\Dashboard.exe""
Thanks Greg, this is good information. We have been seeing slow loop times, probably in the 100ms range, but not understanding why. Is there a way to know for sure the size of the image received from the camera?
In the dashboard display, just below the image, it lists the size, then the pixel the mouse is over, and finally the RGB of the pixel. That is commonly what I use.
Greg McKaskle
I don’t have the display in front of me but I think it shows the image size (160x120) not the 16K number that you referred to. Is there a way to see if we are above or below the 16K size? We changed to 160x120 and left the compression at 30% and it did run faster. We changed the compression to 5%, as you mentioned, and it slowed down again. It would be interesting to know the size being generated.
If you dig into the get image VI in the dashboard, there is a size indicator. If you want to display it on the dashboard, you’ll need to wire it out of each connector pane up to the top level.
Ah. I misunderstood what size you wanted. The JPEG image size is the string size coming from the TCP read on the attachment. To find the VI, you can Ctl-F, change to text, click the checkbox at the bottom to include vi.lib VIs, and search for 1.0.0.0 or something similar.
As for the difference at 5%. I saw a pretty strong correlation to the 16K number using LV. Joe Hershberger saw less of a correlation using C++. I believe his numbers started dropping earlier, but didn’t drop as fast. He and I were also running with a DS update which is still being tested and is not released. So, I’d pay less attention to the exact numbers and more to the fact that the image size, and thus the camera resolution and compression has a pretty big impact on frame rate back to the dashboard. A bigger impact than I’d thought.
Greg McKaskle
EDIT: Spoke too soon, nevermind 
This may seem like an odd question but does the size of the file change depending on the complexity of the image being processed. We had our C++ code running like it should, made a very simple change, like reading an analog value, and the processor suddenly slowed down. I was sending a printf to the console to check the analog value and I noticed it was slow and downloads started taking about 20 seconds instead of 4 or 5. I commented all of this out and it still runs slow. I remember when I first started last night the camera was pointed up toward the ceiling with very bright lights and a dark ceiling. A very simple image. At some point I moved the camera to look across the room which was a very busy image. Before I left last night I didn’t think to aim it back at the ceiling to see if the system speeded up. I did check the camera config and it had not changed (160x120, 30 compression, 10 frames/sec). Is there another setting that I’m missing? Any ideas?
I should also mention that we are using WPILib 4.0. We had a problem with 4.1 and have not tried 4.2 yet.
The image size definitely changes with image complexity, but I would guess that’s not the source of the slowdown. Perhaps you are getting a lot of particle detections that all need to be analyzed. Also, you should have your vision processing in an independent task, so as not to affect your robot control rates.
When you were seeing slow downloads, do you mean downloads of images or of new robot programs or what? If it was slow to run your code, you probably have some task that is chewing up the CPU and need to reboot your cRIO.
Another common source of slow downs is camera exposure. If you are shining a light directly into the lens, the camera can run full speed. If it’s a dim scene, it can make the image stream significantly slower.
-Joe
The acquisition of the image is done in a seperate task just like the demo code. The code that searches for targets is also very much like the demo code but was not running. If you are talking about searching for targets in a separate task, that would be great. Is there any example code available?
The slow down I was referring to is both downloading code to the cRIO and the execution of the code. It is what Greg was writing about in a previous reply where loop times can get to be around 100ms. There are no other tasks running that I can think of. The gyro if that’s considered a task.
We did some testing, which happened to be in a small room, to see how the camera setup affected loop speed and found it had a big effect. Last night I was in a much bigger room and, initially when the system was running normally, the camera was pointed toward some flourescent lights. After I repositioned the camera the image on the dashboard was much dimmer. Like I said my code change was very small and the only thing I can think of that would be chewing up CPU time is the camera task. Again, the target code was not running.
Steve
Joe you were right. It had nothing to do with the camera. Someone had commented out the Continuous functions in IterativeRobot and I forgot that I had uncommented them. We started again from code that we new worked and when I uncommented them the problem returned - slow downloads and slow execution times. We commented them out again and the problem has not returned (knock on wood). I’ve seen at least one other post about these same functions causing problems.
Steve
WOW thanks so much this fixed ours!
We just need a live camera on the dashboard for the drivers to see when they have the ball in the shooter.
I commented out or deleted all of the vision code on the robot other than in Begin.vi and modified the dashboard to just be not more than the “image get” vi.
It works quite well and we’re getting 20+FPS on the dashboard with a fraction of a second lag.