Go to Post If you are "retiring" the old ball, make all the lights green, hang it 8 feet in the air, line up a bunch of 'bots and let 'em shoot poofs. RoboPinata! (warning kids - don't eat the "candy" that falls out!) - Taylor [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
 
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 07-09-2009, 09:41
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: LabView/C++ equally capable?

Your initial question is a reasonable one. I suspect this will be a perennial topic aimed at identifying what will lead to more success, and a particular technology or approach is always an easy difference to gravitate towards -- similar to the wood vs aluminum, six wheels vs four discussions.

To make progress on improving vision, what part of the camera code worked, what didn't? Any questions that you want to ask?

Greg McKaskle
  #2   Spotlight this post!  
Unread 07-09-2009, 10:59
Ivan Helmrich's Avatar
Happy Birthday! Ivan Helmrich Ivan Helmrich is offline
Registered User
FRC #1775 (Tigerbytes)
Team Role: Mentor
 
Join Date: Nov 2008
Rookie Year: 2006
Location: Kansas City, MO
Posts: 73
Ivan Helmrich is a splendid one to beholdIvan Helmrich is a splendid one to beholdIvan Helmrich is a splendid one to beholdIvan Helmrich is a splendid one to beholdIvan Helmrich is a splendid one to beholdIvan Helmrich is a splendid one to behold
Re: LabView/C++ equally capable?

The problems I had with the vision code was frame rate and repeatable performance. Sometimes one color would be picked up, sometimes both. I tried the various changes in values that were suggested in a couple of threads here on CD but never got anything stable enough to rely on.

To be honest, we ended up bailing on trying to use vision for any automation about half way through the build. There were other, non-control issues that needed more attention. I had followed a couple of discussions on the camera but I couldn't dedicate the time to try all of the suggestions. Part of the reason for my original question was that I couldn't find anything in the LV implementation that that would have lead me to some of the methods people seemed to be talking about and so assumed, apparently wrongly, that the C++ code or perhaps comments was more descriptive of the process, less black box like, and that was part of the success some were having.

I was looking for a frame rate somewhere in the 20 fps range. I don't think we got out of single digits. The things I remember reading that should have helped are:
More light
Lower resolution
Turning off some things in the 2 color vi.
There was also some LV specific issue where the code would only process every other frame, I forget the specifics.

The purpose of the thread was to ask the question. If it will be instructive for many of us to change it to a discussion of how to get good camera performance, great. Since many teams are probably starting back up with the new school year, perhaps it is a good time to do this.

So, what really has to change in the 2 color demo to get the frame rate up? Once I can get that working, then I'll work on the reliability.

Thanks,

Ivan
__________________
“The knowledge of what is relevant, what is not and the skill to cleverly work around or approximate solutions is the essence of engineering.” - James C. King
  #3   Spotlight this post!  
Unread 07-09-2009, 13:17
Eugene Fang's Avatar
Eugene Fang Eugene Fang is offline
The Blue Alliance
no team
Team Role: Alumni
 
Join Date: Jan 2007
Rookie Year: 2000
Location: Bay Area, CA -> Pittsburgh, PA
Posts: 772
Eugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond repute
Re: LabView/C++ equally capable?

Quote:
Originally Posted by Ivan Helmrich View Post
So, what really has to change in the 2 color demo to get the frame rate up? Once I can get that working, then I'll work on the reliability.
Well, the main things we changed were setting the resolution to 160x120 (set in Begin.vi if you're using the advanced framework, i believe) and disabling the "mask display" in the 2 color demo. These two should improve your FPS.

A tip on getting the lighting to work under various lighting conditions:
for the HSL values, define H (hue) as narrow as possible but keep the S and L values relatively wide and set the "brightness" control to automatic (in Begin.vi). That way if the lighting changes you will still be able to track. It's kind of a balance between tracking robustness and picking up extra noise (although the noise sholudnt be that big of an issue because its unlikely that you will have two large blobs of "noise" of green and pink one on top of the other)

During the season we had it set on a fixed brightness and very closely defined HSL values. At SVR it worked great because of the lighting consistency but at Atlanta, the semi-transparent roof cover screwed us up badly.

Hope this helps!
__________________
Eugene Fang
2010 Silicon Valley Regional Dean's List Finalist

Various FLL Teams - Student (2000-2006), Mentor (2007-2010)
FRC Team 604 - Student (2007-2010), Mentor/Remote Advisor (2011-2015)
FRC Team 1323 - Mentor/Remote Advisor (2011-2014)

The Blue Alliance | TBA GameDay | TBA Android App
  #4   Spotlight this post!  
Unread 07-09-2009, 15:59
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: LabView/C++ equally capable?

There are a couple things that will affect frame rate. I'll cover the ones I remember, and then talk about why frame rate isn't necessarily that important.

Frame Rate:
One obvious thing that can limit frame rate is the frame rate setting. Setting it to a low number will delay the request for the frame. Setting it too high will request the next as soon as one arrives and will go as fast as other factors allow.

Another issue is the resolution. Each resolution change is a 4x pixels difference. 640x480 images are nearly 1MB bit and take 100ms simply to decompress. All processing will be about four times as expensive as the 320x240. The 320x240 images take about 22ms to decode, and this was the size I used for the examples. This was really just a built in performance handicap, and it is about 4x slower than the 160x120 image. The small image takes 8ms to decode and the processing will similarly be about four times faster.

The next issue, which affects LV more than C++ is the setup of the camera. If you don't add the FRC, FRC account on the camera, it takes multiple requests for the cRIO to get an image from the camera. The driver doesn't know which account will work, so it goes through three of them in sequence. For performance, you'd like it to succeed on the first, the FRC, FRC one.

The last issue has to do with various camera settings. The camera will lower the frame rate if it doesn't have enough light for a good exposure. The settings that affect this are the Exposure, Exposure Priority, and Brightness.

The other things mentioned such as the width of the hue will not have a large affect on performance, but since they will produce more blobs in the mask to analyze, they will have some affect. Also, the Saturation and Luminance will have some affect as well, since any pixels that can be eliminated by Sat or Lum are cheaper than having to do the calculations for Hue. Again, I think these settings are secondary for performance.

Performance isn't everything:
This may be counter intuitive, but FPS isn't really super important. More important is the lag, or the latency. This is defined as the time between when something happens in the real world, and when the image processing can notice it. It may seem that higher FPS would fix this, but think about how the awards shows have a 10 second delay to allow the censors to block things that aren't supposed to be televised. They don't change the FPS to do this, instead they buffer the images. The places images can be buffered include in the camera TCP stack, the cRIO TCP stack, and in the user's program. To measure the latency, I used the LED on the front of the cRIO itself, but you can use one off of a digital card if you'd prefer. Turn the LED on, and time the amount of time it takes for vision to receive an image with the LED on. Because the camera exposure and the LED will be unsynchronized, you'll need to look at numerous measurements and do some statistics to see how things behave.

When I measured this, both the 320x240 and 160x120 sizes had around 60ms of latency with the simplest processing I could have. Clearly this will go up as the processing becomes more complex. What this means is that everything the cRIO senses through the camera is really delayed by some amount based on the settings. For this year's processing, I think the amount was probably about 80ms. So by the time the cRIO "sees" something, it has already happened by about 80ms.

Why is this important? In order to hit a moving target, you don't want to shoot where something is. You certainly don't want to shoot where it used to be. You want to shoot where it will be. If the ball traveled instantaneously, you'd want to estimate relative velocity and aim about 80ms ahead. Of course the orbit balls are anything but instantaneous flyers, and the further away the target is, the longer the flight time. I dont' have any measured numbers, and it probably depends quite a bit on the delivery mechanism.

Anyway, the point is that a higher fps will give you a better estimate of the velocity, but will not allow you to ignore the latency issue.

I actually don't have a measurement for latency using C++. It is possible that the numbers are very different.

None of this performance related talk has anything to do with it seeing only one color or the other. Those are tuning issues. The camera has many different color settings for white balance, and lighting will change considerably from event to event. Tilting the target to and from the light will also affect the saturation quite a bit.

The best way to deal with these is to capture images and take them into vision assistant where you can do a line profile or look at a mask and come to understand how these environmental changes will affect the values that the camera will give you. Then you can try different things out to have the camera behave better, mount the camera better, etc. I put some images up on flicker last year that demonstrate some of the issues.

Greg McKaskle
  #5   Spotlight this post!  
Unread 11-09-2009, 16:16
Ivan Helmrich's Avatar
Happy Birthday! Ivan Helmrich Ivan Helmrich is offline
Registered User
FRC #1775 (Tigerbytes)
Team Role: Mentor
 
Join Date: Nov 2008
Rookie Year: 2006
Location: Kansas City, MO
Posts: 73
Ivan Helmrich is a splendid one to beholdIvan Helmrich is a splendid one to beholdIvan Helmrich is a splendid one to beholdIvan Helmrich is a splendid one to beholdIvan Helmrich is a splendid one to beholdIvan Helmrich is a splendid one to behold
Re: LabView/C++ equally capable?

Greg, thanks for the information. The regular day job got in the way again, finally got the camera set up and working again last night. What you say about latency vs. frame rate makes good sense. I'm a little stumped about how to measure it though. I can see it on the display when I move the camera but I'm not following what you said about using an LED to get timing. Did you just use a stopwatch?

Ivan
__________________
“The knowledge of what is relevant, what is not and the skill to cleverly work around or approximate solutions is the essence of engineering.” - James C. King
  #6   Spotlight this post!  
Unread 12-09-2009, 16:06
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: LabView/C++ equally capable?

I'll see if I can find the test code. It was basically pointing the camera at the cRIO LEDs, the ones near the power and ethernet plugs. There is an RT function for controlling the LED, and I figured it was close to instantaneous compared to the camera, so I turned it off, and started the camera to acquiring.

At some point, I'd turn the LED on, record the time on the cRIO, then loop inspecting images until one showed up with the LED lit. To detect the LED being lit, I use the regular Camera Get which uncompresses the image, then I measured the intensity of a pixel over the LED. A that point, turn the LED back off, wait for things to settle, and do it over again. I decided to wait a random amount with the LED off. This gave me a pretty good statistical picture of the latency. It shows the minimum, the typical and the maximum time you could expect for a given camera setup.

Greg McKaskle
  #7   Spotlight this post!  
Unread 14-09-2009, 20:54
sami9145's Avatar
sami9145 sami9145 is offline
Registered User
AKA: Samantha Gainey
FRC #2333
Team Role: Mentor
 
Join Date: Sep 2009
Rookie Year: 2009
Location: Sapulpa OK
Posts: 40
sami9145 will become famous soon enough
Re: LabView/C++ equally capable?

you know you can use Java right?
__________________
FRC Team 2333
2008-2009 team member
2009-2010 co-captain

FTC Team 3679
2009-2010 team captain


  #8   Spotlight this post!  
Unread 14-09-2009, 21:19
EricH's Avatar
EricH EricH is offline
New year, new team
FRC #1197 (Torbots)
Team Role: Engineer
 
Join Date: Jan 2005
Rookie Year: 2003
Location: SoCal
Posts: 19,825
EricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond repute
Re: LabView/C++ equally capable?

Quote:
Originally Posted by sami9145 View Post
you know you can use Java right?
Last year, you couldn't. The only data we have is from last year (other than beta test data from this year). Therefore, we have to make this comparison until such time as Java camera code becomes available through some means.
__________________
Past teams:
2003-2007: FRC0330 BeachBots
2008: FRC1135 Shmoebotics
2012: FRC4046 Schroedinger's Dragons

"Rockets are tricky..."--Elon Musk

  #9   Spotlight this post!  
Unread 14-09-2009, 22:09
sami9145's Avatar
sami9145 sami9145 is offline
Registered User
AKA: Samantha Gainey
FRC #2333
Team Role: Mentor
 
Join Date: Sep 2009
Rookie Year: 2009
Location: Sapulpa OK
Posts: 40
sami9145 will become famous soon enough
Re: LabView/C++ equally capable?

Quote:
Originally Posted by EricH View Post
Last year, you couldn't. The only data we have is from last year (other than beta test data from this year). Therefore, we have to make this comparison until such time as Java camera code becomes available through some means.
that makes sense. we have no data from last year cuz all our programers graduated and didnt bother passing on their knowlege. so we gotta start from scratch. and we only know how to text program not visual so my team is gunna use java
__________________
FRC Team 2333
2008-2009 team member
2009-2010 co-captain

FTC Team 3679
2009-2010 team captain


  #10   Spotlight this post!  
Unread 15-09-2009, 10:53
Alan Anderson's Avatar
Alan Anderson Alan Anderson is offline
Software Architect
FRC #0045 (TechnoKats)
Team Role: Mentor
 
Join Date: Feb 2004
Rookie Year: 2004
Location: Kokomo, Indiana
Posts: 9,113
Alan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond repute
Re: LabView/C++ equally capable?

Quote:
Originally Posted by sami9145 View Post
...all our programers graduated and didnt bother passing on their knowlege. so we gotta start from scratch. and we only know how to text program not visual so my team is gunna use java
If you're starting from scratch, don't dismiss LabVIEW out of hand. For someone new to programming for FRC robots, I think it's a whole lot faster to learn.
  #11   Spotlight this post!  
Unread 15-09-2009, 11:06
EricH's Avatar
EricH EricH is offline
New year, new team
FRC #1197 (Torbots)
Team Role: Engineer
 
Join Date: Jan 2005
Rookie Year: 2003
Location: SoCal
Posts: 19,825
EricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond reputeEricH has a reputation beyond repute
Re: LabView/C++ equally capable?

Quote:
Originally Posted by Alan Anderson View Post
If you're starting from scratch, don't dismiss LabVIEW out of hand. For someone new to programming for FRC robots, I think it's a whole lot faster to learn.
On the other hand, if they already know Java, it might be a whole lot faster to learn how to adapt it to robot programming. Ditto for C++.
__________________
Past teams:
2003-2007: FRC0330 BeachBots
2008: FRC1135 Shmoebotics
2012: FRC4046 Schroedinger's Dragons

"Rockets are tricky..."--Elon Musk

  #12   Spotlight this post!  
Unread 15-09-2009, 11:21
EricVanWyk EricVanWyk is offline
Registered User
no team
 
Join Date: Jan 2007
Rookie Year: 2000
Location: Boston
Posts: 1,597
EricVanWyk has a reputation beyond reputeEricVanWyk has a reputation beyond reputeEricVanWyk has a reputation beyond reputeEricVanWyk has a reputation beyond reputeEricVanWyk has a reputation beyond reputeEricVanWyk has a reputation beyond reputeEricVanWyk has a reputation beyond reputeEricVanWyk has a reputation beyond reputeEricVanWyk has a reputation beyond reputeEricVanWyk has a reputation beyond reputeEricVanWyk has a reputation beyond repute
Send a message via AIM to EricVanWyk
Re: LabView/C++ equally capable?

Quote:
Originally Posted by sami9145 View Post
that makes sense. we have no data from last year cuz all our programers graduated and didnt bother passing on their knowlege. so we gotta start from scratch. and we only know how to text program not visual so my team is gunna use java
Not knowing how to do something is a poor excuse for not doing it. Give it a shot before you dismiss it. You might like it, you might hate it, but you'll be better off for having tried it.
  #13   Spotlight this post!  
Unread 15-09-2009, 14:05
sami9145's Avatar
sami9145 sami9145 is offline
Registered User
AKA: Samantha Gainey
FRC #2333
Team Role: Mentor
 
Join Date: Sep 2009
Rookie Year: 2009
Location: Sapulpa OK
Posts: 40
sami9145 will become famous soon enough
Re: LabView/C++ equally capable?

weve been trying to figure it out but we havent figured it out. we can make the computer make a random noise but thats about it.
were just gunna stick with robotc and labview.
__________________
FRC Team 2333
2008-2009 team member
2009-2010 co-captain

FTC Team 3679
2009-2010 team captain


  #14   Spotlight this post!  
Unread 15-09-2009, 16:36
rsisk's Avatar
rsisk rsisk is online now
The GURU Channel
AKA: Richard Sisk
FRC #2493 (Robokong)
Team Role: Mentor
 
Join Date: Jan 2008
Rookie Year: 2007
Location: Riverside, CA
Posts: 2,750
rsisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond reputersisk has a reputation beyond repute
Send a message via MSN to rsisk
Re: LabView/C++ equally capable?

Quote:
Another issue is the resolution. Each resolution change is a 4x pixels difference. 640x480 images are nearly 1MB bit and take 100ms simply to decompress. All processing will be about four times as expensive as the 320x240. The 320x240 images take about 22ms to decode, and this was the size I used for the examples. This was really just a built in performance handicap, and it is about 4x slower than the 160x120 image. The small image takes 8ms to decode and the processing will similarly be about four times faster.
At what point does a decreased resolution become unusable? Obviously a 1 pixel resolution will not do much good, but decreasing from 604x480 to 160x120 seems like a significant improvement.
  #15   Spotlight this post!  
Unread 26-09-2009, 18:54
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: LabView/C++ equally capable?

The Axis 206 camera supports three resolutions. These three steps will be the best improvements overall. To get other sizes, you'll end up decimating or subsetting one of the larger sizes. This is still often worth it.

As for when resolution cuts are no longer worth it, in the real world, the nice cameras have an amazing flexibility with resolution, even to the point of having an image one pixel tall and X pixels wide. This may be useful when material is moving on a conveyer beneath the camera and you really only have time to process the new material.

Greg McKaskle
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Labview 8.5 vs. FRC Labview 8.5 pyr0b0y NI LabVIEW 0 01-02-2009 22:20
Labview tseres Programming 2 23-05-2007 00:27
can bots with omni wheels capable of going up robots? Tri_Lam Technical Discussion 12 13-01-2007 16:15
Is this hardware capable of balance? sanddrag Electrical 15 02-08-2005 15:36


All times are GMT -5. The time now is 02:00.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi