Go to Post oooh tight leather bumpers...sexy! - Nuttyman54 [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rating: Thread Rating: 2 votes, 5.00 average. Display Modes
  #1   Spotlight this post!  
Unread 23-02-2013, 22:40
wre136's Avatar
wre136 wre136 is offline
Registered User
AKA: Will Edds
FRC #2461 (The METAL-SKINS)
Team Role: Programmer
 
Join Date: Jan 2012
Rookie Year: 2002
Location: Oklahoma City
Posts: 30
wre136 is an unknown quantity at this point
Tracking Rectangles with Perspective Distortion

Hi,

I'm extremely new to vision tacking. I'm taking several pictures of an Ultimate Ascent rectangle goal (It matches the 2 point goal but is 60% the size due to lack of material) from different angles.

I'm trying to use the Vision Assistant and its shape matching function to find the shapes but it will only work if I am look straight at it and not at an angle.

I can use edge detection to detect the left, right, top and bottom edges of the rectangle and use the caliper function to find lines of intersection, angles and lengths which is very effective for giving me the info to determine distance and what not. However, then I run into the issue with having more than 1 rectangle on the screen and now edge detection no longer works.

I am trying to look into the "Vision Targets White Paper" (See attached document) and its section where it talks about Perspective Distortion (page 11). It shows where due to it, we cannot find shapes. But then by using "some work", we can detect shapes with the quadrilaterals. I'm looking for help on what this work is.

I've tried looking into Contour Analysis to try and get something I can use to match what the doc is doing but I am not making any ground on it.

I'm looking for any help on how to deal with this distortion issue or Vision Tracking in general with Vision Assistant. Thanks!
  #2   Spotlight this post!  
Unread 24-02-2013, 08:38
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,753
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Tracking Rectangles with Perspective Distortion

Can you post an image you are trying to process? It would help to know how much distortion we are talking about.

Greg McKaskle
  #3   Spotlight this post!  
Unread 24-02-2013, 14:49
wre136's Avatar
wre136 wre136 is offline
Registered User
AKA: Will Edds
FRC #2461 (The METAL-SKINS)
Team Role: Programmer
 
Join Date: Jan 2012
Rookie Year: 2002
Location: Oklahoma City
Posts: 30
wre136 is an unknown quantity at this point
Re: Tracking Rectangles with Perspective Distortion

I've attached the 5 pictures I am trying to use to do this with. Thanks!
Attached Thumbnails
Click image for larger version

Name:	20130219_20-28-37.jpg
Views:	50
Size:	9.5 KB
ID:	14159  Click image for larger version

Name:	20130219_20-29-06.jpg
Views:	16
Size:	9.6 KB
ID:	14160  Click image for larger version

Name:	20130219_20-29-40.jpg
Views:	20
Size:	9.7 KB
ID:	14161  Click image for larger version

Name:	20130219_20-30-14.jpg
Views:	18
Size:	9.4 KB
ID:	14162  Click image for larger version

Name:	20130220_20-50-36.jpg
Views:	34
Size:	11.6 KB
ID:	14163  

  #4   Spotlight this post!  
Unread 24-02-2013, 15:40
Alan Anderson's Avatar
Alan Anderson Alan Anderson is offline
Software Architect
FRC #0045 (TechnoKats)
Team Role: Mentor
 
Join Date: Feb 2004
Rookie Year: 2004
Location: Kokomo, Indiana
Posts: 9,113
Alan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond reputeAlan Anderson has a reputation beyond repute
Re: Tracking Rectangles with Perspective Distortion

I wouldn't worry about trying to make your code deal with the fifth image. A robot can't be high enough to get that view.
  #5   Spotlight this post!  
Unread 24-02-2013, 17:21
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,753
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Tracking Rectangles with Perspective Distortion

Assuming that the camera is an M1011, the default code described in the white paper gives a distance range of 12.0 to 9.9

Picture 1:
95, 96, 99, 68, 98 and distance of 11.5

Picture 2:
83, 93, 94, 59, 88 and distance of 10.2

Picture 3:
74, 96, 89, 53, 79 and distance of 9.9

Picture 4:
91, 95, 93, 77, 85 and distance of 12.0

As mentioned, Picture 5 has the camera above the image, but for grins, it is categorized as a center target scoring 86, 97, 85, 96, 57 and distance of 9.5.

This doesn't seem bad for a quick particle approach. In order to get the mask to work well, I changed to monochrome and chose Luminance as the plane and set the minimum value to about 220. If you intend to use color masking, you should set the exposure much lower so that your colors are more saturated. The tape is practically white.

The question is, do you need better distance calculations than that? You can get fancier with line fits or angle measurements, and let me know if you decide to.

Greg McKaskle
  #6   Spotlight this post!  
Unread 24-02-2013, 17:43
wre136's Avatar
wre136 wre136 is offline
Registered User
AKA: Will Edds
FRC #2461 (The METAL-SKINS)
Team Role: Programmer
 
Join Date: Jan 2012
Rookie Year: 2002
Location: Oklahoma City
Posts: 30
wre136 is an unknown quantity at this point
Re: Tracking Rectangles with Perspective Distortion

Greg,

I really appreciate the help you are giving me.

So two things. How did you detect the rectangles even though in some of the pictures they were not rectangles? I can get some of the same results as you but I have to use edge detection, points of intersections and then I can measure angles and distance. However, that approach fails when I introduce multiple rectangles into the picture. My goal would be to measure angles and lengths or multiple rectangles even if I'm looking at them at some extreme angle.

Did you use the simple Shape Detection feature or did you use something else to detect the distorted rectangles? If you used something else or somehow tweeked the shape detector feature, it would be great if you could direct me to what settings I can play with to detect distorted rectangles. Thanks again for the quick response!
Attached Thumbnails
Click image for larger version

Name:	Multiple Rectangles.jpg
Views:	30
Size:	4.4 KB
ID:	14169  Click image for larger version

Name:	Multiple Rectangles3.jpg
Views:	68
Size:	21.8 KB
ID:	14170  
  #7   Spotlight this post!  
Unread 24-02-2013, 20:17
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,753
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Tracking Rectangles with Perspective Distortion

I used the approach that was used for 2012 and the white paper. The approach is to inspect each particle to see if it is a good hollow rectangle. It does this by calculating a handful of simple measurements and scoring them to how a rectangle would score. The last step is to compare the scores to thresholds to determine how non rectangular you will allow the particles to be.

I'd suggest you compare your approach to the ones in the paper and identify the differences, the shortcuts, and things you'd like to modify.

If you have any questions, ask away.

Greg McKaskle
  #8   Spotlight this post!  
Unread 24-02-2013, 22:00
JewishDan18's Avatar
JewishDan18 JewishDan18 is offline
Registered User
FRC #1700
Team Role: Engineer
 
Join Date: Feb 2009
Rookie Year: 2007
Location: Sunnyvale, CA
Posts: 185
JewishDan18 is a splendid one to beholdJewishDan18 is a splendid one to beholdJewishDan18 is a splendid one to beholdJewishDan18 is a splendid one to beholdJewishDan18 is a splendid one to beholdJewishDan18 is a splendid one to behold
Re: Tracking Rectangles with Perspective Distortion

A quick (and very dirty) way to do it is to fit to parallelograms, and use the difference between the parallelogram and a rectangle with the same height and a fixed width that is dependent on the height (since you know the aspect ratio you are looking for in a target) to solve for additional information.
  #9   Spotlight this post!  
Unread 24-02-2013, 23:18
wre136's Avatar
wre136 wre136 is offline
Registered User
AKA: Will Edds
FRC #2461 (The METAL-SKINS)
Team Role: Programmer
 
Join Date: Jan 2012
Rookie Year: 2002
Location: Oklahoma City
Posts: 30
wre136 is an unknown quantity at this point
Re: Tracking Rectangles with Perspective Distortion

Greg,

The white paper I've been using so far is the one you are talking about. It has the info you were talking about with scoring rectangles.

However, my next question is how can I change those factors? I assume we are using the Shape Detection feature under Machine Vision and it does not give me the ability to change any of the factors I am reading about. If that isn't the case, please correct me.
  #10   Spotlight this post!  
Unread 24-02-2013, 23:20
wre136's Avatar
wre136 wre136 is offline
Registered User
AKA: Will Edds
FRC #2461 (The METAL-SKINS)
Team Role: Programmer
 
Join Date: Jan 2012
Rookie Year: 2002
Location: Oklahoma City
Posts: 30
wre136 is an unknown quantity at this point
Re: Tracking Rectangles with Perspective Distortion

Quote:
Originally Posted by JewishDan18 View Post
A quick (and very dirty) way to do it is to fit to parallelograms, and use the difference between the parallelogram and a rectangle with the same height and a fixed width that is dependent on the height (since you know the aspect ratio you are looking for in a target) to solve for additional information.
JewishDan18,

How do you fit the image to parallelograms? Is that some in Shape Detection or something else?
  #11   Spotlight this post!  
Unread 25-02-2013, 06:56
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,753
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Tracking Rectangles with Perspective Distortion

It intentionally doesn't use high level shape detection so that it is more accessible.
The rectangularity score is based on area/bounding rectangle.

The aspect ratio is based on the width and height of the bounding rect, and to make it a bit more robust to distortion, it also uses something called the equivalent rectangle.
The equivalent rectangle uses the area of the particle and the perimeter of the particle and solves for 2X+2Y=perimeter and X*Y=area.

The hollowness counts the pixels that are on for each vertical column of pixels and again for each horizontal row of pixels and compares those counts to thresholds that expect a strong outer band and weak inner band.

Each of these is scaled so that 100 is a good score and lower is not as good. The initial cutoffs are just based on some initial images and are very easy to change.

Can you think of other simple geometric measures that the code could score on the targets?

Greg McKaskle
  #12   Spotlight this post!  
Unread 25-02-2013, 14:27
wre136's Avatar
wre136 wre136 is offline
Registered User
AKA: Will Edds
FRC #2461 (The METAL-SKINS)
Team Role: Programmer
 
Join Date: Jan 2012
Rookie Year: 2002
Location: Oklahoma City
Posts: 30
wre136 is an unknown quantity at this point
Re: Tracking Rectangles with Perspective Distortion

Quote:
Originally Posted by Greg McKaskle View Post
It intentionally doesn't use high level shape detection so that it is more accessible.
The rectangularity score is based on area/bounding rectangle.

The aspect ratio is based on the width and height of the bounding rect, and to make it a bit more robust to distortion, it also uses something called the equivalent rectangle.
The equivalent rectangle uses the area of the particle and the perimeter of the particle and solves for 2X+2Y=perimeter and X*Y=area.

The hollowness counts the pixels that are on for each vertical column of pixels and again for each horizontal row of pixels and compares those counts to thresholds that expect a strong outer band and weak inner band.

Each of these is scaled so that 100 is a good score and lower is not as good. The initial cutoffs are just based on some initial images and are very easy to change.

Can you think of other simple geometric measures that the code could score on the targets?

Greg McKaskle
Greg,

What I would like to work out is some system that detects all edges on the image and if the edges seem to touch (or are with in a few pixels of each other), then they would be considered a corner. If 4 corners could be found consecutively, then we have a quadrilateral which is the shape we want to analyze.

The only thing that stinks about even trying to start my approach is that the edge finding ability of the vision assistant can only find 1 edge per run and you can only run it in the 4 main directions, left-right, right-left, up-down, down-up. Once each of those finds an edge, I cannot detect anymore edges unless I can find a way to subtract those edges from the image and then keep rerunning the edge detection. If I could at least do that, then I could detect the shapes and do the same measurements and calculations we are now. If you can think of anything about this, that would be helpful.

I do have a question about edge detection though. Does it work off of how many pixels in a column report an edge? For example, if all the pixels in column X say they have found an edge, then is the edge found? If some pixels in columns X, Y, Z (columns that are right next to each other) report an edge, then would that be an edge but be partially diagonal? If that is the case, I wonder if and how we can play with that so we can detect edges that are really skewed. Just thinking out loud.

Again, thanks for the help!
  #13   Spotlight this post!  
Unread 26-02-2013, 10:20
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,753
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Tracking Rectangles with Perspective Distortion

I opened the NI Example Finder, under the Help menu, and searched for Straight Edge. If you set the Number of lines to 2 and drag a rectangle around the image, it will locate both the top and bottom lines. You most likely want to use the mechanism in the white paper to find the bounding box of particles, then use the straight edge to get precise fits to the edges.

Again, this will be a bit tedious, your original image had stronger edges between the board and the background than the tape and board. So without a good bounding rectangle, the edge detection may do what you ask and not what you want.

Contours are another way to go with this. The attached image shows the contour and the graph to the right shows the sharp corner transition. You may want to use the bounding box to set the ROI to where you expect each corner to be and process then one at a time, and then recombine.

Greg McKaskle
Attached Thumbnails
Click image for larger version

Name:	Clipboard 1.png
Views:	59
Size:	472.8 KB
ID:	14192  

Last edited by Greg McKaskle : 26-02-2013 at 10:22. Reason: wrong image
  #14   Spotlight this post!  
Unread 03-04-2013, 02:47
faust1706's Avatar
faust1706 faust1706 is offline
Registered User
FRC #1706 (Ratchet Rockers)
Team Role: College Student
 
Join Date: Apr 2012
Rookie Year: 2011
Location: St Louis
Posts: 498
faust1706 is infamous around these partsfaust1706 is infamous around these parts
Re: Tracking Rectangles with Perspective Distortion

sorry about being so late on this reply:

Quote:
Originally Posted by Greg McKaskle View Post
.
The aspect ratio is based on the width and height of the bounding rect, and to make it a bit more robust to distortion, it also uses something called the equivalent rectangle.
How I went about this issue was also incorporating an aspect ratio. I took my 4 corners, 0, 1, 2, 3 which correspond to top left, top right, bottom left, and bottom right, respectively. I did (x^2 + y^2)^.5 to get distance between two points. Just the simple distance equation where x and y are pixel coordinates. I did that for the 4 sides. Then I added the top and bottom distances and divided them by the sum of the right and left.

If your program relies on being perpendicular to the target, meaning straight in front of it, then I'd suggest using image moments. Perspective images hardly alter the center of the target with this method.

What I did with this aspect ratio is differentiate between the 2 and 3 point target. The longer and thinner the target, the higher my aspect ratio. I created a simulation program that simulates the field (might I add with pyramid that I will be uploading on here probably tomorrow). I did this to easily decide what a good lower limit of the aspect ration is. I discovered 2.4 was the magic number. I also added an upper limit to the aspect ratio, which turned out to be 4. If the target I find does not have an aspect ration between these two values, I call it a two point then send the driver the image moment's x rotation so the robot may turn towards the 3 point, even if the camera is unable to detect it.
__________________
"You're a gentleman," they used to say to him. "You shouldn't have gone murdering people with a hatchet; that's no occupation for a gentleman."
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 00:01.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi