Go to Post The Tesla came in the blue tote. This is why you should always inventory your KOP before the replacement window closes! :p - sparrowkc [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Reply
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 01-07-2018, 09:51 PM
robokid5970 robokid5970 is offline
Registered User
FRC #5970
 
Join Date: Sep 2017
Location: Oregon
Posts: 4
robokid5970 is an unknown quantity at this point
New to object detection - team 5970

Our team programs in python 3.0 and have never used vision for either of the two competitions we've been in. This year we are attempting vision (given the significant ranking point incentive from Auto-Quest). I have been experimenting writing scripts for haar cascades (mine is training at the moment - stage 8) using OpenCV. I was wondering how this could be implemented on the robot and if any teams have experience with this kind of object detection in the past. We were considering running it via rasbian on the robot. What kind of camera would be best? Is speed an issue with this? Should I turn down saturation in OpenCV or value or some other attribute? Advice would be appreciated
Reply With Quote
  #2   Spotlight this post!  
Unread 01-07-2018, 10:17 PM
David Lame David Lame is offline
Registered User
FRC #0247
Team Role: Mentor
 
Join Date: Feb 2015
Rookie Year: 2014
Location: Berkley, MI
Posts: 195
David Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond repute
Re: New to object detection - team 5970

I'm not an expert, but....

My very limited experience with Haar cascades is that they aren't very useful in FIRST. In the off season after Stronghold, I tried to train a Haar cascade to recognize the balls from that season. Grey things about the size of a kickball. After generating the data sets and doing the training, it didn't work worth a hoot.

I looked into them a little bit, and noticed that most of the demos were about face recognition. I then looked a little bit into the actual recognizers, and saw that they were particularly well suited for some pretty structured objects, with particular patterns, but slight variations on those patterns. i.e. two eyes and a nose, but with eyes at different widths. For regular geometric objects, like balls, cubes, and lines, they didn't work as well.

I had encountered something similar way back in my college days at the University of Illinois in the mid '80s. I was into neural nets, and as a TA, gave a design to some undergrads for a neural net processor, based on material from a paper I had read (Something called WISARD, by some guy named Aleksandr), which could recognize faces. It worked awesomely, and wowed people who saw it, but it couldn't do anything except faces. (And it couldn't do many faces, but for the mid '80s, it was pretty darned impressive as a demo.)

In other words, I'm not sure Haar cascades will work all that well, although I would love to be wrong about that, especially if source code is provided. I'm going to try the sort of things that can be done by detecting lines, circles, polygonal approximations of contours, etc. I started working with some things today just fiddling with GRIP settings, and I'm pretty sure that will be adequate to find the location of cubes on the floor, or strings of red and blue lights.

(And,did I mention I'm not an expert? The work I've done with Haar cascades could best be described as "fiddling", not real work. It just didn't work for me, and in searching for papers, I couldn't find anything comparable to something I would want to try in FIRST. If someone came along with better experience, I would gladly listen.)
Reply With Quote
  #3   Spotlight this post!  
Unread 01-07-2018, 11:05 PM
robokid5970 robokid5970 is offline
Registered User
FRC #5970
 
Join Date: Sep 2017
Location: Oregon
Posts: 4
robokid5970 is an unknown quantity at this point
Re: New to object detection - team 5970

Thanks for the feedback. Do you know of a different method using opencv? Also, if not stated already, I would also not be classified as an expert. My experience in programming could be described as maybe one and a half days messing with these cascade scripts in python.
Reply With Quote
  #4   Spotlight this post!  
Unread 01-08-2018, 01:02 AM
Levansic's Avatar
Levansic Levansic is offline
Registered User
AKA: Len Evansic
FRC #0585 (Cyber Penguins)
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2008
Location: Tehachapi, CA
Posts: 214
Levansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud ofLevansic has much to be proud of
Re: New to object detection - team 5970

There is geek cred in doing a machine learning solution, but I would advise walking before running.

Since your team hasnít used vision before, play around a little with GRIP. At the very least, it exposes you to some of the concepts that OpenCV offers for doing basic vision.

Last year, our team leapt from using GRIP to using Python + OpenCV. With basic threshholding, smart use of camera settings, and a light ring, you can easily get the vision targets on the side of the switch. Simple and way less computationally expensive than Haar cascades.

We added a second mode to find and quantify position of the yellow gears that didnít have retroreflectance (HSV is your friend). A similar trick can be used to find the bright yellow covers on the milk crates. Again, this doesnít require exotic bits of OpenCV.

Simple and working beats ambitious and flakey every time.

Team 900 has a white paper that touches on these methods.
Iíd also recommend http://pyimagesearch.com as an excellent resource. Just be careful there as Dr. Rosebrock is now heavily into the deep learning aspects that are very cool, but overkill for this game.
Reply With Quote
  #5   Spotlight this post!  
Unread 01-08-2018, 06:26 AM
David Lame David Lame is offline
Registered User
FRC #0247
Team Role: Mentor
 
Join Date: Feb 2015
Rookie Year: 2014
Location: Berkley, MI
Posts: 195
David Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond reputeDavid Lame has a reputation beyond repute
Re: New to object detection - team 5970

Quote:
Originally Posted by robokid5970 View Post
Thanks for the feedback. Do you know of a different method using opencv? Also, if not stated already, I would also not be classified as an expert. My experience in programming could be described as maybe one and a half days messing with these cascade scripts in python.
I'll echo what Levansic said, and add a couple of elements.

Generally, you are looking for something specific. It might be the red or blue lights on the plates of the scale. It might be the retroreflective tape, illuminated by LEDs from an LED ring. It might be a crate.

You start with an HSV filter to get rid of things that are the wrong color. The next step is to find something the right shape. There are openCV functions to find lines, circles, and the best approximate polygon of a certain number of sides. For example, to find a square, you find everything of the right color, and then find polygons that approximate the outline of everything found, and then throw out everything that doesn't have four sides. (You can't generally apply an equal length side test initially, because that assumes you are looking at the square from the front.)

And to figure out how to do a lot of this? You play with Grip. Grip is a truly amazing program. Download it, take some pictures of the things you want to find, and start playing with settings until you see what looks like something close to what you want. Then have it generate some code for you by selecting "generate code" from the menu, and you can see the OpenCV functions used. (I don't think it has the polygon approximation thing. Google for cv2.approxPolyDP, or "find rectangles using OpenCV")

There's also a lot of code in all of these code reveal threads that have been posted. I'm afraid that we aren't reusing anything ourselves, so we didn't do a code reveal, but lots of other teams have. Hopefully, someone will pipe up with an easy to follow link that does something an awful lot like what you are thinking of doing.

That should get you started.
Reply With Quote
  #6   Spotlight this post!  
Unread 01-08-2018, 07:04 AM
gerthworm's Avatar
gerthworm gerthworm is offline
Making the 1's and 0's
FRC #1736 (Robot Casserole)
Team Role: Mentor
 
Join Date: Jan 2015
Rookie Year: 2015
Location: Peoria, IL
Posts: 532
gerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond repute
Re: New to object detection - team 5970

Quote:
Originally Posted by Levansic View Post
Simple and working beats ambitious and flakey every time.
This is probably the key to most of FRC, especially vision.

The challenge is usually that it's not just about detecting an object, it's about doing something useful and quickly with that info.
Reply With Quote
  #7   Spotlight this post!  
Unread 01-08-2018, 01:30 PM
lobrien lobrien is offline
Registered User
FRC #3880
Team Role: Mentor
 
Join Date: Jan 2016
Rookie Year: 2016
Location: Kailua Kona, Hawaii
Posts: 16
lobrien is an unknown quantity at this point
Re: New to object detection - team 5970

I'm going to echo the others and say that Haar cascades are unlikely to be the most efficient way to achieve your goals. Our CV pipeline last year was very basic: convert to HSV, blur (to smooth out the pixels), `inRange` to isolate the retroreflective tape echoing our green LEDs back to us, and then `findContours`to get our initial list of Regions of Interest.

I could _imagine_ Haar cascades for detecting non-retroreflective tape shapes (such as cube tracking) but you're trading off training time for iterating your solution. If I were facing that, I'd probably jump straight to Convolutional Neural Nets (CNNs) and deploy using a Movidius stick. But one way or the other, if you're using machine learning / differentiable programming, there are a lot of moving parts (data pipeline, training iterations, deployment process). I'm a big fan of jumping into the deep end, but there'd be very high technical risk associated with such a strategy.
Reply With Quote
  #8   Spotlight this post!  
Unread 01-08-2018, 01:40 PM
lobrien lobrien is offline
Registered User
FRC #3880
Team Role: Mentor
 
Join Date: Jan 2016
Rookie Year: 2016
Location: Kailua Kona, Hawaii
Posts: 16
lobrien is an unknown quantity at this point
Re: New to object detection - team 5970

Quote:
Originally Posted by robokid5970 View Post
Is speed an issue with this? Should I turn down saturation in OpenCV or value or some other attribute? Advice would be appreciated
On the specific technical questions: speed may or may not be an issue. We achieved 13-15FPS last year at 640x480 on an R Pi. That was sufficient for us and actually our big performance problem was latency from Network Tables! Network Tables is easy, but a challenge for targeting (which was what we were doing last year). This year, I think we'll use raw sockets.

We found that manually controlling the exposure was critical, since the retroreflective is _so_ effective that it blows out the pixels and they come across as white (and thus can be confused with background lights). We found an old Webcam that supported:

EXPOSURE = 0.05
self.cap = cv2.VideoCapture(video_device_index)
self.cap.set(cv2.cv.CV_CAP_PROP_BRIGHTNESS, EXPOSURE)

Once we were able to get hue from the retro tape, we switched to HSV, but note that OpenCV does the "H" in HSV from 0-180, not the 0-360 from typical HSV discussions, so:

# Convert to HSL so that we can focus in on green
# Note that OpenCV Hue ranges only to 180 (not 360), so use 1/2 values shown in graphics program
hsl = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

# Smooth out pixels
blurred = cv2.blur(hsl, (5, 5))

# Modify this as necessary
min_green = (40, 100, 100)
max_green = (80, 255, 255)
inrange_green = cv2.inRange(blurred, min_green, max_green)
Reply With Quote
Reply


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 01:14 PM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi