OCCRA
Go to Post "He who seeks originality will often find novelty, but not great design." - IKE [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Reply
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 07-27-2018, 01:29 AM
Mills_AXY Mills_AXY is offline
Registered User
FRC #0253
 
Join Date: Sep 2016
Location: Millbrae
Posts: 2
Mills_AXY is an unknown quantity at this point
Help with vision to align with cubes

This year, we had an issue with the intake on our robot, wherein it would only intake cubes whose sides were perfectly parallel to the sides of the intake arms. Cubes in other orientations would get stuck at a 45 degree angle and become impossible to correct. This meant we had to align perfectly with each cube to intake it. This was simple when we had a direct line of sight with the robot, but got significantly harder the further the robot got from the alliance station.

It was also a huge waste of time to have to drive around the cube to approach it from a proper angle. I was wondering if it's possible to use vision to align with these cubes (not just drive up to a cube, but to drive up at an angle where it would be possible to intake the cube).

Understandably, the simplest solution would be to amend the intake design to correct this issue. However, we do not currently have access to our build space, and as a result can only make minimal changes to the robot.

Any ideas?
Reply With Quote
  #2   Spotlight this post!  
Unread 07-27-2018, 01:33 AM
Brian Maher's Avatar
Brian Maher Brian Maher is offline
#JusticeForPinecone
FRC #2791 (Shaker Robotics)
Team Role: College Student
 
Join Date: Apr 2014
Rookie Year: 2012
Location: Troy, NY
Posts: 1,215
Brian Maher has a reputation beyond reputeBrian Maher has a reputation beyond reputeBrian Maher has a reputation beyond reputeBrian Maher has a reputation beyond reputeBrian Maher has a reputation beyond reputeBrian Maher has a reputation beyond reputeBrian Maher has a reputation beyond reputeBrian Maher has a reputation beyond reputeBrian Maher has a reputation beyond reputeBrian Maher has a reputation beyond reputeBrian Maher has a reputation beyond repute
Re: Help with vision to align with cubes

If you don't have access to your build space, are you going to be able to test and refine your software solution?

Also, this sounds like an incredibly overkill software fix to what sounds like a fairly straightforward mechanical problem, especially in the post season when numerous teams have released their CAD. If you want CAD for ours, feel free to PM me.
__________________
2016-present, Mentor, FRC 2791 - Shaker Robotics
2018: Central New York Winner (2791, 340, 5030, 7081), Tech Valley Finalist (2791, 20, 3624, 5123), Darwin Division QF (4946, 2791, 2834, 3461), IRI Invite
2017: Tech Valley Winner (333, 2791, 5952), NYC SF (2791, 2383, 1880), Curie Division QF (2481, 2791, 203, 1807), IRI Invite, Robot Rumble Winner (2791, 5881, 1880)
2016: Tech Valley SF (5236, 2791, 3624), Finger Lakes SF (5254, 2791, 2383), Battlecry Winner (195, 2791, 501), Robot Rumble Winner (2791, 195, 6463)

2012-2015, Student, FRC 1257 - Parallel Universe
2015: Mount Olive QF (1257, 1923, 1811) , North Brunswick Finalist (11, 193, 1257)
2014: Clifton Winner (1626, 869, 1257), MAR CMP QF (1257, 293, 303)
2012: Mount Olive QF (204, 303, 1257)

2017-present, SLFF - Questionable Decisionmakers
2018: Regular Season Finalist, Championship 3rd Place, Fantasy FIM Finalist
2017: Regular Season Winner, Championship Winner, Fantasy FIM Finalist, IRI Winner

Last edited by Brian Maher : 07-27-2018 at 01:36 AM.
Reply With Quote
  #3   Spotlight this post!  
Unread 07-27-2018, 02:19 AM
UnofficialForth's Avatar
UnofficialForth UnofficialForth is offline
Registered User
FRC #1646 (Precision Guessworks)
Team Role: Mentor
 
Join Date: Mar 2017
Rookie Year: 2010
Location: Indiana
Posts: 57
UnofficialForth is a splendid one to beholdUnofficialForth is a splendid one to beholdUnofficialForth is a splendid one to beholdUnofficialForth is a splendid one to beholdUnofficialForth is a splendid one to beholdUnofficialForth is a splendid one to beholdUnofficialForth is a splendid one to behold
Re: Help with vision to align with cubes

This is definitely possible, though I would heed the warning of those who think it is going to be much harder compared to just changing the mechanical design.

Your options:
Just a camera
To simplify positioning from a far distance, purely using a camera to allow the drivers to see a first-person view of the robot and its intake might be enough without going through the additional hassle of auto-align code. This would give you "direct line of sight" for anything in front of your robot. This I suggest being your first goal, as just setting up a camera and sending feedback to the drivers can be a large task in itself. I'll include a couple resources at the end of this post.

Grip
Grip is a wonderful software for easy vision detection code. FIRST actually has a couple decent tutorials on their website as to how to use different filters. Unfortunately, they don't talk too much about how it's directly implemented into code. Never less, once you can figure that out, you can get values such as the x and y position of the cube on your camera, and the width of the cube. The width can be used to determine what angle the cube is at. If you are approaching from a corner, the cube has the widest width; if you approach directly from a side it has the thinnest width. You can use this to determine what angle the cube is at. The largest issue with this method is that you still need to "drive" around the cube, which you seemed hesitant of, due to the time it takes.

The above are two methods of rectifying your situation, one which is much more difficult than the other. Putting a camera on to see a first-person view for cube line up seems like a good idea, but I would definitely post either a CAD or image of your intake design along with what tools you have available so we of Chief can help you determine the best solution to your issue. Feel free to contact me! Good luck!
__________________
2010-2013 Nerd Herd Lego League Student
2014-2017 Lafayette Area Lego League Mentor
2014-2017 FRC Team 1747 Student
2016-2017 FTC Team 10133 Student Mentor
2018-Present FRC Team 1646 College Mentor
Current: Purdue FIRST Program's 1646 Director
Reply With Quote
  #4   Spotlight this post!  
Unread 07-27-2018, 05:34 AM
wsh32's Avatar
wsh32 wsh32 is offline
Registered User
AKA: Wesley Soo-Hoo
FRC #0687 (The Nerd Herd) & FRC #4201 (The Vitruvian Bots)
Team Role: Mentor
 
Join Date: Sep 2014
Rookie Year: 2014
Location: Los Angeles, CA
Posts: 54
wsh32 is a jewel in the roughwsh32 is a jewel in the roughwsh32 is a jewel in the rough
Re: Help with vision to align with cubes

Using vision processing to autonomously align with cubes is tricky...there have been a few teams that have gotten cube tracking down, but most that I have seen only track the locations of the cube, rather than the orientation of the cube. If a vision processing solution is what you decide to proceed with, I would suggest looking into a deep learning solution such as YOLO to do your tracking and processing.

Getting the robot to use the tracking data is a different problem altogether. Unless you have an omnidirectional drivetrain of sorts, your robot will have to drive all the way around the cube, and unless your camera is on a pan/tilt system, the cube will probably leave the field of view of the camera.

Like previously stated, you also have the option to have a camera to "guide" your driver a little bit. Unless you are willing to go down the rabbit hole of deep learning vision systems or get access to your shop, this might be the safest bet.
__________________





FRC 4201 (2018- ) -- Mentor
FRC 687 (2014-) -- Mentor, Team Captain
Los Angeles Regional 2018 3rd seed, 2nd alliance, Semifinalists
Las Vegas Regional 2018 7th seed, 4th alliance captain, Finalists
Newton Division 2018 15th seed, 5th alliance, Semifinalists
Los Angeles Regional 2017 Dean's List Finalist
Reply With Quote
  #5   Spotlight this post!  
Unread 07-27-2018, 06:34 AM
gerthworm's Avatar
gerthworm gerthworm is offline
Making the 1's and 0's
FRC #1736 (Robot Casserole)
Team Role: Mentor
 
Join Date: Jan 2015
Rookie Year: 2015
Location: Peoria, IL
Posts: 640
gerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond reputegerthworm has a reputation beyond repute
Re: Help with vision to align with cubes

Quote:
Originally Posted by Mills_AXY View Post
I was wondering if it's possible to use vision to align with these cubes (not just drive up to a cube, but to drive up at an angle where it would be possible to intake the cube).
Possible? In theory, maybe? But....

With a camera, I would rank this problem as "extremely difficult", as identifying the edges & faces of the (single color) cube with vision processing seems hard to do. Very dependent on the lighting situation. Most folks this year only did detection of the centroid of the cube - AKA, their algorithm was designed to detect the center of a blob of yellow. Finding edges & faces will require a much more sophisticated algorithm.

2D Lidar would likely be a better sensor solution for detecting cube orientation relative to the robot.

Also, does your robot have the ability to strafe side to side? Without this, the motion required to re-approach a cube from the right vector is going to require some path planning, as well as take a good chunk of time.

With a lidar and strafe ability and access to the robot, the problem seems "difficult but possible to solve".

However, without access to the robot in a space where you could test, I rank the problem as "nearly impossible".

TL;DR - probably better to fix the mechanism.

Last edited by gerthworm : 07-27-2018 at 06:37 AM.
Reply With Quote
  #6   Spotlight this post!  
Unread 07-27-2018, 08:26 AM
rlance's Avatar
rlance rlance is offline
Registered User
FRC #5550
 
Join Date: Dec 2015
Location: Bethany, OK
Posts: 146
rlance has a reputation beyond reputerlance has a reputation beyond reputerlance has a reputation beyond reputerlance has a reputation beyond reputerlance has a reputation beyond reputerlance has a reputation beyond reputerlance has a reputation beyond reputerlance has a reputation beyond reputerlance has a reputation beyond reputerlance has a reputation beyond reputerlance has a reputation beyond repute
Re: Help with vision to align with cubes

We had the same issue with the cube getting stuck at 45 degrees and started messing around with having the intake rotate the cube. We eventually had the wheel on one side intaking at about 60% and the other side outtaking at 40%, which caused the cube to rotate the way we wanted. We ended up adding a mechanical switch to the intake to detect when the cube was rotated to a point where we could then have both sides intake the cube and secure the cube.
Reply With Quote
  #7   Spotlight this post!  
Unread 07-27-2018, 09:41 AM
Loveless Loveless is offline
Registered User
no team
Team Role: Engineer
 
Join Date: Mar 2017
Rookie Year: 2011
Location: St Louis
Posts: 69
Loveless has a spectacular aura aboutLoveless has a spectacular aura about
Re: Help with vision to align with cubes

Quote:
Originally Posted by wsh32 View Post
I would suggest looking into a deep learning solution such as YOLO to do your tracking and processing.
I wouldn't. Just use a depth camera.
Reply With Quote
  #8   Spotlight this post!  
Unread 07-27-2018, 02:19 PM
Clayton Yocom's Avatar
Clayton Yocom Clayton Yocom is offline
Programming Mentor
FRC #0027 (RUSH)
Team Role: Mentor
 
Join Date: Jan 2011
Rookie Year: 2011
Location: Clarkston, MI
Posts: 174
Clayton Yocom has a spectacular aura aboutClayton Yocom has a spectacular aura about
Send a message via AIM to Clayton Yocom Send a message via MSN to Clayton Yocom Send a message via Yahoo to Clayton Yocom
Re: Help with vision to align with cubes

Quote:
Originally Posted by Mills_AXY View Post
This year, we had an issue with the intake on our robot, wherein it would only intake cubes whose sides were perfectly parallel to the sides of the intake arms. Cubes in other orientations would get stuck at a 45 degree angle and become impossible to correct.
...
Any ideas?
Our solution was to detect when this happens by sensing the larger current load on the motors, spin the cube one direction, then continue intaking.

You can see it in action here

Here it is in action during our autonomous mode
__________________
Member of FRC Team 45 TechnoKats : 2011 - 2016
Member of FRC Team 27 Team RUSH : 2017 - Present
Reply With Quote
  #9   Spotlight this post!  
Unread 07-27-2018, 03:22 PM
GeeTwo's Avatar
GeeTwo GeeTwo is offline
Mentor
AKA: Gus Michel II
FRC #3946 (Tiger Robotics)
Team Role: Mentor
 
Join Date: Jan 2014
Rookie Year: 2013
Location: Slidell, LA
Posts: 5,824
GeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond reputeGeeTwo has a reputation beyond repute
Re: Help with vision to align with cubes

I concur with many that using vision to align with the cubes is likely to be engineering a million dollar solution to a hundred dollar problem.

We didn't have a problem nearly as bad as yours, but if we hit the cube at nearly 45 degrees, the cube will occasionally get stuck between the wheels. When this happens, we have buttons (though it could be done autonomously) which rotate the intakes on both sides the same direction (one does right, the other left). This gives the cube a nice twist. Usually we wind up losing immediate control of the cube, but it ends up in front of us well aligned for an intake.

As an autonomous-type solution to this intake problem, you may be able to put some IR rangefinders (ultrasonic would be too long-range and low-res for this unless you spent a lot more money), one inside your left intake and one inside your right intake, looking forward and sufficiently inward to not see the intake. Then, reduce speed to the intake on the side where the cube is closer, giving the other time to "catch up". From overhead, it would look something like this for a four wheel intake, with the carets showing the rangefinders.

Code:
   |O          O|
   |            |
   |O          O|
   |___^____^___|
__________________

If you can't find time to do it right, how are you going to find time to do it over?
If you don't pass it on, it never happened.
Robots are great, but inspiration is the reason we're here.
Friends don't let friends use master links.
[Quoting brennonbrimhall]: We design a new robot every year, but we can't forget that we also design a new team every year as folks come and go.
Reply With Quote
  #10   Spotlight this post!  
Unread 07-27-2018, 03:40 PM
jtrv's Avatar
jtrv jtrv is offline
The Stupid District
AKA: Justin
FRC #5254 (HYPE)
Team Role: Team Spirit / Cheering
 
Join Date: Jan 2013
Rookie Year: 2012
Location: Rochester, NY
Posts: 512
jtrv has a reputation beyond reputejtrv has a reputation beyond reputejtrv has a reputation beyond reputejtrv has a reputation beyond reputejtrv has a reputation beyond reputejtrv has a reputation beyond reputejtrv has a reputation beyond reputejtrv has a reputation beyond reputejtrv has a reputation beyond reputejtrv has a reputation beyond reputejtrv has a reputation beyond repute
Re: Help with vision to align with cubes

Jumping on the bandwagon that vision is not what I would consider the go-to solution.

The dead simplest solution, IMO, is to map a button on your controller to spin one side of the intake slightly faster than the other. You should experiment with the actual values of these speeds, since it will largely vary based on the intake.

If the intake wheels are not actually touching the cube when it's jammed, then you have a bigger problem. This is one of the times where I would say "this isn't a code problem, but a mechanical problem." But you can't really fix a mechanical problem right now.

A possible software fix for this more complex issue is to have the robot drive backwards 1-6 inches while your intake is spinning at different speeds, then either drive forward at some point during the command or once you've fixed the cube orientation (by manually looking at it).
Reply With Quote
  #11   Spotlight this post!  
Unread 07-27-2018, 04:51 PM
Anon10W1z Anon10W1z is offline
Registered User
FRC #3256 (WarriorBorgs)
Team Role: Programmer
 
Join Date: Jan 2017
Rookie Year: 2017
Location: San Jose
Posts: 91
Anon10W1z is on a distinguished road
Re: Help with vision to align with cubes

Our software solution to this problem was (upon the robot detecting that itís intaking a cube) running the wheels in reverse to outtake and then running the wheels to intake after a very short duration.
__________________
My views and opinions expressed on these forums do not necessarily reflect the views of my team or other team members.




2017 (Team 3256) - Central Valley Semifinalist (with 5852 and 4543), San Francisco Finalist (with 604 and 2035), Idaho Quarterfinalist (with 6546 and 3250) + Chairman's, Carver Quarterfinalist (with 4334, 2471, and 599)
2018 (Team 3256) - San Diego Semifinalist (with 1572 and 2984), Sacramento Quarterfinalist (with 5507 and 5817), Idaho Semifinalist (with 1983 and 4598)
Reply With Quote
  #12   Spotlight this post!  
Unread 07-27-2018, 10:28 PM
jhill42 jhill42 is offline
Registered User
FRC #1747
 
Join Date: Apr 2018
Rookie Year: 2015
Location: IN
Posts: 8
jhill42 is an unknown quantity at this point
Re: Help with vision to align with cubes

From what I understand (although I did not work on this area of our programming this year), we were able to get software that could accurately generate a profile to line up with and pick up a cube using some overpowered libraries, but it was not fast enough to be feasible to use in competition.
Reply With Quote
  #13   Spotlight this post!  
Unread 07-28-2018, 11:38 AM
billbo911's Avatar
billbo911 billbo911 is offline
I prefer you give a perfect effort.
AKA: That's "Mr. Bill"
FRC #2073 (EagleForce)
Team Role: Mentor
 
Join Date: Mar 2005
Rookie Year: 2005
Location: Elk Grove, Ca.
Posts: 2,645
billbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond repute
Re: Help with vision to align with cubes

OK, so you've read all the "Vision is not the best solution" answers, and I agree!
That said, you asked about vision, so I will try to help answer that piece.

The simplest solution to using vision is to stream a live image to the DS.

Now for target detection, the fastest results seem to come from detecting the cube via a simple color detection. Deep learning in it's various forms can work, but mostly just render location. The are also a bit slower and require more processing power. This is not to say anything negative about deep learning, but in it's current iterations, HSV just gives better results in our application.

One option is to take the aspect ratio of the detected cube. Then use it to determine if you are squared up to the target.
__________________
CalGames 2009 Autonomous Champion Award
2012 Sacramento Semi-Finals, 2012 Sacramento Innovation in Control Award, 2012 SVR Judges Award.
2012 CalGames Autonomous Challenge Award winner ($$$).
2014 2X Rockwell Automation: Innovation in Control Award (CVR and SAC).
Curie Division Gracious Professionalism Award.
2014 Capital City Classic Winner AND Runner Up. Madtown Throwdown: Runner up.
2015 Innovation in Control Award, Sacramento.
2016 Chezy Champs Finalist, 2016 MTTD Finalist
2017 Utah Regional Winner!, Sacramento Finalist
Innovation in Control Newton/Carver Divisions , Newton #5 Captain
2018 WFFA Sacramento, Creativity Award Galileo-Robling Divisions
Reply With Quote
  #14   Spotlight this post!  
Unread 07-28-2018, 01:18 PM
Prateek M's Avatar
Prateek M Prateek M is offline
Kotlin Master Race
FRC #5190 (Green Hope Falcons)
Team Role: Programmer
 
Join Date: May 2018
Rookie Year: 2018
Location: Cary, North Carolina
Posts: 33
Prateek M is on a distinguished road
Re: Help with vision to align with cubes

Quote:
Originally Posted by billbo911 View Post
One option is to take the aspect ratio of the detected cube. Then use it to determine if you are squared up to the target.
We were messing around with something similar with the JeVois camera for Vision. We found some problems with aspect ratios and detecting distance using simple trig when the cube was at too much of an angle, or if the cube was too close to the robot where the height of the cube began to get cut off.

Eventually, though, we went for a trajectory tracking approach that accurately took us to the switch fence cubes during autonomous.
Reply With Quote
  #15   Spotlight this post!  
Unread 07-31-2018, 12:33 PM
iwilcove's Avatar
iwilcove iwilcove is offline
It's not the code
AKA: Isaac Wilcove
FRC #7308 (DeepVision)
Team Role: Programmer
 
Join Date: Jan 2018
Rookie Year: 2017
Location: California, USA
Posts: 63
iwilcove is on a distinguished road
Re: Help with vision to align with cubes

If you have any interest deep learning, check this out:

https://github.com/frc7308/powercube-detection

The PowerCubeNet.caffemodel file is the important bit; If you do some reading here you should be able to inference from it in no time. It will give you coordinates of where it sees power cubes in an image, so if you take pictures from a camera and inference using those, you can use one of the X values from the returned coordinates to get the angle of a power cube relative to your robot, and turn to face it.

If you want to train your own model, I recommend you use TensorFlow instead, which has a lot more support and is easier to learn and use. You can learn about TensorFlow here.

You can also DM me if you want 1000+ labeled images of power cubes or have any questions about deep learning in general
__________________
DV8
Reply With Quote
Reply


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 03:58 PM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi