Using Vision to confirm Shots in Autonomous

Technical Programming
, , , ,
#1

This build season while programming our 5 ball, we found that confirming balls were shot so the robot could transition to the next step in our autonomous sequence wasn’t particularly reliable. We attempted reliable detection using motor rpm, but found that it was difficult to tune due to inconsistencies in our shooter design. Now while doing this, we found that using external sensors (colour sensors, switches etc) were reliable, but added weight and also would detect the ball before it being shot, so a ball getting stuck was an concern.

We found an efficient solution in the use of the limelight. Our limelight sat directly behind the shooter, so the arc of the ball would cover the target from site for roughly 60ms. Using the same logic as a switch, we allowed it to detect when balls were shot and out of the shooter on their way to the hub. With a properly configured limelight this had almost a 100% success rate.

From here it’s a ramble so just ignore this if you’re not interested

While this was only used by us this season in auto (it was implemented late season), this could have many interesting uses. We used it to detect if we SHOT balls, but it could be used to detect scored balls (by running a check on tx and maybe the associated shooter speed for the distance?) There are quite a few interesting things that could be done with this and I’m interested to see how it could be used

Yeah this may not be relevant to many teams but I thought we could share it anyway in case anyone found it useful/interesting.

8 Likes
#2

Would you be willing to share some shooter code…

I like this concept here as we just used some timeouts. Did you also include some timeouts incase you failed to load a ball and only had 1 ball to shoot?

3 Likes
#3

Yeah sure, I’ll send it through some point today :slight_smile:

#4

We had a lot of success detecting the drops in the shooter’s RPM after we thought we shot a ball to confirm shots. It sounds like you tried that and it didn’t work, though. What exactly was your problem with that? For us it was very consistent even though we have a very inconsistent shooter.

#5

I know I have used current draw as our metric to see if shots were taken.

1 Like
#6

We just had inconsistencies wheel position so the compression on the ball was gradually changing. we could not get it consistently detecting even through all PID and everything. I’d be interested in seeing how you implemented it with full consistency

#7

So this code didn’t have much time in implementation so it lacks ‘cleanliness’ but we are going to be releasing a modulated version of our code at some point but here it is:

public class autoBallDetectionLimelight extends SequenceTransition{
private boolean assumedBallStatus = false;
private double numBalls = 0;
private double maxNumBalls = 0;
private int numLoops = 0;
public void setNumBalls(double aNumBalls) {
maxNumBalls = aNumBalls;
}

@Override
public void transStart() {
    numBalls = 0;
    numLoops = 0;
    assumedBallStatus = false;
    SmartDashboard.putNumber("balls shot", numBalls);
    // TODO Auto-generated method stub
}


@Override
public boolean transUpdate() {
    boolean sensorStatus = Limelight.getInstance().getTargetAcquired();
        if(sensorStatus != true){
                assumedBallStatus = true;
                numLoops++;
        }
        else{
            if(assumedBallStatus == true && numLoops >= 3){
                ballDetected();
                assumedBallStatus = false;
                numLoops = 0;
            }
            else if(assumedBallStatus == true || sensorStatus != true){
                numLoops++;
            }
            else{
                numLoops++;
                System.out.println("Something just went wrong . if you are seeing this just throw your joystick or controller away god has left, science is no more and computers are dead.a");
            }
        }

    return isTransComplete();
}

private void ballDetected()
{
    numBalls = numBalls + 1;
    SmartDashboard.putNumber("balls shot", numBalls);
}



@Override
public boolean isTransComplete() {
    if (numBalls >= maxNumBalls)
    {
        return true;        
    }
    return false;
}

}

and that basically it, it then sits in our sequencer and we define it like this:

   autoBallDetectionLimelight ball = new autoBallDetectionLimelight();
   timedStep T01 = new timedStep();
   T01.setDelay(1.2);

…

   shoot.setNextTrans(ball, T01); 
   shoot.setNextSteps(shoot);
#8

We found methods like that to have an roughly 85-95% accuracy rate, and we just felt like we’d want a better one since with 5 balls that leaves a relatively high chance of only shooting 3-4. we also used a double flywheel which may have affected our reliability.

#9

Well the wheel position and compression of the ball shouldn’t affect this too much. Every time you shoot a ball the RPM should drop at least a little and you should be able to measure that. Do you know how to get the RPM of the flywheel at any given moment? My recommendation is to constantly get the flywheel RPM and then compare it to the flywheel RPM in the last “tick” of some periodic method. Then if you see the difference is over a certain amount you know that something hit the flywheel and slowed it down and if you are expecting a ball to get shot, you know that a ball was shot there and can do whatever you want with that info. If you want more specific code examples I can give you those.

1 Like
#10 
  if (waitCounts > 0)

    { 
        waitCounts --;
    }
    double currRPM = Shooter.getInstance().getShooterRPM();    
    // compare currRPM with pastRPM[pastLength]
    if ( (currRPM / pastRPM[0]) < 0.87)
    {
    //ball detected
        if (waitCounts == 0)
        {
            waitCounts = waitTime;
            ballDetected();
        }
    }
    for ( int ii = 0 ; ii < pastLength - 1 ; ii++)
    {
        pastRPM[ii] = pastRPM[ii+1];
    }
    pastRPM[pastLength-1] = currRPM;
    return isTransComplete();

this was our code we just didnt get good results from any multiplier that didnt either underdetect or overdetect, especially considering the fact that our robot had our shooter always running

#11

(somewhat) Pseudo code:

private int lastRPM = shooter.getRPM();
@Override
public void enabledPeriodic() {
     int currentRPM = shooter.getRPM();
     if (lastRPM - currentRPM >= 70 /* est rpm drop after shot, 70 worked for us */) {
            ballDetected();
            // Add a "debounce" timer like waitCounts
     }
     lastRPM = currentRPM;
}

This worked very well for us, like I said, it didn’t “miss” a ball getting shot out during any of last 3 events (including worlds).

Also for having it overdetect when the shooter’s already running without a ball, only run this “algorithm” when you expect to shoot a ball.

#12

We did this and it turned out slightly less then consistent or less ‘immediate’

:man_shrugging: I don’t know maybe it’s a difference in code structure it just didn’t work for us, this was more accurate

1 Like
#13

Also note that NEO built-in encoders have 112 ms of delay so you won’t even know about and RPM drop until 0.1 s after it happens.

#14

Well we aren’t that worried about 0.1 seconds in our situation, I can see how it may be a problem for others though.

#15

Interesting, this is really cool though, great job on your solution!

#16

How complex was the code within getTargetAcquired()?

Theoretically, looking at vision might be more inherently noisy due to false negatives. Shooter RPM can also be noisy due to measurement noise. This might be noisy comparing this against more reliable solutions such as beam breaks which should give you a much more definitive answer about a ball leaving the robot.

Overall, interesting idea!

I think the applications for detecting if a ball is scored would vary on the game (e.g. driving to a “rebound” point if you know you missed).

#17

It was just a time based get, if it was nissing for 60 ms or longer it assumed a ball was there.