On the quality and complexity of software within FRC

[Andrew Schreiber]

Originally Posted by gblake

Before is slips off of everyone's radar for good, I thought I would give this thread one more poke.

Honestly, games where autonomous has tiered rewards that are actually worth something and that are not penalized for attempting them.

2015 - Pretty terrible, the only task you could accomplish on your own was REALLY hard. The other tasks all required your partners to also do something. (I don't count can burglaring as an auton task)

2014 - Almost good, the penalty for attempting to score a ball was pretty harsh though.

2013 - Great. 0 penalty for attempting to score in any of the goals. Even drive forward and dump 2 in the low goal was viable and provided a reasonable reward. And the reward -> difficulty scaled appropriately to even the upper tier.

2012 - Scoring was MUCH harder than 2013 so meh.

2011 - Most teams struggled to score, let alone scoring uber tubes autonomously.

2010 - Literally 0 point.

2009 - There was a game?

2008 - Great. Even just driving forward was worth points, bonus points if you could turn at the end of it.

2007 - See 2011 only strike the word uber

2006 - See 2013

2005 - meh, not a whole lot of teams attempted it. Vision was REALLY hard.

2004 - Very few teams attempted to knock off the balls. But a lot of folks prepped for teleop, kinda decent but not really.

2003 - Robot Demolition Derby isn't really a good auton, sorry.


If teams have a reason to write good code they probably will write some. But if they are penalized for attempting auton teams will just pass because the risk is not worth the reward.

[Kevin Leonard]

Originally Posted by Andrew Schreiber

Honestly, games where autonomous has tiered rewards that are actually worth something and that are not penalized for attempting them.

2015 - Pretty terrible, the only task you could accomplish on your own was REALLY hard. The other tasks all required your partners to also do something. (I don't count can burglaring as an auton task)

2014 - Almost good, the penalty for attempting to score a ball was pretty harsh though.

2013 - Great. 0 penalty for attempting to score in any of the goals. Even drive forward and dump 2 in the low goal was viable and provided a reasonable reward. And the reward -> difficulty scaled appropriately to even the upper tier.

2012 - Scoring was MUCH harder than 2013 so meh.

2011 - Most teams struggled to score, let alone scoring uber tubes autonomously.

2010 - Literally 0 point.

2009 - There was a game?

2008 - Great. Even just driving forward was worth points, bonus points if you could turn at the end of it.

2007 - See 2011 only strike the word uber

2006 - See 2013

2005 - meh, not a whole lot of teams attempted it. Vision was REALLY hard.

2004 - Very few teams attempted to knock off the balls. But a lot of folks prepped for teleop, kinda decent but not really.

2003 - Robot Demolition Derby isn't really a good auton, sorry.


If teams have a reason to write good code they probably will write some. But if they are penalized for attempting auton teams will just pass because the risk is not worth the reward.

I consider canburgling an auto task- THAT WAS STILL REALLY HARD, especially if you wanted to do it at more than a regional level.

2012 autonomous was just as good as 2013 IMO, because scoring low baskets was easy, and worth 4pts/score (vs 2013's 2 pts/score), and feeding balls into a partner was another great autonomous task that was easy.

2014 would have been perfect as well, were it not so punishing to miss autonomous.

Really the GDC has gotten autonomous right 3 times. 2008, 2012, and 2013.

I think 2012 was the best year for programmers. Improved controls turned into improved results for most teams. Improved autonomous was valuable, and there were effective tasks to do for teams at every level, programming-wise.

[plnyyanks]

Originally Posted by Kevin Leonard

I think 2012 was the best year for programmers. Improved controls turned into improved results for most teams. Improved autonomous was valuable, and there were effective tasks to do for teams at every level, programming-wise.

Definitely agree. That year was the holy grail of good game design, a nice correlation between automation difficulty and point return, and the tools to make it happen. The robot code I wrote that year is one of the few things from that long ago I'm still proud of

[Monochron]

I'll throw out my 2 cents for what I think is the main thing that holds back the evolution of programming on a team:

Getting a mechanical system to the base state of "it works"(regardless of how well) takes a lot more effort and time then programming does. By that I mean that code changes can be done quickly and efficiently with minimal peoples' effort and mechanical changes often involve a team of people machining, bolting, cutting, lifting, etc. This may sound like programming could evolve quickly but what usually happens is that mechanical issues take precedence in the design process. When engineers are making a big modification to a mechanical part they often like to keep all other variables static. Which means programming changes don't go through if the mechanism still needs to be tested out / modified.

Once the code "works" it can be hard to justify changing it when you know that you are already sinking time into changing mechanical or electrical systems.


A good way to avoid these situations are to ensure that your programming team has an adequate testing environment so that code can evolve in isolation from ever changing mechanical parts. Set up a branching model so that you can give the mechanical folks a working build and then continue to develop in parallel. This is one of the things we strove for this past year and it, I think, made a big difference in the quality of our code.

[MrRoboSteve]

What effects would this change to the rules have on software quality?

R15 Teams must stay “hands-off” their bagged ROBOT elements during the following time periods:

A. between Stop Build Day and their first event,

B. during the period(s) between their events, and

C. outside of Pit hours while attending events.

Additional time is allowed as follows:

D. After Kickoff, there are no restrictions on when software may be developed.

E. On days a team is not attending an event, they may continue development of any items permitted per R17, including items
listed as exempt from R17, but must do so without interfacing with the ROBOT.

F. Teams attending 2-day events may access their ROBOTS per the rules defined in the 2015 Administrative Manual Section 5.6: ROBOT Access Period - for Teams Attending District Events.

G. ROBOTS may be exhibited per 2015 Administrative Manual Section 5.5.3: Robot Displays.

H. Teams may operate their ROBOT for the purpose of testing software updates.

I purposely left out fixing the robot in the new language.

[GeeTwo]

Originally Posted by MrRoboSteve

What effects would this change to the rules have on software quality?

H. Teams may operate their ROBOT for the purpose of testing software updates.

I purposely left out fixing the robot in the new language.

If teams can't fix the robot (or even effectively troubleshoot wiring, which usually amounts to the same thing), they won't be able to operate it to test software once something breaks. If the rule is actually followed, it would have about the same average effect as moving bag and tag back to 12:15 am on Wednesday.

[EricH]

Originally Posted by MrRoboSteve

What effects would this change to the rules have on software quality?

H. Teams may operate their ROBOT for the purpose of testing software updates.

I purposely left out fixing the robot in the new language.

Minimal, at best, and it might actually make it worse.

As pointed out, no fixing (or other electro-mechanical work, presumably) would be allowed, thus as soon as something broke, no further testing of software could be done.

But... most upgrades of software tend to work with (and follow after) upgrades in hardware. No upgrades in hardware mean software doesn't need upgrading.

And there is one other item that I can see happening. This is why I think it could become WORSE code, not better.

--A team could, theoretically, upload a base code right before the bag that has driving disabled, make one upgrade (enabling the drive code), and spend the rest of the allowed time "testing the upgrade"--which to everybody else is "driver practice". As a side note, a good, practiced driver can often do at least as well with lousy code as with good code--just a touch of compensating needed maybe.

[GeeTwo]

I'm only going back to 2012, as befits my team's experience:

Originally Posted by Andrew Schreiber

2015 - Pretty terrible, the only task you could accomplish on your own was REALLY hard. The other tasks all required your partners to also do something. (I don't count can burglaring as an auton task)

2014 - Almost good, the penalty for attempting to score a ball was pretty harsh though.

2013 - Great. 0 penalty for attempting to score in any of the goals. Even drive forward and dump 2 in the low goal was viable and provided a reasonable reward. And the reward -> difficulty scaled appropriately to even the upper tier.

2012 - Scoring was MUCH harder than 2013 so meh.

...

If teams have a reason to write good code they probably will write some. But if they are penalized for attempting auton teams will just pass because the risk is not worth the reward.

Apart from the mobility bonus "gimme" in 2014 (really? drive across a line?) I thought that 2012 and 2014 were nearly identical. You got bonus points for scoring in hybrid/auto. If you miss, you have to pick up the ball again to score it for no bonus. And while the two years of experience certainly helped, scoring unopposed in AA seemed a lot easier than in RR, both high and low.

2015 was the only one that failed to reward incrementally, and the number of things that could go wrong caused a number of teams (including mine) to decide that none of our routines was worth the risk. I am surprised at how many teams did NOT have a "drive into the auto zone" auto. Granted, it was only three points, but it was essentially the same as the mobility bonus in 2014, and it seemed like the great majority of teams did it.

Originally Posted by Kevin Leonard

I consider canburgling an auto task- THAT WAS STILL REALLY HARD, especially if you wanted to do it at more than a regional level.

I do not consider canburgling as an auto task, but I can see the point - it was a scarce worm that went to the early bird. The two reasons not were that it was not rewarded directly because it was autonomous, and (more importantly) most of the canburglar programming was a single actuator with no sensor feedback. That is, it was best solved as a mechanical problem, not an automation problem.

Originally Posted by Kevin Leonard

I think 2012 was the best year for programmers. Improved controls turned into improved results for most teams. Improved autonomous was valuable, and there were effective tasks to do for teams at every level, programming-wise.

I don't recall Rebound Rumble this way at all, but I wasn't mentoring yet. As I recall, if you didn't do the kinect (and I saw few teams that did), you had either very easy (score preloaded balls; tip one bridge) or rather hard tasks (both; tip multiple bridges; pick up balls and score them) in auto/hybrid. Please expand on this.

[Monochron]

Originally Posted by GeeTwo

Apart from the mobility bonus "gimme" in 2014 (really? drive across a line?) I thought that 2012 and 2014 were nearly identical. You got bonus points for scoring in hybrid/auto. If you miss, you have to pick up the ball again to score it for no bonus.

The big difference cor 2014 was that unscored auto balls could NOT have ASSIST points applied to them which was the primary means of scoring that year. So if you missed an auto shot (or 3) you then needed to clear those balls before you could start scoring points in the range that your opponents could. You lost valuable cycle time to playing cleanup for a relatively meager amount of points.

[Kevin Leonard]

Originally Posted by GeeTwo

I'm only going back to 2012, as befits my team's experience:



Apart from the mobility bonus "gimme" in 2014 (really? drive across a line?) I thought that 2012 and 2014 were nearly identical. You got bonus points for scoring in hybrid/auto. If you miss, you have to pick up the ball again to score it for no bonus. And while the two years of experience certainly helped, scoring unopposed in AA seemed a lot easier than in RR, both high and low.

2015 was the only one that failed to reward incrementally, and the number of things that could go wrong caused a number of teams (including mine) to decide that none of our routines was worth the risk. I am surprised at how many teams did NOT have a "drive into the auto zone" auto. Granted, it was only three points, but it was essentially the same as the mobility bonus in 2014, and it seemed like the great majority of teams did it.



I do not consider canburgling as an auto task, but I can see the point - it was a scarce worm that went to the early bird. The two reasons not were that it was not rewarded directly because it was autonomous, and (more importantly) most of the canburglar programming was a single actuator with no sensor feedback. That is, it was best solved as a mechanical problem, not an automation problem.


I don't recall Rebound Rumble this way at all, but I wasn't mentoring yet. As I recall, if you didn't do the kinect (and I saw few teams that did), you had either very easy (score preloaded balls; tip one bridge) or rather hard tasks (both; tip multiple bridges; pick up balls and score them) in auto/hybrid. Please expand on this.

In Rebound Rumble during auto, you could:
Feed a partner balls
Score low goals
Score middle goals
Score high goals
Grab the two balls off the side bridge and shoot those as well
Grab the two balls off the middle bride and shoot those as well

In Aerial Assist, if you missed balls, you couldn't score during teleop until those balls were scored. Also, in 2012 if you missed, you could score those balls from the key during teleop, where opponents couldn't defend you, where in 2014 you got rammed if you tried to pick up your misses.

[GeeTwo]

Originally Posted by Monochron

The big difference cor 2014 was that unscored auto balls could NOT have ASSIST points applied to them which was the primary means of scoring that year. So if you missed an auto shot (or 3) you then needed to clear those balls before you could start scoring points in the range that your opponents could. You lost valuable cycle time to playing cleanup for a relatively meager amount of points.

Before the season, that's what I expected. The actuality was that most of our matches seemed to devolve into one robot on offense and two on defense on both sides; no assist points, and few truss shots. It was really disappointing seeing how poor many teams' ball pickups were.

I only recall one time where we had to chase a wayward auto ball for more than a few seconds. Our pickup roller was somehow still in the track of our launcher; it started upright to be within the frame perimeter. The ball went backwards (not quite a reverse truss shot). If it had happened in teleop, the effect would have been the same.

I guess if you had a low percentage auto, it was not worth loading them at all in 2014, or stuffing them in the low goal. If you were much over 60%, the risk was more than justified. And even a box on wheels should be able to a score a low goal auto pretty consistently.

Originally Posted by Kevin Leonard

In Rebound Rumble during auto, you could: ..
Feed a partner balls

OK, I missed that. I don't remember passing balls between robots being illegal, but I don't recall it ever happening, either. Or is this something else?

Originally Posted by Kevin Leonard

Also, in 2012 if you missed, you could score those balls from the key during teleop, where opponents couldn't defend you, where in 2014 you got rammed if you tried to pick up your misses.

This was a difference in teleop, not auto. The key was protected whether the balls you were carrying had been preloaded in the robot (or if you didn't have any balls). In both games a loose ball was a loose ball, whether it was initially loaded in the robot; in RR it was available but not a liability, and in AA you couldn't get another ball until you scored it. The "opportunity cost" of missing a shot was usually the same whether the shot was taken in auto or teleop in both games.

[Kevin Leonard]

Originally Posted by GeeTwo

Before the season, that's what I expected. The actuality was that most of our matches seemed to devolve into one robot on offense and two on defense on both sides; no assist points, and few truss shots. It was really disappointing seeing how poor many teams' ball pickups were.

I only recall one time where we had to chase a wayward auto ball for more than a few seconds. Our pickup roller was somehow still in the track of our launcher; it started upright to be within the frame perimeter. The ball went backwards (not quite a reverse truss shot). If it had happened in teleop, the effect would have been the same.

I guess if you had a low percentage auto, it was not worth loading them at all in 2014, or stuffing them in the low goal. If you were much over 60%, the risk was more than justified. And even a box on wheels should be able to a score a low goal auto pretty consistently.


OK, I missed that. I don't remember passing balls between robots being illegal, but I don't recall it ever happening, either. Or is this something else?

This was a difference in teleop, not auto. The key was protected whether the balls you were carrying had been preloaded in the robot (or if you didn't have any balls). In both games a loose ball was a loose ball, whether it was initially loaded in the robot; in RR it was available but not a liability, and in AA you couldn't get another ball until you scored it. The "opportunity cost" of missing a shot was usually the same whether the shot was taken in auto or teleop in both games.

Did you watch 2014 Einstein Finals-2?
Missed autonomous shots caused the 254-469-2848-74 alliance to lose a match because of how long it took them to re-score those missed balls. Whereas missed autonomous balls on Einstein in 2012 didn't mean you automatically lost the match.
What this meant was that if you messed up auto at most events during eliminations, you lost the match.

As for feeding balls to partners in 2012, 4334 did it throughout Archimedes Eliminations, as well as on Einstein. 20 did it as the third robot of the Connecticut Regional Championship alliance.

Basically the problem with 2014 autonomous was that letting your partner run their autonomous routine was a major liability if they failed, whereas in 2012 and 2013, missing auto shots just lost you that autonomous score.

[GeeTwo]

Originally Posted by Kevin Leonard

Did you watch 2014 Einstein Finals-2?

no, I admit that after Bayou I lost interest.

Originally Posted by Kevin Leonard

As for feeding balls to partners in 2012, 4334 did it throughout Archimedes Eliminations, as well as on Einstein. 20 did it as the third robot of the Connecticut Regional Championship alliance.

Was this only in eliminations? Was the ball passed on the floor, or was there some sort of hand shake? Honestly, I was surprised more teams couldn't to a bumper-to-bumper or short pass ball transfer in Aerial Assist where you could effectively score 10 or 20 points each time you did it; it's nice to know that some teams did this in a game that didn't directly call for it.

[Knufire]

The two ways I saw ball passing occur in 2012 was either on the floor (through the shooting robots intake) or a light toss into the shooting robots' hopper if they had one. I know 3322 did it very early in the season.

[evanperryg]

Originally Posted by GeeTwo

The actuality was that most of our matches seemed to devolve into one robot on offense and two on defense on both sides; no assist points, and few truss shots.
...
I only recall one time where we had to chase a wayward auto ball for more than a few seconds.
...
And even a box on wheels should be able to a score a low goal auto pretty consistently.

Interesting that your experience with AA was entirely the opposite of mine. What I saw was teams all contributing in some way offensively, then contributing in some way defensively when they completed their offensive job for that cycle. What I also saw was teams doing their best at every level of play to keep opposing alliance members away from their auto balls. (see: Crossroads, Einstein finals, IRI) I also saw lots of boxes getting caught on walls, or just not moving at all in auto.

Originally Posted by GeeTwo

I do not consider canburgling as an auto task, but I can see the point - it was a scarce worm that went to the early bird. The two reasons not were that it was not rewarded directly because it was autonomous, and (more importantly) most of the canburglar programming was a single actuator with no sensor feedback. That is, it was best solved as a mechanical problem, not an automation problem.

Having worked with 548's brutal can pullers, I can attest to the fact that the fastest (or at least some of the fastest) did use sensors. A potentiometer was used do detect when the pullers were all the way down, so they would move forward as fast as possible.

As for the quality and complexity of FRC code, I think we can all agree it'd be great to teach great programming. It'd also be great if more teams actually did great programming. Yet, ultimately, we have the kind of hardware that allows us to be sloppy, libraries that are poorly made themselves, and generally diminishing returns for making code more efficient. Great code can be made in FRC and it can have good returns, but unless you have everything else about your robot down to laser-precise perfection, there's probably something else that could be made better more easily, whose returns would scale better with the effort you put into it. For the sake of inspiration, have at it because programming is vital to our future generation of engineers. However, for the sake of making a consistently successful team, there is often something that will give greater returns for less effort.