On the quality and complexity of software within FRC

[gblake]

Before is slips off of everyone's radar for good, I thought I would give this thread one more poke.

Originally Posted by gblake

OP wanted to raise the bar for the FRC software quality.

What are some ways to do that? ...

[connor.worley]

After thinking about this more, I'm not sure if it's even a problem. I've never expected FIRST to take in a bunch of kids and spit out seasoned engineers. The main goal is just to get them to check a STEM box when they're choosing a major for college. So let the enthusiasts develop as much as they please, but I think the average experience is already pretty good as far as accomplishing FIRST's goal goes. Just getting a joystick to control a motor is pretty exciting for most people.

[marshall]

Originally Posted by connor.worley

After thinking about this more, I'm not sure if it's even a problem. I've never expected FIRST to take in a bunch of kids and spit out seasoned engineers. The main goal is just to get them to check a STEM box when they're choosing a major for college. So let the enthusiasts develop as much as they please, but I think the average experience is already pretty good as far as accomplishing FIRST's goal goes.

Agreed.

Originally Posted by connor.worley

Just getting a joystick to control a motor is pretty exciting for most people.

Hello worl...AHHHH!!! It's out of control! Jane, stop this crazy thing!

[Kevin Leonard]

I think its great if my students develop advanced programming and controls. Its a great thing to learn and can be incredibly inspiring to see the robot perform amazing functions on the field that would be impossible or very difficult otherwise.
However, if I have a few students who go from no programming experience to some programming experience, and this makes them want to pursue it further, thats just as good to me, if not more in the lines of FIRST's goals.

I do, however, wish I knew how to keep a large programming team engaged (and perhaps thats the topic for another thread), as its difficult to let every programming student work on robot code when you have a large team.

[GeeTwo]

Originally Posted by Kevin Leonard

However, if I have a few students who go from no programming experience to some programming experience, and this makes them want to pursue it further, thats just as good to me, if not more in the lines of FIRST's goals.

I think that falls under inspiration. Last time I checked, it was one of FIRST's goals.

Originally Posted by Kevin Leonard

I do, however, wish I knew how to keep a large programming team engaged (and perhaps thats the topic for another thread), as its difficult to let every programming student work on robot code when you have a large team.

Perhaps this recent thread?

[Andrew Schreiber]

Originally Posted by gblake

Before is slips off of everyone's radar for good, I thought I would give this thread one more poke.

Honestly, games where autonomous has tiered rewards that are actually worth something and that are not penalized for attempting them.

2015 - Pretty terrible, the only task you could accomplish on your own was REALLY hard. The other tasks all required your partners to also do something. (I don't count can burglaring as an auton task)

2014 - Almost good, the penalty for attempting to score a ball was pretty harsh though.

2013 - Great. 0 penalty for attempting to score in any of the goals. Even drive forward and dump 2 in the low goal was viable and provided a reasonable reward. And the reward -> difficulty scaled appropriately to even the upper tier.

2012 - Scoring was MUCH harder than 2013 so meh.

2011 - Most teams struggled to score, let alone scoring uber tubes autonomously.

2010 - Literally 0 point.

2009 - There was a game?

2008 - Great. Even just driving forward was worth points, bonus points if you could turn at the end of it.

2007 - See 2011 only strike the word uber

2006 - See 2013

2005 - meh, not a whole lot of teams attempted it. Vision was REALLY hard.

2004 - Very few teams attempted to knock off the balls. But a lot of folks prepped for teleop, kinda decent but not really.

2003 - Robot Demolition Derby isn't really a good auton, sorry.


If teams have a reason to write good code they probably will write some. But if they are penalized for attempting auton teams will just pass because the risk is not worth the reward.

[Kevin Leonard]

Originally Posted by Andrew Schreiber

Honestly, games where autonomous has tiered rewards that are actually worth something and that are not penalized for attempting them.

2015 - Pretty terrible, the only task you could accomplish on your own was REALLY hard. The other tasks all required your partners to also do something. (I don't count can burglaring as an auton task)

2014 - Almost good, the penalty for attempting to score a ball was pretty harsh though.

2013 - Great. 0 penalty for attempting to score in any of the goals. Even drive forward and dump 2 in the low goal was viable and provided a reasonable reward. And the reward -> difficulty scaled appropriately to even the upper tier.

2012 - Scoring was MUCH harder than 2013 so meh.

2011 - Most teams struggled to score, let alone scoring uber tubes autonomously.

2010 - Literally 0 point.

2009 - There was a game?

2008 - Great. Even just driving forward was worth points, bonus points if you could turn at the end of it.

2007 - See 2011 only strike the word uber

2006 - See 2013

2005 - meh, not a whole lot of teams attempted it. Vision was REALLY hard.

2004 - Very few teams attempted to knock off the balls. But a lot of folks prepped for teleop, kinda decent but not really.

2003 - Robot Demolition Derby isn't really a good auton, sorry.


If teams have a reason to write good code they probably will write some. But if they are penalized for attempting auton teams will just pass because the risk is not worth the reward.

I consider canburgling an auto task- THAT WAS STILL REALLY HARD, especially if you wanted to do it at more than a regional level.

2012 autonomous was just as good as 2013 IMO, because scoring low baskets was easy, and worth 4pts/score (vs 2013's 2 pts/score), and feeding balls into a partner was another great autonomous task that was easy.

2014 would have been perfect as well, were it not so punishing to miss autonomous.

Really the GDC has gotten autonomous right 3 times. 2008, 2012, and 2013.

I think 2012 was the best year for programmers. Improved controls turned into improved results for most teams. Improved autonomous was valuable, and there were effective tasks to do for teams at every level, programming-wise.

[plnyyanks]

Originally Posted by Kevin Leonard

I think 2012 was the best year for programmers. Improved controls turned into improved results for most teams. Improved autonomous was valuable, and there were effective tasks to do for teams at every level, programming-wise.

Definitely agree. That year was the holy grail of good game design, a nice correlation between automation difficulty and point return, and the tools to make it happen. The robot code I wrote that year is one of the few things from that long ago I'm still proud of

[Monochron]

I'll throw out my 2 cents for what I think is the main thing that holds back the evolution of programming on a team:

Getting a mechanical system to the base state of "it works"(regardless of how well) takes a lot more effort and time then programming does. By that I mean that code changes can be done quickly and efficiently with minimal peoples' effort and mechanical changes often involve a team of people machining, bolting, cutting, lifting, etc. This may sound like programming could evolve quickly but what usually happens is that mechanical issues take precedence in the design process. When engineers are making a big modification to a mechanical part they often like to keep all other variables static. Which means programming changes don't go through if the mechanism still needs to be tested out / modified.

Once the code "works" it can be hard to justify changing it when you know that you are already sinking time into changing mechanical or electrical systems.


A good way to avoid these situations are to ensure that your programming team has an adequate testing environment so that code can evolve in isolation from ever changing mechanical parts. Set up a branching model so that you can give the mechanical folks a working build and then continue to develop in parallel. This is one of the things we strove for this past year and it, I think, made a big difference in the quality of our code.

[MrRoboSteve]

What effects would this change to the rules have on software quality?

R15 Teams must stay “hands-off” their bagged ROBOT elements during the following time periods:

A. between Stop Build Day and their first event,

B. during the period(s) between their events, and

C. outside of Pit hours while attending events.

Additional time is allowed as follows:

D. After Kickoff, there are no restrictions on when software may be developed.

E. On days a team is not attending an event, they may continue development of any items permitted per R17, including items
listed as exempt from R17, but must do so without interfacing with the ROBOT.

F. Teams attending 2-day events may access their ROBOTS per the rules defined in the 2015 Administrative Manual Section 5.6: ROBOT Access Period - for Teams Attending District Events.

G. ROBOTS may be exhibited per 2015 Administrative Manual Section 5.5.3: Robot Displays.

H. Teams may operate their ROBOT for the purpose of testing software updates.

I purposely left out fixing the robot in the new language.

[GeeTwo]

Originally Posted by MrRoboSteve

What effects would this change to the rules have on software quality?

H. Teams may operate their ROBOT for the purpose of testing software updates.

I purposely left out fixing the robot in the new language.

If teams can't fix the robot (or even effectively troubleshoot wiring, which usually amounts to the same thing), they won't be able to operate it to test software once something breaks. If the rule is actually followed, it would have about the same average effect as moving bag and tag back to 12:15 am on Wednesday.

[EricH]

Originally Posted by MrRoboSteve

What effects would this change to the rules have on software quality?

H. Teams may operate their ROBOT for the purpose of testing software updates.

I purposely left out fixing the robot in the new language.

Minimal, at best, and it might actually make it worse.

As pointed out, no fixing (or other electro-mechanical work, presumably) would be allowed, thus as soon as something broke, no further testing of software could be done.

But... most upgrades of software tend to work with (and follow after) upgrades in hardware. No upgrades in hardware mean software doesn't need upgrading.

And there is one other item that I can see happening. This is why I think it could become WORSE code, not better.

--A team could, theoretically, upload a base code right before the bag that has driving disabled, make one upgrade (enabling the drive code), and spend the rest of the allowed time "testing the upgrade"--which to everybody else is "driver practice". As a side note, a good, practiced driver can often do at least as well with lousy code as with good code--just a touch of compensating needed maybe.

[GeeTwo]

I'm only going back to 2012, as befits my team's experience:

Originally Posted by Andrew Schreiber

2015 - Pretty terrible, the only task you could accomplish on your own was REALLY hard. The other tasks all required your partners to also do something. (I don't count can burglaring as an auton task)

2014 - Almost good, the penalty for attempting to score a ball was pretty harsh though.

2013 - Great. 0 penalty for attempting to score in any of the goals. Even drive forward and dump 2 in the low goal was viable and provided a reasonable reward. And the reward -> difficulty scaled appropriately to even the upper tier.

2012 - Scoring was MUCH harder than 2013 so meh.

...

If teams have a reason to write good code they probably will write some. But if they are penalized for attempting auton teams will just pass because the risk is not worth the reward.

Apart from the mobility bonus "gimme" in 2014 (really? drive across a line?) I thought that 2012 and 2014 were nearly identical. You got bonus points for scoring in hybrid/auto. If you miss, you have to pick up the ball again to score it for no bonus. And while the two years of experience certainly helped, scoring unopposed in AA seemed a lot easier than in RR, both high and low.

2015 was the only one that failed to reward incrementally, and the number of things that could go wrong caused a number of teams (including mine) to decide that none of our routines was worth the risk. I am surprised at how many teams did NOT have a "drive into the auto zone" auto. Granted, it was only three points, but it was essentially the same as the mobility bonus in 2014, and it seemed like the great majority of teams did it.

Originally Posted by Kevin Leonard

I consider canburgling an auto task- THAT WAS STILL REALLY HARD, especially if you wanted to do it at more than a regional level.

I do not consider canburgling as an auto task, but I can see the point - it was a scarce worm that went to the early bird. The two reasons not were that it was not rewarded directly because it was autonomous, and (more importantly) most of the canburglar programming was a single actuator with no sensor feedback. That is, it was best solved as a mechanical problem, not an automation problem.

Originally Posted by Kevin Leonard

I think 2012 was the best year for programmers. Improved controls turned into improved results for most teams. Improved autonomous was valuable, and there were effective tasks to do for teams at every level, programming-wise.

I don't recall Rebound Rumble this way at all, but I wasn't mentoring yet. As I recall, if you didn't do the kinect (and I saw few teams that did), you had either very easy (score preloaded balls; tip one bridge) or rather hard tasks (both; tip multiple bridges; pick up balls and score them) in auto/hybrid. Please expand on this.

[Monochron]

Originally Posted by GeeTwo

Apart from the mobility bonus "gimme" in 2014 (really? drive across a line?) I thought that 2012 and 2014 were nearly identical. You got bonus points for scoring in hybrid/auto. If you miss, you have to pick up the ball again to score it for no bonus.

The big difference cor 2014 was that unscored auto balls could NOT have ASSIST points applied to them which was the primary means of scoring that year. So if you missed an auto shot (or 3) you then needed to clear those balls before you could start scoring points in the range that your opponents could. You lost valuable cycle time to playing cleanup for a relatively meager amount of points.

[Kevin Leonard]

Originally Posted by GeeTwo

I'm only going back to 2012, as befits my team's experience:



Apart from the mobility bonus "gimme" in 2014 (really? drive across a line?) I thought that 2012 and 2014 were nearly identical. You got bonus points for scoring in hybrid/auto. If you miss, you have to pick up the ball again to score it for no bonus. And while the two years of experience certainly helped, scoring unopposed in AA seemed a lot easier than in RR, both high and low.

2015 was the only one that failed to reward incrementally, and the number of things that could go wrong caused a number of teams (including mine) to decide that none of our routines was worth the risk. I am surprised at how many teams did NOT have a "drive into the auto zone" auto. Granted, it was only three points, but it was essentially the same as the mobility bonus in 2014, and it seemed like the great majority of teams did it.



I do not consider canburgling as an auto task, but I can see the point - it was a scarce worm that went to the early bird. The two reasons not were that it was not rewarded directly because it was autonomous, and (more importantly) most of the canburglar programming was a single actuator with no sensor feedback. That is, it was best solved as a mechanical problem, not an automation problem.


I don't recall Rebound Rumble this way at all, but I wasn't mentoring yet. As I recall, if you didn't do the kinect (and I saw few teams that did), you had either very easy (score preloaded balls; tip one bridge) or rather hard tasks (both; tip multiple bridges; pick up balls and score them) in auto/hybrid. Please expand on this.

In Rebound Rumble during auto, you could:
Feed a partner balls
Score low goals
Score middle goals
Score high goals
Grab the two balls off the side bridge and shoot those as well
Grab the two balls off the middle bride and shoot those as well

In Aerial Assist, if you missed balls, you couldn't score during teleop until those balls were scored. Also, in 2012 if you missed, you could score those balls from the key during teleop, where opponents couldn't defend you, where in 2014 you got rammed if you tried to pick up your misses.