What would you do to improve the FIRST experience?

[Monochron]

Originally Posted by Road Rash

So in a nutshell, are you're basically saying that the safety advisors are treating safety as an audit of your systems, and overlooking actual safe practices taking place?
...
I'll also train them to be able to point out our procedures to the safety advisors, just in case they are unable to comfortably answer a question for whatever reason. I just want them to make good, sound decisions while working as relaxed as we can possibly be at the moment.

Yeah, that's a good analogy. And to be fair, there are plenty of instances of them recommending good safety practices and ensuring that intelligent safety does get recognized. It can just be a challenge to deal with the unfortunate examples. When a student saw me get accidentally arm-barred by a safety volunteer a couple of years ago, it was an awkward conversation to have to ensure her that the volunteer was acting with my best interests at heart, he was maybe just a little over aggressive.

Overall, I think our students (and it sounds like yours too) have a healthy respect for their own and others' safety while working. After all, they made "Safety Always" one of team slogans!

[gblake]

Originally Posted by FarmerJohn

FIRST needs to stop being so arrogant/selfish and admit that IFI has better lower and middle level robotics competitions, and an overall better competition structure. FRC tops all, but VRC is better than FTC in every way possible, and VIQ gives teams more out of their season than FLL does. And they both cost a whole lot less. If FIRST really cared about inspiring students they'd either try their best to actually make these lower and middle level competitions better or just tell students to do VIQ and VRC. If you're promoting an educational program whose alternative is better in every way, you're not in it for the students, you're in it for yourself.

I am a huge fan of the V** programs, but FTC does have at least one significant, positive difference from VRC, and there are other more debatable differences.

If you want to design and create custom parts, FTC offers far more opportunities. Designing and creating more custom parts doesn't equal more inspiration (at least not in my book); but it can equal more fun. It can also equal less fun, and/or a barrier to entry. YMMV

My bottom line: Let's not ascribe malice to either program, or attempt to kick either to the curb. Both encourage students to stick their toes into STEM waters at low-ish cost, and with low-ish mentoring requirements. If both tripled in size, they/we still would only be nibbling at satisfying the total North American (not to mention the rest of the world) needs.

Tell students, adults, and sponsors about both and let them pick the one that is right for them. Don't attempt to make the choice for them, graciously and professionally let them choose the one that best suits their circumstances.

Blake

[Road Rash]

Originally Posted by Monochron

Overall, I think our students (and it sounds like yours too) have a healthy respect for their own and others' safety while working. After all, they made "Safety Always" one of team slogans!

I just started a couple of weeks ago, just after the season was effectively over for us. I had heard that we did need to improve on safety after the regional. I haven't had the chance to see what kind of safety program is currently in place yet.

It isn't such a bad thing to be exposed to what audits are like in a corporate structure, though. No matter what system in place is being audited, it can be a painful experience. It can be a powerful lesson in grace under pressure.

[plnyyanks]

Originally Posted by Citrus Dad

I have an idea of how to structure this arrangement but will wait until after Champs to write it up.

Originally Posted by Andrew Schreiber

And one of the metrics it looks at is "number of times played with or against".

Originally Posted by plnyyanks

Additionally, one of the things the FTA/Scorekeeper looks at after generating a candidate schedule is the number of unique partners and opponents for each team. If one team has a wildly different number for one of those metrics, that is grounds to run the MatchMaker algorithm a second time.

Richard, et all,

For your reference while working on an improved algorithm, I wrote up a script that calculated the {max, min, median, mean} for the number of unique partners each team played with and against for each event this year. These are the numbers that the FTA/Scorekeeper look at after they generate a schedule. Since it's a large amount of data, I won't post it here, but you can access the files on GitHub.

Hope that information is helpful!

[gblake]

Originally Posted by Andrew Schreiber

And let's stop calling the current schedule random. It is not. And one of the metrics it looks at is "number of times played with or against".

Originally Posted by plnyyanks

Additionally, one of the things the FTA/Scorekeeper looks at after generating a candidate schedule is the number of unique partners and opponents for each team. If one team has a wildly different number for one of those metrics, that is grounds to run the MatchMaker algorithm a second time.

After dabbling with creating a scheduler a few years ago (I still have the code), I believe that there is a huge unnecessary burden in the methods I *think* are still being used to create schedules for both FIRST and VEX tournaments.

Think about this.

It is possible to create and store, in advance of any tournament, for any/all possible numbers of participants, "canned" match schedules that satisfy any constraints desired, and that include N matches (where N is much larger than the number matches expected in any reasonable tournament).

The number of schedules to be created and the storage space they would consume are both more than zero, but both are trivial in modern computers.

On the day of a tournament:

If T teams are participating, the canned schedule designed for T teams is used.

The actual teams participating can be randomly assigned to the T placeholder teams used to generate that canned schedule (Real team X becomes team 1 in the canned schedule, real team Y, becomes team 2 in the canned schedule, etc.).

The actual match schedule to be used is then produced by picking a random starting point with the N sets of matches in the canned schedule, and simply using the next M sets of matches for the tournament.

Even a slow, modern computer could accomplish (or repeat) the day-of-the-tournament part of the process in a trivial amount of time.

Furthermore, I can't think of any reason not to publish the pre-computed, canned schedules.

If the canned schedules were published in advance, then, on the day of a tournament (or well before?), tournament organizers could publish that event's random real-to-placeholder team assignments, and the event's randomly chosen starting point within the N matches of the canned schedule being used.

Once that info (above) is published - voila! - scouts/anyone could easily produce a perfect copy of the entire event's schedule, without having to retype it, scan it, web scrape it, etc.

I believe that adopting this approach to creating match schedules would improve the FIRST/VEX/whatever experience.

Blake

PS: Also, just think how much fun people could have debating the "fairness" of pre-computed schedules (the entire thing, or selected stretches). If the practice was adopted, someone could start a thread on both evaluating that, and determining how many angels can dance on the head of a pin. Many hours of popcorn-munching entertainment would follow.

[gblake]

Big picture:

• Make it easier for students and adults to participate (lower barriers to entry).
• Focus more on giving all members of teams opportunities to make hands-on contributions to their team's STEM product.
• Make it easier to put on formal tournaments, off-season tournaments, and scrimmage events (I'm thinking about much more than switching to "districts").
• Have a growth path for students who become inspired to try introductory STEM, and then want to see how far they can develop their skills while still in school.

For me this adds up to a near reversal of the current relative emphasis on FRC vs FTC.

In order to reach more students, and especially in areas that might need STEM programs more that others, I would make FTC (or something similar) the primary/flagship high school emphasis of FIRST, and would let FRC become an advanced program, for those students and adults in locations that want to pay (in time and money) for a more complex challenge.

FTC is the far easier way (far less intimidating, faster tangible results, easier what-if experimentation, etc. etc.) to introduce an uninspired novice to STEM robotics. FTC is the far less costly program. FTC is the far less time-consuming program. FTC is the far more hands-on program. FTC events are far easily to produce, etc.

Blake
PS: Remember that simply convincing K-12 students to try a STEM education/career is a success. It isn't necessary to actually train them (deeply) in any of the various STEM disciplines. That happens in vocational schools, colleges and afterward.

[Mr V]

Originally Posted by plnyyanks

Additionally, one of the things the FTA/Scorekeeper looks at after generating a candidate schedule is the number of unique partners and opponents for each team. If one team has a wildly different number for one of those metrics, that is grounds to run the MatchMaker algorithm a second time.

Not for this season, I halfheartedly complained to the FTAs at our first event this season that our schedule had us playing against a known powerhouse team, that did end up ranked #1, 3 times at a 40 team 12 match event. He told me that as of this season they are not allowed to run the match scheduling program a second time without calling HQ first. I know they weren't messing with me because I've known two of them since well before they were FTAs as we were mentors for the same team for many years before I moved to my new team.

[plnyyanks]

Originally Posted by Mr V

Not for this season, I halfheartedly complained to the FTAs at our first event this season that our schedule had us playing against a known powerhouse team, that did end up ranked #1, 3 times at a 40 team 12 match event. He told me that as of this season they are not allowed to run the match scheduling program a second time without calling HQ first.

I'm pretty sure I remember Danny saying this year that one of the few cases where a second run would be acceptable is a drastic mismatch in the generated schedule stats. So if all teams play against ~25 opponents, but a few play against only 15, for example. Of course, you'd still have to check with HQ, but that's one of the situations where they could allow it.

I don't think it would have applied to your particular case though, since the with/against numbers for your team (and all others) could still have been in an acceptable range, even with a couple repeated opponents.

[gblake]

Originally Posted by Mr V

Not for this season, I halfheartedly complained to the FTAs at our first event this season that our schedule had us playing against a known powerhouse team, ...

Originally Posted by plnyyanks

I'm pretty sure I remember Danny saying this year that one of the few cases where a second run would be acceptable is a drastic mismatch in the generated schedule stats. ...

And, just to reiterate an advantage of pre-computed match schedules (described here). Any/all "unfairness" could be scrubbed out of them ahead of time, with the result guaranteed to satisfy FIRST's fairness criteria, whatever they might be.

[techhelpbb]

Originally Posted by gblake

In order to reach more students, and especially in areas that might need STEM programs more that others, I would make FTC (or something similar) the primary/flagship emphasis of FIRST, and would let FRC become the follow-on program, for those students and adults who want to graduate from FTC.

FTC is the far easier way (far less intimidating, faster tangible results, easier what-if experimentation, etc. etc.) to introduce an uninspired novice to STEM robotics. FTC is the far less costly program. FTC is the far less time-consuming program. FTC is the far more hands-on program. FTC events are far easily to produce, etc.

Blake
PS: Remember that simply convincing K-12 students to try a STEM education/career is a success. It isn't necessary to actually train them (deeply) in any of the various STEM disciplines. That happens in vocational schools, colleges and afterward.

I agree with this. It doesn't remove FRC and it doesn't stop FRC from teaching skills to people that are nearly adults. It reaches out to students that are about to have greater freedom to make elections towards their future before they do so. Hopefully that way if they choose to step into FRC they can set their goals realistically.

To put that in perspective: I was a vocational student before I left high school. I left the later part of the day and was taken to the vocational school to spend several hours working on a real world skill. FIRST wasn't available to me back then, my motivator was I already knew what I liked doing because I did it in my family business. I don't know if 2 years at the start of high school would drive most people to commit like that. 2 years in high school and a few years before that would give a lot of people a great idea where they want to be.

[Citrus Dad]

Originally Posted by Lil' Lavery

I've participated in quite a few different sized tournaments, and ran the gamut on types of schedules encountered.

2007 was a disaster. The basis of the scheduling algorithm was that it divided teams into three "bins" based on team number (lowest third, middle third, and upper third). Each alliance contained one member from each bin. As far as I recall, this was done by the algorithm maker without the solicitation of FIRST HQ. There were improvements made to the system as the season progressed, but the concept was fundamentally flawed.

In 2007 week one, the system was impossibly bad. If you study team 116's schedule from VCU you'll begin to see why. Because VCU had 66 teams, a number divisible by 6 (the quantity of teams in each match), VCU ended up with a very rigid schedule. The teams in lowest age bracket ended up facing one another every single match. In the case of 116, that meant all 8 qualification matches were against 122. 116's middle bracket opponents in one match would be their middle bracket partners in their next match (in graduating numerical order). This is obviously unacceptable.

While FIRST and the algorithm designers rectified the most egregious of those errors as the season progressed, both the fundamental concept aned execution were still flawed. While the practice match schedules from Championship 2007 don't exist explicitly, you can determine them by looking at the first few matches on any given teams' qualification schedule (they were identical, albeit with filler line teams as necessary). That's both an execution and a conceptual issue. While the best of the best teams could often overcome the biased schedules, the rankings were rather skewed that season. Younger teams capable of executing the game had a distinct advantage. As a result, you saw a number of "weaker" alliance captains. 1712 was not a top 8 robot on Galileo that season, but being a sophomore team capable of scoring consistently gave 1712 a very favorable schedule when matched against predominantly other second and third year teams. This is a conceptual flaw.

Regardless of how you determine team skill, whether it be age, OPR, district points, or some other metric, attempting to create a biased schedule creates inequality. When you create a metric-based strength of schedule constraint on the scheduling algorithm, it ends up creating additional reward for teams who outperform their previous metric (and implicitly punishing teams unfortunate enough to draw them). The same applies in reverse to teams that underperform their metrics. Think of how a rookie star like 5985 or 5817 would fit into such a system, and the impact they would have on scheduling. Think of how a powerhouse team that lost key mentors would impact the system.

An example of this is looking at the OPR for 2007 Galileo. Every team in the top 15 was a member of the highest numbered (youngest) bin. While the exponential scoring on 2007 makes OPR essentially useless for that game, this demonstrates the scheduling bias in play (and also demonstrates how introducing a strength of schedule constraint ends up invalidating the metrics you're using to create the strength of schedule).

More importantly, once you start adding additional constraints to the schedule, you have to be more flexible on the existing constraints. That was one of the huge issues with the 2007 algorithm, and is a fundamental problem with any attempt at adding a strength of schedule constraint. When you factor in strength of schedule, suddenly you have to be more willing to flex on one or more of the other constraints (minimum time between matches, round parity, minimum schedule repeats, etc). While the execution of the 2007 scheduling algorithm was tremendously poor in this respect (namely in terms of minimizing repeats), it's not purely an execution issue.

Clearly if two teams played each other 8 times then something was very wrong with the scheduling algorithm. I will note that it is very common now to see 2 teams play with each other in one match and then against each other the next so that issue still exists and clearly is acceptable.

I think it's very interesting this year how younger teams are ranking much higher than previously. Look at how many events had rookies as the first qualifier. At SVR this year, 3 alliance captains were rookies or 2nd year teams. This is a function of RPs often not being a function of W-L records. I suspect that the 2007 results may have been just as much about the scoring system as the match scheduling.

(BTW, something is wrong int he OPR calculations for Galielo in 2007--they imply that the OPRs for the other teams are strongly negative. Archimedes and Newton have the same problem. Curie might be correct. I suspect the problem is in the bin-method of scheduling took away a key element of solving the matrix problem. So it's not the scoring method that messed with the OPRs; it's the way that teams were matched up. So the bottom line is that the OPRs are worthless for comparison in 2007.)

Don't confuse random and lucky with fair. I'm not sure how being lucky creates equality of scheduling. And as someone pointed out earlier the schedule isn't truly random--it's already constrained, AND it's subject to the judgement of an official that it looks "sufficiently" balanced. Why not make the balancing method transparent rather than heaping arbitrary on top of arbitrary.

I have an idea of how to structure the schedule in a very simple way that solves the constraint problems and can be executed very quickly. But I don't have time to put a demo together until after Champs (I have other scouting duties to attend to first.) I'll have something in May to show.

[Citrus Dad]

Originally Posted by plnyyanks

Richard, et all,

For your reference while working on an improved algorithm, I wrote up a script that calculated the {max, min, median, mean} for the number of unique partners each team played with and against for each event this year. These are the numbers that the FTA/Scorekeeper look at after they generate a schedule. Since it's a large amount of data, I won't post it here, but you can access the files on GitHub.

Hope that information is helpful!

Thanks. I'll take a look. I also agree with Blake that there is an easy way to preset the schedules and then just randomize the team numbers to be assigned to slots.

[Gefowl]

Originally Posted by Andrew Schreiber

Stop focusing on growth. unpopular statement but - teams that don't move don't inspire. Let's focus less on "a team in every school" and more on building sustainable programs. Would a school start a Division 1 football program without a coach with some basic experience or knowledge? No. Why are we doing FRC teams that way?

Those last two really have to go together - don't start teams with people who have no experience with teams. But how do they get experience? By being at these events.

Cannot agree with you more.

However, One of the odd things with FIRST is that it's difficult to convince middle/elementary aged kids (and in an extent parents) to invest their time in FIRST unless there is an FRC team they can see themselves joining later in high school. If there isn't an FRC team that students know they can join later being on a JFLL-FTC team may not seem to be worthwhile of their time.

To (quickly) make sustainable FIRST programs it almost has to begin with an FRC team. I think FIRST needs to put more emphasis on the importance the lower levels of FIRST are to a successful and sustainable FRC team to rookie teams.

[Lil' Lavery]

Originally Posted by Citrus Dad

Clearly if two teams played each other 8 times then something was very wrong with the scheduling algorithm. I will note that it is very common now to see 2 teams play with each other in one match and then against each other the next so that issue still exists and clearly is acceptable.

I'm not contesting that the execution of that algorithm was very poor. However, that poor execution highlighted some of the fundamental issues with strength of schedule as an algorithm parameter.

In terms of current partners becoming opponents, it's not a direct issue per se. However, the consistency of it in the week one schedules helps allow for reverse engineering of how the algorithm functioned, and demonstrates the rigidity of an over constrained algorithm.

Originally Posted by Citrus Dad

(BTW, something is wrong int he OPR calculations for Galielo in 2007--they imply that the OPRs for the other teams are strongly negative. Archimedes and Newton have the same problem. Curie might be correct. I suspect the problem is in the bin-method of scheduling took away a key element of solving the matrix problem. So it's not the scoring method that messed with the OPRs; it's the way that teams were matched up. So the bottom line is that the OPRs are worthless for comparison in 2007.)

That was my point. When you create imbalanced schedules, it invalidates the metrics you used to establish those schedules. In this extreme case, it quite literally almost broke the matrix. In a less extreme case, using a district point based system would create schedules that would lower the district points of the high end and raise the district points of the low end (harming the metric in future iterations of the schedule).

With regards to OPR in 2007, I still stand by it being a pretty poor metric. The end game was cooperative and the primary scoring method was both exponential and cooperative (multiple teams building a row together). Both of those things play very poorly with OPR. Further still, facing off against tougher competition actually hurt your scores, since smart placement of their tubes denied longer rows. I suspect this played a significant role in why low numbered teams were implied such large negative contributions.

[qscgy]

Reform safety awards, and pit safety in general. The teams that go around giving out mostly useless safety giveaways (Easter eggs with 2 Band-Aids and a hair tie, for example) are not actually trying to improve other teams' safety. Every team I've seen has those supplies already. They just want an award. In my experience, the best way to be safe is to make sure everyone in the pit knows how to safely use their tools and has some common sense. I don't know how you give an award for that , though.

OK, that rant's over. On to the next thing, which is doing something about teams where the mentors build the robot. I know that this is often thrown around unfairly, but at least where I am (CHS), at almost every event I have been to, I've seen at least one team where the mentors are fixing the robot with no students around. This indicates that the mentors understand the robot better than the students, which means that they likely designed it. I understand that the mentors take an active role on many teams, and some teams do need a lot of help, but when the mentors are working without any students around to help or at least observe, the students don't learn anything and in many cases, get to play with one of the best robots at the event. This is not only unfair to the students on the team (always veteran teams, I might add), since they get little out of FRC, but to the other teams where students were heavily involved in the build process, only to get beaten by professional engineers. It has gotten to the point where, on my team, any winning robot is dismissed as "mentor-built". This is often untrue, and is especially unfair to the teams who win without needing a mentor-built robot. My point is, FIRST should be about learning, not just about getting a winning robot.