What would you do to improve the FIRST experience?

[ATannahill]

Originally Posted by Citrus Dad

I've lost the earlier thread (I think started by Boltman) about match scheduling at Central Valley Regional. I suggested that there is a better way to schedule those matches so as that we don't have oddities where the best teams (yes, there is sorting of abilities in FRC) don't play each other or end up in alliances together during qualifications. We saw the same thing happen at Silicon Valley this year. 1678, 254, 971 and 368 never played each other and 1678 was allied with all 3 others. 3 teams were unbeaten and 368 could have been unbeaten as well.

It turns out that the NFL goes through substantial schedule permutations to arrive at equitable constrained schedules for the season. (Baseball, basketball and hockey don't have this problem because each team plays the others multiple times.)

Let's not confuse "arbitrary" with "random." Random doesn't guarantee fair, especially if we don't have multiple draws. Playing 1-3 events each season doesn't guarantee that everyone will have an equitable chance. (And BTW, the current method is not truly "random" because it requires a seed value to get it started. This has long been an issue in statistical analysis.) The situation requires intervention to arrive at a fair schedule. No scheduling method will be perfect, but most efforts will be better than the arbitrary method used now.

For anyone wondering, you can find out about the current scheduling algorithm here. You can also download it and try to create schedules for yourself.

[Zagar]

I have to agree with the ability to have more teams work together in the build season.

This was my first season with FRC, and Metal Muscle 1506 is incredibly fortunate to have the FIRST Center at Kettering which we now share with 7 other teams. All of the teams located in the FIRST Center learned from each other, worked together and were able to lend a helping hand whenever needed.

I know building such a facility is no small endeavor, but it would be incredible if FIRST were able to have more of these facilities around the country and the world. I know they are building another one soon in Grand Rapids, but wouldn't it be great to have more of them scattered around? Even teams that are not part of the Center could come in and practice on the field, talk about the robots, awards, designs and collaborate on everything FIRST.

[Richard Wallace]

Originally Posted by Zagar

I have to agree with the ability to have more teams work together in the build season.

This was my first season with FRC, and Metal Muscle 1506 is incredibly fortunate to have the FIRST Center at Kettering which we now share with 7 other teams. All of the teams located in the FIRST Center learned from each other, worked together and were able to lend a helping hand whenever needed.

I know building such a facility is no small endeavor, but it would be incredible if FIRST were able to have more of these facilities around the country and the world. I know they are building another one soon in Grand Rapids, but wouldn't it be great to have more of them scattered around? Even teams that are not part of the Center could come in and practice on the field, talk about the robots, awards, designs and collaborate on everything FIRST.

+1^

This idea is a much better application of major sponsor funding, contrasted with another round of unsustainable rookie grants.* Kettering's program is an example that other tech universities and corporate consortia should emulate.
-----------

*Which go straight to HQ, funding neither team development nor local event improvement.

[JamesBrown]

Originally Posted by rtfgnow

For anyone wondering, you can find out about the current scheduling algorithm here. You can also download it and try to create schedules for yourself.

Interesting, I never realized this information was available.

For the Citrus Dad, if you notice the Algorithm rtfdnow linked to is from 2008. 2007 attempted to sort teams so the best teams would play against each other, rather than only with each other. They essentially proke the teams into 3 tiers by team age (team number) and each alliance was made up of one team per tier. Since Average team performance tends to be higher in older teams, this lead to high performing, young teams seeding well ahead of older teams with similar performing robots. This was widely seen as a disaster by just about everyone.

If memory serves FIRST actually solicited algorithm suggestions during/after the 2007 season.

If you have a good solution to the issue then write it up, and submit it to first. I am sure they would be willing to listen.

I don't think anyone thinks the algorithm is perfect, but the luck of match schedules is part of the game.

[Citrus Dad]

Originally Posted by JamesBrown

Interesting, I never realized this information was available.

For the Citrus Dad, if you notice the Algorithm rtfdnow linked to is from 2008. 2007 attempted to sort teams so the best teams would play against each other, rather than only with each other. They essentially proke the teams into 3 tiers by team age (team number) and each alliance was made up of one team per tier. Since Average team performance tends to be higher in older teams, this lead to high performing, young teams seeding well ahead of older teams with similar performing robots. This was widely seen as a disaster by just about everyone.

If memory serves FIRST actually solicited algorithm suggestions during/after the 2007 season.

If you have a good solution to the issue then write it up, and submit it to first. I am sure they would be willing to listen.

I don't think anyone thinks the algorithm is perfect, but the luck of match schedules is part of the game.

The ranking should be done by an actual performance measure such as district points equivalent or OPR. (We'd probably use the previous year as a reasonable proxy.) Team number isn't a particularly good metric these days.

I have an idea of how to structure this arrangement but will wait until after Champs to write it up. The problem with "luck of match schedules" is that it is arbitrary, and teams are not viewing it as unfair. I'm not sure why it should be considered "part of the game"--it's not an obvious consideration. And as I pointed out "random" doesn't equal "fair" over such a small number of opportunities. The CVR and SVR situations are not uncommon in the large regionals. I suspect that the 2007 schedule was implemented incorrectly.

[Andrew Schreiber]

Originally Posted by Citrus Dad

The ranking should be done by an actual performance measure such as district points equivalent or OPR. (We'd probably use the previous year as a reasonable proxy.) Team number isn't a particularly good metric these days.

I have an idea of how to structure this arrangement but will wait until after Champs to write it up. The problem with "luck of match schedules" is that it is arbitrary, and teams are not viewing it as unfair. I'm not sure why it should be considered "part of the game"--it's not an obvious consideration. And as I pointed out "random" doesn't equal "fair" over such a small number of opportunities. The CVR and SVR situations are not uncommon in the large regionals. I suspect that the 2007 schedule was implemented incorrectly.

No, it was just BAD.

And let's stop calling the current schedule random. It is not. And one of the metrics it looks at is "number of times played with or against".

[plnyyanks]

Originally Posted by Andrew Schreiber

And let's stop calling the current schedule random. It is not. And one of the metrics it looks at is "number of times played with or against".

Additionally, one of the things the FTA/Scorekeeper looks at after generating a candidate schedule is the number of unique partners and opponents for each team. If one team has a wildly different number for one of those metrics, that is grounds to run the MatchMaker algorithm a second time.

[Lil' Lavery]

I've participated in quite a few different sized tournaments, and ran the gamut on types of schedules encountered.

2007 was a disaster. The basis of the scheduling algorithm was that it divided teams into three "bins" based on team number (lowest third, middle third, and upper third). Each alliance contained one member from each bin. As far as I recall, this was done by the algorithm maker without the solicitation of FIRST HQ. There were improvements made to the system as the season progressed, but the concept was fundamentally flawed.

In 2007 week one, the system was impossibly bad. If you study team 116's schedule from VCU you'll begin to see why. Because VCU had 66 teams, a number divisible by 6 (the quantity of teams in each match), VCU ended up with a very rigid schedule. The teams in lowest age bracket ended up facing one another every single match. In the case of 116, that meant all 8 qualification matches were against 122. 116's middle bracket opponents in one match would be their middle bracket partners in their next match (in graduating numerical order). This is obviously unacceptable.

While FIRST and the algorithm designers rectified the most egregious of those errors as the season progressed, both the fundamental concept aned execution were still flawed. While the practice match schedules from Championship 2007 don't exist explicitly, you can determine them by looking at the first few matches on any given teams' qualification schedule (they were identical, albeit with filler line teams as necessary). That's both an execution and a conceptual issue. While the best of the best teams could often overcome the biased schedules, the rankings were rather skewed that season. Younger teams capable of executing the game had a distinct advantage. As a result, you saw a number of "weaker" alliance captains. 1712 was not a top 8 robot on Galileo that season, but being a sophomore team capable of scoring consistently gave 1712 a very favorable schedule when matched against predominantly other second and third year teams. This is a conceptual flaw.

Regardless of how you determine team skill, whether it be age, OPR, district points, or some other metric, attempting to create a biased schedule creates inequality. When you create a metric-based strength of schedule constraint on the scheduling algorithm, it ends up creating additional reward for teams who outperform their previous metric (and implicitly punishing teams unfortunate enough to draw them). The same applies in reverse to teams that underperform their metrics. Think of how a rookie star like 5985 or 5817 would fit into such a system, and the impact they would have on scheduling. Think of how a powerhouse team that lost key mentors would impact the system.

An example of this is looking at the OPR for 2007 Galileo. Every team in the top 15 was a member of the highest numbered (youngest) bin. While the exponential scoring on 2007 makes OPR essentially useless for that game, this demonstrates the scheduling bias in play (and also demonstrates how introducing a strength of schedule constraint ends up invalidating the metrics you're using to create the strength of schedule).

More importantly, once you start adding additional constraints to the schedule, you have to be more flexible on the existing constraints. That was one of the huge issues with the 2007 algorithm, and is a fundamental problem with any attempt at adding a strength of schedule constraint. When you factor in strength of schedule, suddenly you have to be more willing to flex on one or more of the other constraints (minimum time between matches, round parity, minimum schedule repeats, etc). While the execution of the 2007 scheduling algorithm was tremendously poor in this respect (namely in terms of minimizing repeats), it's not purely an execution issue.

[plnyyanks]

Originally Posted by Citrus Dad

I have an idea of how to structure this arrangement but will wait until after Champs to write it up.

Originally Posted by Andrew Schreiber

And one of the metrics it looks at is "number of times played with or against".

Originally Posted by plnyyanks

Additionally, one of the things the FTA/Scorekeeper looks at after generating a candidate schedule is the number of unique partners and opponents for each team. If one team has a wildly different number for one of those metrics, that is grounds to run the MatchMaker algorithm a second time.

Richard, et all,

For your reference while working on an improved algorithm, I wrote up a script that calculated the {max, min, median, mean} for the number of unique partners each team played with and against for each event this year. These are the numbers that the FTA/Scorekeeper look at after they generate a schedule. Since it's a large amount of data, I won't post it here, but you can access the files on GitHub.

Hope that information is helpful!

[Citrus Dad]

Originally Posted by plnyyanks

Richard, et all,

For your reference while working on an improved algorithm, I wrote up a script that calculated the {max, min, median, mean} for the number of unique partners each team played with and against for each event this year. These are the numbers that the FTA/Scorekeeper look at after they generate a schedule. Since it's a large amount of data, I won't post it here, but you can access the files on GitHub.

Hope that information is helpful!

Thanks. I'll take a look. I also agree with Blake that there is an easy way to preset the schedules and then just randomize the team numbers to be assigned to slots.

[Lil' Lavery]

Originally Posted by Citrus Dad

Clearly if two teams played each other 8 times then something was very wrong with the scheduling algorithm. I will note that it is very common now to see 2 teams play with each other in one match and then against each other the next so that issue still exists and clearly is acceptable.

I'm not contesting that the execution of that algorithm was very poor. However, that poor execution highlighted some of the fundamental issues with strength of schedule as an algorithm parameter.

In terms of current partners becoming opponents, it's not a direct issue per se. However, the consistency of it in the week one schedules helps allow for reverse engineering of how the algorithm functioned, and demonstrates the rigidity of an over constrained algorithm.

Originally Posted by Citrus Dad

(BTW, something is wrong int he OPR calculations for Galielo in 2007--they imply that the OPRs for the other teams are strongly negative. Archimedes and Newton have the same problem. Curie might be correct. I suspect the problem is in the bin-method of scheduling took away a key element of solving the matrix problem. So it's not the scoring method that messed with the OPRs; it's the way that teams were matched up. So the bottom line is that the OPRs are worthless for comparison in 2007.)

That was my point. When you create imbalanced schedules, it invalidates the metrics you used to establish those schedules. In this extreme case, it quite literally almost broke the matrix. In a less extreme case, using a district point based system would create schedules that would lower the district points of the high end and raise the district points of the low end (harming the metric in future iterations of the schedule).

With regards to OPR in 2007, I still stand by it being a pretty poor metric. The end game was cooperative and the primary scoring method was both exponential and cooperative (multiple teams building a row together). Both of those things play very poorly with OPR. Further still, facing off against tougher competition actually hurt your scores, since smart placement of their tubes denied longer rows. I suspect this played a significant role in why low numbered teams were implied such large negative contributions.

[techhelpbb]

Originally Posted by ConnorTheCoch

While FIRST is professional, many a team overemphasize stoicism over a light, joking environment. Obviously there needs to be a balance, but there is a difference between professionalism and being a micromanaging, overregulating, no fun zone dictatorship. Let's lighten the mood to make the environment that much more encouraging.

Silence! The beatings will continue until morale improves!
I mean you're from North Korea I bet the commute to the South for competition days is interesting.

[Gefowl]

Originally Posted by gblake

I have never encountered that mindset. I'm not saying it doesn't exist, but I do think (in my limited experience) it is pretty rare.

Have you encountered it often?

Blake

Re-reading it I think I worded what I was trying to say wrong.

I've haven't ran into much difficulty getting students into FIRST, but getting them to commit time to FIRST can be difficult. The difficulty of having them commit time raises the older students get because they have likely already commit time to other extracurricular and part-time work.

Essentially I think middle school age and older elementary school students will be more likely to commit their time to FIRST if they know there is an FRC team waiting for them in high school. But if an FRC team doesn't exist in a high school they'd be more eager to dedicate time to other extracurricular activities over FIRST.

[Citrus Dad]

Originally Posted by Lil' Lavery

I've participated in quite a few different sized tournaments, and ran the gamut on types of schedules encountered.

2007 was a disaster. The basis of the scheduling algorithm was that it divided teams into three "bins" based on team number (lowest third, middle third, and upper third). Each alliance contained one member from each bin. As far as I recall, this was done by the algorithm maker without the solicitation of FIRST HQ. There were improvements made to the system as the season progressed, but the concept was fundamentally flawed.

In 2007 week one, the system was impossibly bad. If you study team 116's schedule from VCU you'll begin to see why. Because VCU had 66 teams, a number divisible by 6 (the quantity of teams in each match), VCU ended up with a very rigid schedule. The teams in lowest age bracket ended up facing one another every single match. In the case of 116, that meant all 8 qualification matches were against 122. 116's middle bracket opponents in one match would be their middle bracket partners in their next match (in graduating numerical order). This is obviously unacceptable.

While FIRST and the algorithm designers rectified the most egregious of those errors as the season progressed, both the fundamental concept aned execution were still flawed. While the practice match schedules from Championship 2007 don't exist explicitly, you can determine them by looking at the first few matches on any given teams' qualification schedule (they were identical, albeit with filler line teams as necessary). That's both an execution and a conceptual issue. While the best of the best teams could often overcome the biased schedules, the rankings were rather skewed that season. Younger teams capable of executing the game had a distinct advantage. As a result, you saw a number of "weaker" alliance captains. 1712 was not a top 8 robot on Galileo that season, but being a sophomore team capable of scoring consistently gave 1712 a very favorable schedule when matched against predominantly other second and third year teams. This is a conceptual flaw.

Regardless of how you determine team skill, whether it be age, OPR, district points, or some other metric, attempting to create a biased schedule creates inequality. When you create a metric-based strength of schedule constraint on the scheduling algorithm, it ends up creating additional reward for teams who outperform their previous metric (and implicitly punishing teams unfortunate enough to draw them). The same applies in reverse to teams that underperform their metrics. Think of how a rookie star like 5985 or 5817 would fit into such a system, and the impact they would have on scheduling. Think of how a powerhouse team that lost key mentors would impact the system.

An example of this is looking at the OPR for 2007 Galileo. Every team in the top 15 was a member of the highest numbered (youngest) bin. While the exponential scoring on 2007 makes OPR essentially useless for that game, this demonstrates the scheduling bias in play (and also demonstrates how introducing a strength of schedule constraint ends up invalidating the metrics you're using to create the strength of schedule).

More importantly, once you start adding additional constraints to the schedule, you have to be more flexible on the existing constraints. That was one of the huge issues with the 2007 algorithm, and is a fundamental problem with any attempt at adding a strength of schedule constraint. When you factor in strength of schedule, suddenly you have to be more willing to flex on one or more of the other constraints (minimum time between matches, round parity, minimum schedule repeats, etc). While the execution of the 2007 scheduling algorithm was tremendously poor in this respect (namely in terms of minimizing repeats), it's not purely an execution issue.

Clearly if two teams played each other 8 times then something was very wrong with the scheduling algorithm. I will note that it is very common now to see 2 teams play with each other in one match and then against each other the next so that issue still exists and clearly is acceptable.

I think it's very interesting this year how younger teams are ranking much higher than previously. Look at how many events had rookies as the first qualifier. At SVR this year, 3 alliance captains were rookies or 2nd year teams. This is a function of RPs often not being a function of W-L records. I suspect that the 2007 results may have been just as much about the scoring system as the match scheduling.

(BTW, something is wrong int he OPR calculations for Galielo in 2007--they imply that the OPRs for the other teams are strongly negative. Archimedes and Newton have the same problem. Curie might be correct. I suspect the problem is in the bin-method of scheduling took away a key element of solving the matrix problem. So it's not the scoring method that messed with the OPRs; it's the way that teams were matched up. So the bottom line is that the OPRs are worthless for comparison in 2007.)

Don't confuse random and lucky with fair. I'm not sure how being lucky creates equality of scheduling. And as someone pointed out earlier the schedule isn't truly random--it's already constrained, AND it's subject to the judgement of an official that it looks "sufficiently" balanced. Why not make the balancing method transparent rather than heaping arbitrary on top of arbitrary.

I have an idea of how to structure the schedule in a very simple way that solves the constraint problems and can be executed very quickly. But I don't have time to put a demo together until after Champs (I have other scouting duties to attend to first.) I'll have something in May to show.

[Peyton Yeung]

Originally Posted by Citrus Dad

Clearly if two teams played each other 8 times then something was very wrong with the scheduling algorithm. I will note that it is very common now to see 2 teams play with each other in one match and then against each other the next so that issue still exists and clearly is acceptable.

This year at Indiana State Champs we played
45: 2 with 2 against
135: 1 with 2 against
829: 1 with 2 against
1018: 1 with 2 against
1024: 1 with 4 against
1501: 0 with 3 against
1529: 0 with 2 against
1720: 1 with 2 against
1741: 1 with 2 against
3180: 1 with 2 against
3936: 0 with 2 against

The rest of our match ups were fairly low occurrence ( 1 with/against or 0 with/against). I even buy the 1 to 2 relationships since it's a small event. The thing I thought was weird was the times we had including 1024 and 1501. That's a lot of times we are both on the field.