View Single Post
  #14   Spotlight this post!  
Unread 04-19-2016, 06:18 PM
Citrus Dad's Avatar
Citrus Dad Citrus Dad is offline
Business and Scouting Mentor
AKA: Richard McCann
FRC #1678 (Citrus Circuits)
Team Role: Mentor
 
Join Date: May 2012
Rookie Year: 2012
Location: Davis
Posts: 983
Citrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond repute
Re: What would you do to improve the FIRST experience?

Quote:
Originally Posted by Lil' Lavery View Post
I've participated in quite a few different sized tournaments, and ran the gamut on types of schedules encountered.

2007 was a disaster. The basis of the scheduling algorithm was that it divided teams into three "bins" based on team number (lowest third, middle third, and upper third). Each alliance contained one member from each bin. As far as I recall, this was done by the algorithm maker without the solicitation of FIRST HQ. There were improvements made to the system as the season progressed, but the concept was fundamentally flawed.

In 2007 week one, the system was impossibly bad. If you study team 116's schedule from VCU you'll begin to see why. Because VCU had 66 teams, a number divisible by 6 (the quantity of teams in each match), VCU ended up with a very rigid schedule. The teams in lowest age bracket ended up facing one another every single match. In the case of 116, that meant all 8 qualification matches were against 122. 116's middle bracket opponents in one match would be their middle bracket partners in their next match (in graduating numerical order). This is obviously unacceptable.

While FIRST and the algorithm designers rectified the most egregious of those errors as the season progressed, both the fundamental concept aned execution were still flawed. While the practice match schedules from Championship 2007 don't exist explicitly, you can determine them by looking at the first few matches on any given teams' qualification schedule (they were identical, albeit with filler line teams as necessary). That's both an execution and a conceptual issue. While the best of the best teams could often overcome the biased schedules, the rankings were rather skewed that season. Younger teams capable of executing the game had a distinct advantage. As a result, you saw a number of "weaker" alliance captains. 1712 was not a top 8 robot on Galileo that season, but being a sophomore team capable of scoring consistently gave 1712 a very favorable schedule when matched against predominantly other second and third year teams. This is a conceptual flaw.

Regardless of how you determine team skill, whether it be age, OPR, district points, or some other metric, attempting to create a biased schedule creates inequality. When you create a metric-based strength of schedule constraint on the scheduling algorithm, it ends up creating additional reward for teams who outperform their previous metric (and implicitly punishing teams unfortunate enough to draw them). The same applies in reverse to teams that underperform their metrics. Think of how a rookie star like 5985 or 5817 would fit into such a system, and the impact they would have on scheduling. Think of how a powerhouse team that lost key mentors would impact the system.

An example of this is looking at the OPR for 2007 Galileo. Every team in the top 15 was a member of the highest numbered (youngest) bin. While the exponential scoring on 2007 makes OPR essentially useless for that game, this demonstrates the scheduling bias in play (and also demonstrates how introducing a strength of schedule constraint ends up invalidating the metrics you're using to create the strength of schedule).

More importantly, once you start adding additional constraints to the schedule, you have to be more flexible on the existing constraints. That was one of the huge issues with the 2007 algorithm, and is a fundamental problem with any attempt at adding a strength of schedule constraint. When you factor in strength of schedule, suddenly you have to be more willing to flex on one or more of the other constraints (minimum time between matches, round parity, minimum schedule repeats, etc). While the execution of the 2007 scheduling algorithm was tremendously poor in this respect (namely in terms of minimizing repeats), it's not purely an execution issue.
Clearly if two teams played each other 8 times then something was very wrong with the scheduling algorithm. I will note that it is very common now to see 2 teams play with each other in one match and then against each other the next so that issue still exists and clearly is acceptable.

I think it's very interesting this year how younger teams are ranking much higher than previously. Look at how many events had rookies as the first qualifier. At SVR this year, 3 alliance captains were rookies or 2nd year teams. This is a function of RPs often not being a function of W-L records. I suspect that the 2007 results may have been just as much about the scoring system as the match scheduling.

(BTW, something is wrong int he OPR calculations for Galielo in 2007--they imply that the OPRs for the other teams are strongly negative. Archimedes and Newton have the same problem. Curie might be correct. I suspect the problem is in the bin-method of scheduling took away a key element of solving the matrix problem. So it's not the scoring method that messed with the OPRs; it's the way that teams were matched up. So the bottom line is that the OPRs are worthless for comparison in 2007.)

Don't confuse random and lucky with fair. I'm not sure how being lucky creates equality of scheduling. And as someone pointed out earlier the schedule isn't truly random--it's already constrained, AND it's subject to the judgement of an official that it looks "sufficiently" balanced. Why not make the balancing method transparent rather than heaping arbitrary on top of arbitrary.

I have an idea of how to structure the schedule in a very simple way that solves the constraint problems and can be executed very quickly. But I don't have time to put a demo together until after Champs (I have other scouting duties to attend to first.) I'll have something in May to show.
__________________
Reply With Quote