we were sick and tired of compromises: "speed or torque", "traction or maneuverability", etc.; bump that, we want it all. all we had to compromise was simplicity. - RogerR [more]
 Chief Delphi What does "strength of schedule" mean?
 CD-Media CD-Spy
 portal register members calendar search Today's Posts Mark Forums Read FAQ rules

#1
07-15-2018, 04:43 PM
 Caleb Sykes Knock-off Dr. Strange AKA: inkling16 no team Join Date: Feb 2011 Rookie Year: 2009 Location: Minneapolis, Minnesota Posts: 1,674
What does "strength of schedule" mean?

I hear the term "strength of schedule" (SoS) thrown around sometimes and I'm not quite sure I understand what this means in an FRC context. For reference, here and here the most recent thread discussing this topic. I'd like to start a discussion specifically on what (if any) metric makes the most sense to represent an individual team's SoS, and then ideally to expand on this definition to come up with a metric that represents how "fair" or "balanced" an overall match schedule is.

I've made a program that I can use to calculate metrics like this for all of last season's events, the first set of results from that program can be found in my miscellaneous statistics projects thread. I have created a metric which I think has potential as a SoS metric, and I'll copy the details of that metric below. As the thread progresses, I'm happy to revisit that program and calculate other metrics if people want.

Here's my first pass at a SoS metric. I calculate it by finding the probability that a given team will seed better with the actual schedule than they would have with a random schedule. So a "schedule strength" of 0% means that you will never seed higher with the existing schedule than you would have with a random schedule, and a "schedule strength" of 100% means that you are guaranteed to seed higher with the actual schedule than you would have with a random schedule.

• It compares the given schedule against other hypothetical schedules
• It is customized for each team, that is, it compares your hypothetical results with a random schedule with your hypothetical results with the given schedule. I'm not the biggest fan of team-independent metrics since, for example, a schedule full of buddy climb capable partners is amazing for a team without a buddy climber, but just alright for a team that has a good buddy climber, and team-independent metrics would have to give the schedule a single score for both of these teams.
• It's on an interpretable scale (0% to 100%) and has meaningful significance
• It's able to be calculated before the event occurs (I don't like metrics that require hindsight unless maybe we want to use SoS as a tiebreaker or something)
• Incorporates RPs

• Requires a full event simulator to calculate
• Teams that are basically guaranteed to seed first (like 1678 at their later regionals) will inevitably be shown to have bad schedules, since there is no schedule that would give them much of a better chance of seeding higher than their expectation of almost certainly first. Switching to greater than or equal ranks just flips the problem to high scores instead of low scores for these scenarios
• Average value is 48.1% instead of 50%
#2
07-15-2018, 05:00 PM
 Tom Ore Registered User FRC #0525 (Swartdogs) Team Role: Mentor Join Date: Mar 2010 Rookie Year: 2009 Location: Cedar Falls, Iowa Posts: 500
Re: What does "strength of schedule" mean?

I've done a simple strength of schedule calculation of few times. I take some metric, like OPR or ranking points, add up a given team's 2 partner's scores and subtract the 3 opponent's scores for all matches. The given team's own score is not included. I do this for all teams and then sort that total score. Most or even all teams will have a negative score - the least negative has the easiest schedule. Then I compare the actual rank to the SOS rank and look for significant differences. For example, teams that had an easy schedule but finished much lower and teams that had a difficult schedule and finished much higher. I'm not sure how much it really means, other than to maybe look closer at these teams if they were or were not on your pick list.
#3
07-16-2018, 07:45 AM
 efoote868 foote stepped in AKA: E. Foote FRC #0868 Team Role: Mentor Join Date: Mar 2006 Rookie Year: 2005 Location: Noblesville, IN Posts: 1,800
Re: What does "strength of schedule" mean?

Quote:
 Originally Posted by Tom Ore I've done a simple strength of schedule calculation of few times. I take some metric, like OPR or ranking points, add up a given team's 2 partner's scores and subtract the 3 opponent's scores for all matches. The given team's own score is not included. I do this for all teams and then sort that total score. Most or even all teams will have a negative score - the least negative has the easiest schedule. Then I compare the actual rank to the SOS rank and look for significant differences. For example, teams that had an easy schedule but finished much lower and teams that had a difficult schedule and finished much higher. I'm not sure how much it really means, other than to maybe look closer at these teams if they were or were not on your pick list.
I would consider averaging the given metric of the partners versus the average of the opponents. Or maybe substitute a "middle" score instead of the team's own score.

When the college sports pundits talk about "strength of schedule", especially for football teams in the top 25, the team being discussed really is a non-factor. The pundits don't discount Alabama or Ohio State's win against the #24 ranked team simply because they're ranked #1 or #2.

For FRC, strength of schedule helps answer the question, why is team X ranked where they're at? Did they get a favorable schedule or did they claw their way to their ranking despite their schedule?

It also can help detract or bolster the case for team Y as GOAT...
#4
07-16-2018, 09:19 AM
 SenorZ Physics Teacher AKA: Tom Zook FRC #4276 (Surf City Vikings) Team Role: Teacher Join Date: Jan 2011 Rookie Year: 2011 Location: Huntington Beach, California Posts: 1,359
Re: What does "strength of schedule" mean?

I'm on a computer without a spreadsheet program right now, so forgive my ignorance.

How does this SoS account for week 1 events? Especially those with several rookie or year-two teams? As we all know, some rookie teams hit the ground running and do amazing. And some year-two teams perform a much different level than they did rookie year.
__________________
2013-present: FRC Team 4276, Surf City Vikings
2011-2012: FRC Team 3677, The Don Bots
#5
07-16-2018, 09:54 AM
 Caleb Sykes Knock-off Dr. Strange AKA: inkling16 no team Join Date: Feb 2011 Rookie Year: 2009 Location: Minneapolis, Minnesota Posts: 1,674
Re: What does "strength of schedule" mean?

Quote:
 Originally Posted by SenorZ I'm on a computer without a spreadsheet program right now, so forgive my ignorance.
Here's a google sheets version if that is easier.

Quote:
 How does this SoS account for week 1 events? Especially those with several rookie or year-two teams? As we all know, some rookie teams hit the ground running and do amazing. And some year-two teams perform a much different level than they did rookie year.
All rookie teams are started out with an Elo rating of 1450 (roughly a 30th percentile team) and calculated contributions equal to the average calculated contributions of all week 1 rookies.
#6
07-16-2018, 03:49 PM
 Whatever Registered User FRC #2502 Join Date: Apr 2016 Location: MN Posts: 271
Re: What does "strength of schedule" mean?

If you were to treat same placement results as half a point in the better and half in the worse, would that eliminate the dominate team issue?

There is something weird with the Southern Cross Regional. You have the regional listed with 30 teams and only 30 teams listed when there really was 40. Not sure if other events have that same issue.
#7
07-16-2018, 03:52 PM
 Whatever Registered User FRC #2502 Join Date: Apr 2016 Location: MN Posts: 271
Re: What does "strength of schedule" mean?

Quote:
 Originally Posted by Whatever There is something weird with the Southern Cross Regional. You have the regional listed with 30 teams and only 30 teams listed when there really was 40. Not sure if other events have that same issue.
I looked closer and I think I see what is going on. It looks like 40 teams signed up but only 30 teams showed up. So the random simulation has 40 teams in it while the real schedule simulations have 30 teams in it.
#8
07-21-2018, 02:36 PM
 lcraig910 average analysis and average memes AKA: Kevin Swift's Biggest Fan FRC #0910 (The Foley Freeze) Team Role: Student Join Date: Apr 2017 Rookie Year: 2016 Location: Madison Heights Posts: 83
Re: What does "strength of schedule" mean?

When I brought up strength of schedule in the "Power Up good at Rankings" discussion, I never really considered it as match making decider.

Quote:
 Originally Posted by lcraig910 Here is another proposed solution: eliminate extra RP, and then just use the same "strength of schedule" equation to determine the number of points a team really deserves.
However, it could also work as a method of match making. One thing I like about it is this:

Quote:
 Originally Posted by Caleb Sykes What I like about this metric: It compares the given schedule against other hypothetical schedules
I like this because of the similarity it bears to the algorithm that is used to develop District Championship divisions, as described by Jim Zondag and Dan Kimura, where the divisions are compared to other hypothetical divisions until an even 4 (or 2 in Ontario) divisions are created. You look at the 2018 MSC, and the divisions very evenly matched. Every playoff round had a rubber match on FIM-stein, and besides the Dow division (which was dominated by the eventual captain of the Einstein winning alliance, who then picked the eventual captain of the Einstein finalist alliance), every division had their finals matches taken to 3. This shows how equal all of the divisions were, IMO, and why the "strength of schedule" comparing a schedule to other hypothetical schedules to create the best possible one would be a great way to go.

 Thread Tools Display Modes Rate This Thread Linear Mode Rate This Thread: 5 : Excellent 4 : Good 3 : Average 2 : Bad 1 : Terrible

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts vB code is On Smilies are On [IMG] code is On HTML code is Off
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home Announcements     User Announcements FIRST     General Forum         FIRST E-Mail Blast Archive     Rumor Mill     Career     Robot Showcase Technical     Technical Discussion     Robotics Education and Curriculum     Motors     Electrical         CAN     Programming         NI LabVIEW         C/C++         Java         Python     Control System         FRC Control System         Sensors     Pneumatics     Kit & Additional Hardware     CAD         Inventor         SolidWorks         Creo     IT / Communications         3D Animation and Competition         Website Design/Showcase         Videography and Photography         Computer Graphics     National Instruments LabVIEW and Data Acquisition         LabView and Data Acquisition Competition     Unsung FIRST Heroes     Awards         Chairman's Award     Rules/Strategy         Scouting         You Make The Call     Team Organization         Fundraising         Starting New Teams         Finding A Team         College Teams     Championship Event     Regional Competitions     District Events     Off-Season Events     Thanks and/or Congrats     FRC Game Design     OCCRA         OCCRA Q&A         OCCRA Programming Other     Chit-Chat         Games/Trivia             Fantasy FIRST     Car Nack's Corner     College & University Education     Dean Kamen's Inventions     FIRST-related Organizations         Western Region Robotics Forum         Southern California Regional Robotics Forum         The Blue Alliance             Video Archives     FIRST In the News...     FIRST Lego League         Lego Mindstorm Discussion     FIRST Tech Challenge     VEX         VEX Robotics Competition         VEX IQ     Televised Robotics     Math and Science         NASA Discussion ChiefDelphi.com Website     CD Forum Support     Extra Discussion

All times are GMT -5. The time now is 09:57 PM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.

 -- English (12 hour) -- English (24 hour) Contact Us - Chief Delphi - Rules - Archive - Top