Ah AndyMark and your \$2 a piece pool noodles - mlantry [more]
 Chief Delphi How does the FiveThirtyEight Elo formula work?
 portal register members calendar search Today's Posts Mark Forums Read FAQ rules

#1
08-07-2018, 09:46 AM
 Caleb Sykes Knock-off Dr. Strange AKA: inkling16 no team (The Piztons) Join Date: Feb 2011 Rookie Year: 2009 Location: Minneapolis, Minnesota Posts: 1,771
How does the FiveThirtyEight Elo formula work?

Can someone help me to understand the Elo update formula used by FiveThirtyEight? Here is the best article I'm aware of describing how it works. Specifically, I don't understand how they derived their "margin of victory multiplier"

Margin of Victory Multiplier = LN(ABS(PD)+1) * (2.2/((ELOW-ELOL)*.001+2.2))

I don't know where any of the constants in this formula come from, the use of a base e logarithm, the 2.2, and the 0.001. Are these values empirically determined? Or are they related intrinsically to the rest of the model? It also seems like a good tuning parameter that they don't include would be to divide the point differential by some constant A (like maybe the standard deviation of NFL PDs), since there's nothing fundamental about NFL point differentials. Unless something like this is hidden in the other parameters.

I've tried to use this formula before for FRC without success, and I want to take another crack at it, but I think I need a better understanding of how it works first.

Last edited by Caleb Sykes : 08-07-2018 at 09:53 AM.
#2
08-07-2018, 10:21 AM
 JohnSchneider Registered User FRC #3310 (Black Hawk Robotics) Team Role: Mentor Join Date: Mar 2010 Rookie Year: 2010 Location: Dallas Posts: 902
Re: How does the FiveThirtyEight Elo formula work?

This guy explains it pretty well

#3
08-07-2018, 11:26 AM
 Karthik Registered User no team Team Role: Mentor Join Date: Apr 2002 Rookie Year: 1998 Location: Toronto, Ontario, Canada Posts: 2,643
Re: How does the FiveThirtyEight Elo formula work?

Quote:
 Originally Posted by Caleb Sykes I don't know where any of the constants in this formula come from, the use of a base e logarithm, the 2.2, and the 0.001. Are these values empirically determined? Or are they related intrinsically to the rest of the model?

Based on my understanding they were empirically determined by looking at historical distributions of point differentials. The logarithm was chosen to ensure diminishing returns on large point differentials. (As to why base e was specifically chosen, again it comes to down to historical point distributions) The link that John shared explains this better than I have.
__________________
:: Karthik Kanagasabapathy ::
"Enthusiasm is one of the most powerful engines of success. When you do a thing, do it with all your might. Put your whole soul into it. Stamp it with your own personality. Be active, be energetic, be enthusiastic and faithful and you will accomplish your object. Nothing great was ever achieved without enthusiasm" -- R.W. Emerson
My TEDx Talk - The Subtle Secrets of Success
#4
08-07-2018, 12:09 PM
 Matt G 365 --> 4039 AKA: Mattward FRC #4039 (Makeshift) Team Role: Mentor Join Date: Apr 2015 Rookie Year: 2009 Location: Ontario Posts: 71
Re: How does the FiveThirtyEight Elo formula work?

Quote:
 Originally Posted by JohnSchneider This guy explains it pretty well http://andr3w321.com/elo-ratings-par...y-adjustments/
He explains how to get to that equation, but in my opinion doesn't really explain any of the whys about the intricacies of the equation. Still an interesting read though. I also don't have the answer, but I won't let that stop me from speculating based entirely on irrelevant details.

Quote:
 MOVM = LN(ABS(PD)+1) * (2.2/((ELOW-ELOL)*.001+2.2))
I suspect that log_e was chosen over log_n because it doesn't make too much of a difference. The curve can be scaled by the second portion of the equation to get the appropriate response.

The form of the second portion allows for essentially two different responses, one for underdog victory and one for favorite victory. It's not a linear function, so underdog victories result in a larger MOVM than favorite victories of the same point differential. This is most significant for abs(ELOW-ELOL) > 500. For -500>ELOW-ELOL>500, it's mostly linear.

The 0.001 and 2.2 aren't really two different constants, they're interdependent (let's call them a and b, a=0.001 and b=2.2). If you take a=0.01 and b=22 you get the same result. The ratio of the two, a/b, determines the non-linearity of the second portion of the MOVM equation, as discussed previously. If you increase a and keep b constant, the function becomes more non-linear, increasing the difference between the reward for underdog victories and favorite victories.

The two also limit the scope of the function. In this case, for (ELOW-ELOL) <= -2200, it breaks. The more non-linear you'd like the function, the smaller of an elo difference it can allow (only for underdog victories of course, Rocky can lose all he wants but as soon as he beats Apollo you have issues).

So my post doesn't answer any of your questions directly, but hopefully it helps with deciding what values would be acceptable for FRC. I suspect log_e is appropriate for pretty much any competition, and the a/b constant is what would actually be tuned. When tuning a/b it's probably necessary to pay attention to the highest conceivable elo difference for an underdog win. Since you sum individual ELOs to achieve an alliance ELO, I suspect it's not unreasonable that you could break this equation given a large upset, but you can probably look at historical data to make sure.

If a and b aren't guess+check, I would think that they depend on the largest conceivable underdog win ELO difference and the prevalence of underdog victories. If underdog victories are common I would expect a more linear fit, but if they are uncommon I would expect a more non-linear fit. As to how to translate that into a number, , good luck!
#5
08-07-2018, 01:08 PM
 Karthik Registered User no team Team Role: Mentor Join Date: Apr 2002 Rookie Year: 1998 Location: Toronto, Ontario, Canada Posts: 2,643
Re: How does the FiveThirtyEight Elo formula work?

Quote:
 Originally Posted by Matt G The form of the second portion allows for essentially two different responses, one for underdog victory and one for favorite victory. It's not a linear function, so underdog victories result in a larger MOVM than favorite victories of the same point differential. This is most significant for abs(ELOW-ELOL) > 500. For -500>ELOW-ELOL>500, it's mostly linear.

Yup, exactly. This is explained by Silver at the link in the OP.

Quote:
 Originally Posted by Nate Silver Autocorrelation is the tendency of a time series to be correlated with its past and future values. Let me put this into football terms. Imagine I have the Dallas Cowboys rated at 1550 before a game against the Philadelphia Eagles. Their rating will go up if they win and go down if they lose. But it should be 1550 after the game, on average. That’s important, because it means that I’ve accounted for all the information you’ve given me efficiently. If I expected the Cowboys’ rating to rise to 1575 on average after the game, I should have rated them more highly to begin with. It’s true that if I have the Cowboys favored against the Eagles, they should win more often than they lose. But the way I was originally designed, I can compensate by subtracting more points for a loss than I give them for a win. Everything balances out rather elegantly. The problem comes when I also seek to account for margin of victory. Not only do favorites win more often, but when they do win, they tend to win by a larger margin. Since I give more credit for larger wins, this means that their ratings tend to get inflated over time.

Quote:
 Originally Posted by Matt G I suspect log_e is appropriate for pretty much any competition, and the a/b constant is what would actually be tuned. When tuning a/b it's probably necessary to pay attention to the highest conceivable elo difference for an underdog win.

Probably not a coincidence that the natural logarithm was chosen, given that e^2~=7, e^3 ~= 20, and that a "close" game is usually defined as MoV <=7, with diminishing returns are usually seen after MoV > 21 in the NFL. (e.g. The difference between 7 and 14 point win in the NFL is massive, while a 21 and 28 point win is fairly minor)
__________________
:: Karthik Kanagasabapathy ::
"Enthusiasm is one of the most powerful engines of success. When you do a thing, do it with all your might. Put your whole soul into it. Stamp it with your own personality. Be active, be energetic, be enthusiastic and faithful and you will accomplish your object. Nothing great was ever achieved without enthusiasm" -- R.W. Emerson
My TEDx Talk - The Subtle Secrets of Success

Last edited by Karthik : 08-07-2018 at 01:13 PM. Reason: Clarity on y = e^x
#6
08-07-2018, 02:05 PM
 Citrus Dad Business and Scouting Mentor AKA: Richard McCann FRC #1678 (Citrus Circuits) Team Role: Mentor Join Date: May 2012 Rookie Year: 2012 Location: Davis Posts: 1,354
Re: How does the FiveThirtyEight Elo formula work?

Quote:
 Originally Posted by Karthik (As to why base e was specifically chosen, again it comes to down to historical point distributions)
Natural logs have mathematical and statistical properties that fit with statistical theory (that I've now forgotten after 2 decades...)
__________________
#7
08-07-2018, 02:35 PM
 Matt G 365 --> 4039 AKA: Mattward FRC #4039 (Makeshift) Team Role: Mentor Join Date: Apr 2015 Rookie Year: 2009 Location: Ontario Posts: 71
Re: How does the FiveThirtyEight Elo formula work?

I hadn't read the posted article all the way through, only skimmed it. I read the second post article, but I definitely should have read the first one all the way through instead of just looking for numbers and equations.

Quote:
 Originally Posted by Karthik Probably not a coincidence that the natural logarithm was chosen, given that e^2~=7, e^3 ~= 20, and that a "close" game is usually defined as MoV <=7, with diminishing returns are usually seen after MoV > 21 in the NFL. (e.g. The difference between 7 and 14 point win in the NFL is massive, while a 21 and 28 point win is fairly minor)
That's good reasoning that I hadn't considered, however:

since ln(x) = 2.303*log(x), I can rewrite Mr. Silver's equation as:

MOVM = LOG(ABS(PD)+1) * (5.0666/((ELOW-ELOL)*0.002303+5.0666))

and I should get the same values. I think the selection of the base of the logarithm can be accounted for by selecting appropriate constants for the second portion of the equation.

Which led me to think that maybe the equation would be easier to interpret with constants of 1 and some number, rather than 2.2 and some number:

MOVM = Log_1.575(ABS(PD)+1) * (1/((ELOW-ELOL)*.0004545+1))

I like the form of 1/x+1, but it makes the logarithm pretty messy. And it leaves us wondering why Log_1.575 and 0.0004545 are significant... They're pretty ugly numbers.

Log_1.575(7) = 4.3
Log_1.575(21) = 6.7

For Caleb: Nate Silver is relatively accessible via twitter, so you could probably ask him there and have a shot at a reply. Although I'd probably wait until after today's primaries/special election are over, I bet he's pretty busy right now. There's also an email for him on fivethirtyeight.com, so you could try that as well... If you do ever get an answer from him, please post it here, I'm pretty interested.
#8
08-07-2018, 02:45 PM
 Karthik Registered User no team Team Role: Mentor Join Date: Apr 2002 Rookie Year: 1998 Location: Toronto, Ontario, Canada Posts: 2,643
Re: How does the FiveThirtyEight Elo formula work?

Quote:
 Originally Posted by Citrus Dad Natural logs have mathematical and statistical properties that fit with statistical theory (that I've now forgotten after 2 decades...)

Quote:
 Originally Posted by Matt G That's good reasoning that I hadn't considered, however: since ln(x) = 2.303*log(x), I can rewrite Mr. Silver's equation as: MOVM = LOG(ABS(PD)+1) * (5.0666/((ELOW-ELOL)*0.002303+5.0666)) and I should get the same values. I think the selection of the base of the logarithm can be accounted for by selecting appropriate constants for the second portion of the equation. Which led me to think that maybe the equation would be easier to interpret with constants of 1 and some number, rather than 2.2 and some number: MOVM = Log_1.575(ABS(PD)+1) * (1/((ELOW-ELOL)*.0004545+1)) I like the form of 1/x+1, but it makes the logarithm pretty messy. And it leaves us wondering why Log_1.575 and 0.0004545 are significant... They're pretty ugly numbers. Log_1.575(7) = 4.3 Log_1.575(21) = 6.7

Yup, the choice of the natural logarithm is quite possibly a result because the numbers fall into approximately the right space and because it's so readily used in traditional statistics/science.
__________________
:: Karthik Kanagasabapathy ::
"Enthusiasm is one of the most powerful engines of success. When you do a thing, do it with all your might. Put your whole soul into it. Stamp it with your own personality. Be active, be energetic, be enthusiastic and faithful and you will accomplish your object. Nothing great was ever achieved without enthusiasm" -- R.W. Emerson
My TEDx Talk - The Subtle Secrets of Success
#9
08-07-2018, 10:04 PM
 cobbie 1.21 Gigawatts AKA: Evan FRC #1425 (Error Code Xero) Team Role: Alumni Join Date: Mar 2017 Rookie Year: 2015 Location: West Linn, OR Posts: 161
Re: How does the FiveThirtyEight Elo formula work?

Here is an article about how they use it for the NBA.
__________________
Thanks to our 2018 alliance partners:
2471 1359 2046 5468 6445 16 418 4795

Thanks to our event winning alliance partners:
2007 Las Vegas 254 1983 2009 Seattle 488 1569 2012 Seattle Olympic 2557 2928
2013 Central Washington 1983 753 2014 Wilsonville 4488 2002 2014 Oregon St. 2811 4457
2016 Wilsonville 4488 3711 2016 Auburn 4469 3786 2017 CAIS 4488 6445
2018 Wilsonville 2471 1359 2018 Auburn Mountainview 2046 5468
#10
08-09-2018, 08:58 AM
 Caleb Sykes Knock-off Dr. Strange AKA: inkling16 no team (The Piztons) Join Date: Feb 2011 Rookie Year: 2009 Location: Minneapolis, Minnesota Posts: 1,771
Re: How does the FiveThirtyEight Elo formula work?

Quote:
 Originally Posted by cobbie Here is an article about how they use it for the NBA.
Here's the margin of victory multiplier from footnote 2 from that link, the general structure looks pretty similar to the NFL multiplier except using a power funtion instead of a logarithm.

Here's a graph of the MOV versus the multiplier for each of the two formulas for elo_diff = 200:

Note that it's kind of silly to directly compare NFL margins of victory against NBA MOVs since they have different units, but since the average MOV for each sport is ~10 points we can do at least a rough comparison. My guess is that the NFL has higher multipliers near 0 because NFL teams who are winning early can control the pacing of the game much better than NBA teams up by similar margins early can. So if NFL teams were actually playing to maximize margin of victory instead of win probability, teams with leads would probably play more aggressively than they presently do. That still doesn't tell us how either of these multipliers were constructed or tuned though.

I'll be sending Nate Silver an email soon, I'll share the whole email chain here after we're done.

 Thread Tools Display Modes Rate This Thread Linear Mode Rate This Thread: 5 : Excellent 4 : Good 3 : Average 2 : Bad 1 : Terrible

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts vB code is On Smilies are On [IMG] code is On HTML code is Off
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home Announcements     User Announcements FIRST     General Forum     Rumor Mill     Career     Robot Showcase Technical     Technical Discussion     Robotics Education and Curriculum     Motors     Electrical         CAN     Programming         NI LabVIEW         C/C++         Java         Python     Sensors     Control System     Pneumatics     Kit & Additional Hardware     CAD         Inventor         SolidWorks         Creo     IT / Communications         3D Animation and Competition         Website Design/Showcase         Videography and Photography         Computer Graphics Competition     Awards         Chairman's Award     Rules/Strategy         Scouting     Team Organization         Fundraising         Starting New Teams         Finding A Team         College Teams     Championship Event     Regional Competitions     District Events     Off-Season Events     Thanks and/or Congrats     FRC Game Design     OCCRA         OCCRA Q&A Other     Chit-Chat         Games/Trivia         Fantasy FIRST     Car Nack's Corner     College & University Education     Dean Kamen's Inventions     FIRST-related Organizations         The Blue Alliance     FIRST In the News...     FIRST Lego League     FIRST Tech Challenge     VEX     Televised Robotics     Math and Science     Robot in 3 Days (RI3D)     NASA Discussion ChiefDelphi.com Website     CD Forum Support     Extra Discussion

All times are GMT -5. The time now is 02:44 PM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.

 -- English (12 hour) -- English (24 hour) Contact Us - Chief Delphi - Rules - Archive - Top