Team 1410 is proud to finally announce Project Nautilus, a twitter bot which predicts the outcomes of FRC matches.
Nautilus uses 3 different methods to predict match outcomes. The first is a modified ELO system based off of Caleb Sykes’ system (paper: FRC Elo 2008-2016 - CD-Media - Chief Delphi). The second method is a a new match prediction method which we have developed over the past several months that we call MAR. MAR was inspired by David Burge’s Marble game, an effort to create a better ranking system for college football.
In 2018, our modified ELO had a 77.6% accuracy rate in predicting matches and MAR had a 70.9% accuracy rate.
Our third match prediction method is something that we call EMZ. EMZ is a combination of ELO, MAR, and some special sauce. EMZ had an accuracy rate of 72.6% last year, and it tends to have a better brier score than ELO or MAR. Here is a brief explanation of brier scores: Brier Score: Definition, Examples - Statistics How To. All of our match prediction method “run hot” in the sense that the predictions for match 72 are affected by the previous matches played.
We ran Project Nautilus’ prediction systems for all qualification matches during week one and two events. For those events, our modified ELO had a 68.50% accuracy rate in predicting matches, and MAR had an accuracy rate of 66.64%. EMZ had an accuracy rate of 67.5% this year so far. If you decide that any match with less than 65% prediction confidence is too close to call and discount those predictions, the accuracy of ELO jumps to 75.57%, MAR to 71.55%, and EMZ to 78.32%.
Project Nautilus will predict the qualification matches of the San Francisco Regional this week, and if everything goes well, will also predict the Colorado Regional during week four. We will continue to predict matches throughout the season, but have not finalized a list of events we will be predicting. To see our predictions, follow Project Nautilus on Twitter: @Nautilus_1410
As we are just launching this for the first time this week, something might go wrong and we may have to temporarily shut down Project Nautilus — if we do so, we will tweet about it on the @Nautilus_1410 account.
You can find Caleb Sykes’ ELO, which we used as a starting point for our ELO system here: paper: FRC Elo 2008-2016 - CD-Media - Chief Delphi
An explanation of the Marble game for college football that MAR was inspired by can be found here: The Marble Game in R - @alspur.
Special thanks to Lori V. from team 5414 for making Project Nautilus’ profile photo.