Biggest upsets of 2016

[Caleb Sykes]

Originally Posted by weaversam8

Hey Caleb!

If you're working with prediction models for FRC, I'd suggest you checkout what my team, The Wired Wizards, did with ORB.

The link to the forum post is here: https://www.chiefdelphi.com/forums/s...d.php?t=147817

We worked with match prediction specific to Stronghold, doing analysis based on teams affinity with different defenses... there's more information in the post...

Good luck!

Did you ever see how accurate your predictions were? After training with data from 2011-2014, my current Elo model gets a Brier score of 0.1792 for predicting the winners of the 2016 Championship qual and playoff matches. I'm planning to try to predict matches with OPR, CCWM, and win contribution in the future and compare results.

[Caleb Sykes]

Originally Posted by Caleb Sykes

Did you ever see how accurate your predictions were? After training with data from 2011-2014, my current Elo model gets a Brier score of 0.1792 for predicting the winners of the 2016 Championship qual and playoff matches. I'm planning to try to predict matches with OPR, CCWM, and win contribution in the future and compare results.

Alternatively, it predicted 72.3% of the winning alliances correctly.

[weaversam8]

The project was exploratory, we were learning, but off the top of my head I think it was between 60 and 70 percent accurate.

[Chris is me]

Originally Posted by Caleb Sykes

Did you ever see how accurate your predictions were? After training with data from 2011-2014, my current Elo model gets a Brier score of 0.1792 for predicting the winners of the 2016 Championship qual and playoff matches. I'm planning to try to predict matches with OPR, CCWM, and win contribution in the future and compare results.

If you're going to compare to all of the various methods, would be cool to also compare to average qual score; anything less predictive than that is worthless basically.

[Caleb Sykes]

Originally Posted by Chris is me

If you're going to compare to all of the various methods, would be cool to also compare to average qual score; anything less predictive than that is worthless basically.

Do you mean each team's average qual score for the matches they have already played? Average qual score for the event would just predict a tie every match.

[Chris is me]

Originally Posted by Caleb Sykes

Do you mean each team's average qual score for the matches they have already played? Average qual score for the event would just predict a tie every match.

Yes, the average qual score of the team's previous matches; just like OPR uses the data from the team's previous qual matches, etc. OPR was always meant to be an improvement over average score. You can also do average winning margin as an alternate metric. OPR is supposed to be better than average score and CCWM is supposed to be better than average winning margin.

[Andrew Schreiber]

Originally Posted by Chris is me

If you're going to compare to all of the various methods, would be cool to also compare to average qual score; anything less predictive than that is worthless basically.

Wanna be lazier, try just predicting w/l based off existing win ratio for alliance at the event. I have a hunch it's as good as anything else.