Quote:
Originally Posted by dtengineering
I'd say that the "predictions" for upcoming match results will be a good way to determine when the data starts to become significant. It would be great to have an analysis of what fraction of the predictions are correct... it would be an excellent way to show at what point the OPRs become an effective indicator of match outcome.
But OPRs and scouting reports are always useful, if only for entertainment's sake!
Jason
|
Older versions of OPRNet had a "self-check" mode for their predictor - if a regional was complete when someone asked for a prediction, it would tell you how good the predictions would have been if it had made them with the data available after each match. So it would say:
"After match 59, the predictions for matches 60-80 would have been 65% correct"
"After match 60, the predictions for matches 61-80 would have been 64% correct" (it varied a lot)
Generally what I found was that on friday night, saturday predictions based on the sum of OPRs for each alliance were 60-70% correct, which is obviously way better than a coin toss. This depends on the game though - OPR is better or worse depending on what the game is like. See the 1-hour mark of Karthik's presentation at champs last year for more details.
Post-event (aka once you know all the data) it was 80ish% correct, but that's a fairly worthless stat: once the event is over, anyone can get 100% of predictions correct!