Quote:
Originally Posted by nuclearnerd
1) to derive a strength of schedule (basically the predicted winning margin). This is useful for deciding where best to spend limited resources on strategy planning, and alliance-mate repairs.
2) as a first order sort for pick list order
Both of these use cases suffer from even more drastic lack of data points. For the SOS, I use previous tournament data, and for the pick list, we've usually only completed 70% of the matches. For these use cases, it would be valuable to know:
for 1) Which predictor does best when the training set and the testing set are from different tournaments.
For 2) which predictor does best with 70% or less of the matches in the training set. (Maybe this is where MMSE solutions will shine)
|
This whole thing came about because I wrote an App for FTC tournaments where I wanted live estimates of stats and predicted match outcomes. The app predicts future match results based on stats so teams can know which matches are likely to be hard vs. easy. It also does this for sub-parts of the game (e.g., 2 matches from now, our opponents are really good at autonomous but not at the end game, so maybe we should play autonomous defense [in FTC], etc.).
OPR doesn't even exist when the # of matches played is less than the number of teams/2, and then suddenly it exists but is noisy, and then it progressively gets less noisy as more matches are played. So I was looking for a way to show stats, and it seemed like the stats should slowly incorporate information as matches are played.
The app currently predicts match scores and winning margin, but I'd also like to incorporate a "probability of victory" measure to show what kind of confidence exists.
The MMSE approach allows for estimated stats regardless of how many matches are played. I'll try to run some sims with 0-100%of matches played to see how well things work over time.
It also occurred to me to try to predict the match outcomes for the simulated tournaments where the underlying stats are completely known just to see the limits of how well match prediction could be if perfect knowledge of the underlying parameters existed.
Another thing that I could do would be to simulate how picking alliances based on the estimated stats would do vs. picking them based on the underlying stats. For example, if the top 3 teams are picked based on the various estimates (LS OPR, MMSE OPR, MMSE sCPR, etc.) and they are compared with the top 3 teams in the simulated tournaments where the underlying actual data is known (the actual underlying O and D), how many fewer points will the alliance end up scoring on average? This might be the real question that folks want to know... Gotta run now: more later.