Best way to predict match outcomes

[Michael Hill]

3138 has a very good match prediction system. It's good enough to correctly predict who is going to win the regional just after alliance selections, even when "underdog" #7 alliances beat #2 alliances (though that happens quite a bit now).

[Gregor]

Originally Posted by Michael Hill

3138 has a very good match prediction system. It's good enough to correctly predict who is going to win the regional just after alliance selections, even when "underdog" #7 alliances beat #2 alliances (though that happens quite a bit now).

To be fair, that's not that difficult most of the time.

[DampRobot]

I just looked at OPR to project the 2013 SVR Satuday match results and rankings . (See the regional thread for more details.) I accurately predicted a lot of the rankings Friday night, and the regional winners (the captain wasn't even ranked #1 at the time). However, as others have pointed out, this usually isn't all that difficult. Conventional wisedom, as well as my projections, would have put 254 and 118 as the regional winners.

As to projecting matches/seedings to be pointless: it is and it isn't. As many have pointed out, you never know how a match will be played until its actually played. However, it's useful to have a good guess about how rankings will turn out in order to get ready for selections and matches on Saturday.

Maybe it isn't critical to know exactly how each match will turn out, but scouting as a whole is important. Projected rankings play an important role in scouting for aliance selection.

[brennonbrimhall]

Originally Posted by DampRobot

I just looked at OPR to project the 2013 SVR Satuday match results and rankings.

I understand that OPR related models have around 80% accuracy this year, which has been phenomenal. However, this calculation would have meant next to nothing for Rebound Rumble, for instance. What did you guys do that year?

For others, I'll pose the question again. Regardless of the usefulness of the information, can anyone share their match prediction models? I'd love to know how they're calculated, used, and how accurate they are.

[IKE]

When talking about predictors, I think understanding the usefulness is very important. 33 likes to use it is a way to gauge strength of schedule and predict rankings as well as matches to watch out for. First event is always the toughest as there is no "going in" data.
Using a match predictor won't turn a 2/10 team into a 10/2 team, but it often helps go from 9/3 to 10/2. while winning an extra match doesn't sound that impressive, look at many of the standings, and you will find that the difference between loosing 2 and loosing 3 matches is frequently the difference between being a captain or not.
As our data comes in, we will put predicting scoring ranges on our scouting card we give to the coach.
It sounds like Team 20 has a very neat method for doing their predictions, and the accuracy sounds pretty impressive though you have to look at the accuracy relative to the known data you have. Post predicting is way different than pre-predicting.
Overall, it isn't the most critical thing your team can do, but it may help you pay attention to the match you thought would be easy (and end up loosing) or help you strategize turning a loosing match into a win.

For all of these methods, a 75% accuracy level of who is going to win is probably close enough to pay attention to.

For 2012, we would use component OPR to help decide who should be doing the balancing for the Co-Op bridge.

[brennonbrimhall]

Originally Posted by IKE

It sounds like Team 20 has a very neat method for doing their predictions, and the accuracy sounds pretty impressive though you have to look at the accuracy relative to the known data you have.

While invalid data could always throw off a model, our current model's accuracy is rather lacking. I'm hoping that someone would share a model they found useful and worked for them.

Did the Killer Bees use anything like this?

[IKE]

Originally Posted by brennonbrimhall

While invalid data could always throw off a model, our current model's accuracy is rather lacking. I'm hoping that someone would share a model they found useful and worked for them.

Did the Killer Bees use anything like this?

"average" match performance leadign into the match, and OPR Friday night for Ranking predictions. We would then look at the top 16 or so and see if any close matches might go the otehr direction, and how those would effect the rankings.

For the Championship, we will frequently use previous best OPR to run the schedule as soon as it is available looking for close matches to focus on and/or any potential "never going to win" that we could somehow pull a rabbit out of our hats.

Even if your robot isn't in contention, I recommend working on the tools to have your team ready in case it is. We were easily out of the Top 3 on Archimedes (254, 469, and 987 were easily the 3 best on Archi), but some luck with the schedule and skill in some key matches put us in the spot to be the #2 captain. I would put this in a similar tool-kit as making a pick-list. Even if you aren't a captain, you should make a pick-list just in case you are picked by someone without a list, or just to have practice to make a better one.

[Iaquinto.Joe]

At the Bedford District Event my fellow programmers and I took to using the sum of the OPR of an alliance's members to predict match results and rankings. We were about 85% accurate for the 25 or so matches we predicted on saturday. We also predicted our ranking to be 11-13 and we ended 12th. It was fun.

[ThunderChief]

I used Max OPR to predict matches at the Championships. (I have been analyzing the statistics and plan to do a white paper.) I adjusted the Max OPR for each team by adding a factor depending on how well each team did at each regional starting with a couple bonus points if a team went to more than 1 regional. Then for each match, I figured the better teams would improve more than the average teams, so I increased the OPR of the best team on the alliance and decreased the OPR of the weakest team by percentages. Then I adjusted the sum of the OPRs up to equal the predicted, or later, actual average points per alliance.

If this predicted we were going to lose a match, then we had to change our alliance strategy from what was "normal". Maybe we would have one of our alliance partners play defense against the highest OPR opponent. Or maybe use a robot for counter-defense to allow our Full Court Shooter partner to shoot disks without an opponent defender in the way. The prediction was based on the past performance. If it predicted we would lose, then we had to do something different than the past.

Since we predicted we would not be ranked in the top 8, if the prediction was to easily win a match, we would do things to show scouts that we had flexibility like shooting from different locations, climbing a different side or picking up disks from the floor.

We changed a 3 win 5 loss match schedule strength prediction into a 5 and 3 result. And we almost won 2 matches that we lost by a very small margin.

We ended up being picked #3 by Team 303 (also picked Team 1640) and our #3 alliance won the Newton Division Championship.
So we did something right.
Also, my predictions were 100% accurate for the Saturday matches involving the top ranked teams so we knew in advance which teams to talk to that we wanted to pick us.

[Kevin Leonard]

Originally Posted by ThunderChief

Also, my predictions were 100% accurate for the Saturday matches involving the top ranked teams so we knew in advance which teams to talk to that we wanted to pick us.

Kinda funny- we predicted one match incorrectly, and that completely messed up our plan for the final divisional standings in Archimedes.

It certainly changed things.
One match put a team that (according to our data) was supposed to rank #2 above 33. Imagine now if another team had picked 469 instead of 33. Einstein would have looked a LOT different.

I imagine that could be a part of the reason my friend Brennon posted this thread.

I like your method (and your username!) Thunderchief, in that it accounts for teams getting better, but by what criteria did you account for a team getting better? Was it
"I've heard of this team, they probably got better."
Or was it: "This team has been to three regionals, they continuously improved by about 6 points, therefore they'll improve by another 6 for CMP"

A little more info on the exact criteria you used to differentiate between teams would be helpful.

[brennonbrimhall]

Originally Posted by ThunderChief

(I have been analyzing the statistics and plan to do a white paper.) ... my predictions were 100% accurate for the Saturday matches involving the top ranked teams...

Definitely looking forward to that whitepaper!

[Samwaldo]

My method, and most popular.

GUESS.

Makes it all more exciting and often suspensful

[Tim Delles]

One of these days I'll post up my scouting database.

Basically how it works:

1. Every point a team scores / defends is tallied.
2. These are then turned into inputs
3. After a set amount of matches, averages (Average Offensive Contribution/Average Swing Contribution... for those in VEX) for each period of play are created along with standard deviations and confidence intervals.
4. Once this has started to be calculated both strength of schedule (SOS) and Strength of Partners (SOP) are calculated based on the teams you have went against and the teams you have played with.
5. Based on each teams average score, an average alliance score is then computed, along with the max and min scores that fall within 2 deviations. (the more data points... matches... the more accurate this becomes)

Once this is calculated, if you an alliance has an 80% chance or better of winning, the program announces that alliance as the winner and outputs the most likely score.

The downfalls of this, it requires 100% accurate inputs (I started this program in 2011 and was able to accurately predict championship matches to within +/- 2 points). As the students have taken over data collection/entry, games have become more volatile, etc the ability to get extremely close to match scores has become much more difficult (needing more samples).

I'll see what I can do about polishing up the scouting database and posting something up for people to look through.

[JackS]

Andrew Schreiber and I tried to put together and ELO Chess model that wasn't that accurate for 2012. I am suspicious that the bridge might have been responsible for that. We will eventually tweak the model and run it on 2013. Ideally it would produce accurate ratings for teams (more accurate than OPR) and those ratings would be good match predictors.

Personally, I'd trust my gut on a picklist over any list generated by a match predictor. To me, match prediction is some cool math, but isn't that practical.

[brennonbrimhall]

Originally Posted by Tim Delles

The downfalls of this, it requires 100% accurate inputs (I started this program in 2011 and was able to accurately predict championship matches to within +/- 2 points)...I'll see what I can do about polishing up the scouting database and posting something up for people to look through.

Very interesting. I'd love to see what you and your team came up with!

Originally Posted by JackS

Andrew Schreiber and I tried to put together and ELO Chess model...

Could you explain what an ELO Chess model is?