Best way to predict match outcomes

[IKE]

Originally Posted by brennonbrimhall

While invalid data could always throw off a model, our current model's accuracy is rather lacking. I'm hoping that someone would share a model they found useful and worked for them.

Did the Killer Bees use anything like this?

"average" match performance leadign into the match, and OPR Friday night for Ranking predictions. We would then look at the top 16 or so and see if any close matches might go the otehr direction, and how those would effect the rankings.

For the Championship, we will frequently use previous best OPR to run the schedule as soon as it is available looking for close matches to focus on and/or any potential "never going to win" that we could somehow pull a rabbit out of our hats.

Even if your robot isn't in contention, I recommend working on the tools to have your team ready in case it is. We were easily out of the Top 3 on Archimedes (254, 469, and 987 were easily the 3 best on Archi), but some luck with the schedule and skill in some key matches put us in the spot to be the #2 captain. I would put this in a similar tool-kit as making a pick-list. Even if you aren't a captain, you should make a pick-list just in case you are picked by someone without a list, or just to have practice to make a better one.

[qzrrbz]

Originally Posted by EricH

Exactly my thought. The best way to predict the match is to play it.

Consider this: in 2010, 469, 1114, and their third partner (I forget who it was) were favored so heavily to win Championship that very few people would have predicted them not winning. Predictions would probably have showed them winning. That was before Murphy's Law struck that alliance with a vengeance on Einstein--things just started failing for no apparent reason, and they did not win the Championship.

[channel the Church Lady] Hmmm.... Let's see here.... Could it have been ...

CURIE?

[Iaquinto.Joe]

At the Bedford District Event my fellow programmers and I took to using the sum of the OPR of an alliance's members to predict match results and rankings. We were about 85% accurate for the 25 or so matches we predicted on saturday. We also predicted our ranking to be 11-13 and we ended 12th. It was fun.

[ThunderChief]

I used Max OPR to predict matches at the Championships. (I have been analyzing the statistics and plan to do a white paper.) I adjusted the Max OPR for each team by adding a factor depending on how well each team did at each regional starting with a couple bonus points if a team went to more than 1 regional. Then for each match, I figured the better teams would improve more than the average teams, so I increased the OPR of the best team on the alliance and decreased the OPR of the weakest team by percentages. Then I adjusted the sum of the OPRs up to equal the predicted, or later, actual average points per alliance.

If this predicted we were going to lose a match, then we had to change our alliance strategy from what was "normal". Maybe we would have one of our alliance partners play defense against the highest OPR opponent. Or maybe use a robot for counter-defense to allow our Full Court Shooter partner to shoot disks without an opponent defender in the way. The prediction was based on the past performance. If it predicted we would lose, then we had to do something different than the past.

Since we predicted we would not be ranked in the top 8, if the prediction was to easily win a match, we would do things to show scouts that we had flexibility like shooting from different locations, climbing a different side or picking up disks from the floor.

We changed a 3 win 5 loss match schedule strength prediction into a 5 and 3 result. And we almost won 2 matches that we lost by a very small margin.

We ended up being picked #3 by Team 303 (also picked Team 1640) and our #3 alliance won the Newton Division Championship.
So we did something right.
Also, my predictions were 100% accurate for the Saturday matches involving the top ranked teams so we knew in advance which teams to talk to that we wanted to pick us.

[Kevin Leonard]

Originally Posted by ThunderChief

Also, my predictions were 100% accurate for the Saturday matches involving the top ranked teams so we knew in advance which teams to talk to that we wanted to pick us.

Kinda funny- we predicted one match incorrectly, and that completely messed up our plan for the final divisional standings in Archimedes.

It certainly changed things.
One match put a team that (according to our data) was supposed to rank #2 above 33. Imagine now if another team had picked 469 instead of 33. Einstein would have looked a LOT different.

I imagine that could be a part of the reason my friend Brennon posted this thread.

I like your method (and your username!) Thunderchief, in that it accounts for teams getting better, but by what criteria did you account for a team getting better? Was it
"I've heard of this team, they probably got better."
Or was it: "This team has been to three regionals, they continuously improved by about 6 points, therefore they'll improve by another 6 for CMP"

A little more info on the exact criteria you used to differentiate between teams would be helpful.

[brennonbrimhall]

Originally Posted by ThunderChief

(I have been analyzing the statistics and plan to do a white paper.) ... my predictions were 100% accurate for the Saturday matches involving the top ranked teams...

Definitely looking forward to that whitepaper!

[Samwaldo]

My method, and most popular.

GUESS.

Makes it all more exciting and often suspensful

[Tim Delles]

One of these days I'll post up my scouting database.

Basically how it works:

1. Every point a team scores / defends is tallied.
2. These are then turned into inputs
3. After a set amount of matches, averages (Average Offensive Contribution/Average Swing Contribution... for those in VEX) for each period of play are created along with standard deviations and confidence intervals.
4. Once this has started to be calculated both strength of schedule (SOS) and Strength of Partners (SOP) are calculated based on the teams you have went against and the teams you have played with.
5. Based on each teams average score, an average alliance score is then computed, along with the max and min scores that fall within 2 deviations. (the more data points... matches... the more accurate this becomes)

Once this is calculated, if you an alliance has an 80% chance or better of winning, the program announces that alliance as the winner and outputs the most likely score.

The downfalls of this, it requires 100% accurate inputs (I started this program in 2011 and was able to accurately predict championship matches to within +/- 2 points). As the students have taken over data collection/entry, games have become more volatile, etc the ability to get extremely close to match scores has become much more difficult (needing more samples).

I'll see what I can do about polishing up the scouting database and posting something up for people to look through.

[JackS]

Andrew Schreiber and I tried to put together and ELO Chess model that wasn't that accurate for 2012. I am suspicious that the bridge might have been responsible for that. We will eventually tweak the model and run it on 2013. Ideally it would produce accurate ratings for teams (more accurate than OPR) and those ratings would be good match predictors.

Personally, I'd trust my gut on a picklist over any list generated by a match predictor. To me, match prediction is some cool math, but isn't that practical.

[ZRubenfeld]

I have an excel workbook that I programmed. It has a 80% accurate prediction rate. If you want the workbook, I'll send you it via email.

[DampRobot]

Originally Posted by JackS

Andrew Schreiber and I tried to put together and ELO Chess model that wasn't that accurate for 2012. I am suspicious that the bridge might have been responsible for that. We will eventually tweak the model and run it on 2013. Ideally it would produce accurate ratings for teams (more accurate than OPR) and those ratings would be good match predictors.

ELO Chess model? This sounds very interesting. Care to go into a little more detail?

[brennonbrimhall]

Originally Posted by Tim Delles

The downfalls of this, it requires 100% accurate inputs (I started this program in 2011 and was able to accurately predict championship matches to within +/- 2 points)...I'll see what I can do about polishing up the scouting database and posting something up for people to look through.

Very interesting. I'd love to see what you and your team came up with!

Originally Posted by JackS

Andrew Schreiber and I tried to put together and ELO Chess model...

Could you explain what an ELO Chess model is?

[Siri]

Originally Posted by Tim Delles

Every point a team scores / defends is tallied

How do you count every point a team defends?

[runneals]

Our scouting system was dead-on at KC where we predicted Saturday matches with a 97% accuracy. That was using alliance data as a whole, which we pulled directly from our friend's scouting app. However, when we went to North Star, we had to input our scouting cards, as the Blue Alliance was messed up, and we ended up with 88% accuracy of Saturday matches.

Has anyone else gotten +97% accuracy on Saturday match predictions from Friday matches? I think it will be SUPER interesting to see how IRI works with our system

[brennonbrimhall]

Originally Posted by runneals

Our scouting system was dead-on at KC where we predicted Saturday matches with a 97% accuracy.

While your scouting system on your website goes into detail about how you collect your data, it's a bit more vague on what you do with the data you've collected. Can you elaborate on how you arrive at your predictions?