PNW District Championship Projections 2023

Competition
1

PNW District Championship Projections 2023

We’re only two weeks in, but a majority of teams in the district have already played and some have already played two. And we’ve already got 15 teams who we know are in or out of the district championship.

I’ve made predictions for every previous PNW district championship:

2014: 2014 Autodesk PNW District Championship - #9 by SoftwareBug2.0
2015: PNW District Championship Projections 2015
2016: PNW District Championship Projections
2017: PNW District Championship Projections 2017
2018: PNW District Championship Projections 2018
2019: PNW District Championship Projections 2019
2022: PNW District Championship Projections 2022

The methodology is the same as the last couple and you can see details in the previous threads, but I’m happy to answer questions.

Here’s where I think the district championship point cutoff is likely to be:
Points Probability
52 0.001
53 0.0045
54 0.014
55 0.044
56 0.0525
57 0.0805
58 0.107
59 0.121
60 0.1195
61 0.1355
62 0.1115
63 0.0855
64 0.0625
65 0.0335
66 0.015
67 0.008
68 0.003
69 0.0015

And here’s where teams stand now. A quick explanation of the columns:

  1. Team number
  2. Probability of making the district championship
  3. How many more points would be needed to get to a 5% chance of making DCMP
  4. Pts to get to 50% chance
  5. Pts to get to 95% chance

Team # P(DCMP) Pts 5% Pts 50% Pts 95% Nickname
6831 1 0 0 0 A-05 Annex
4125 1 0 0 0 Confidential
2412 1 0 0 0 Robototes
4488 1 0 0 0 Shockwave
1540 1 0 0 0 Flaming Chickens
3218 1 0 0 0 Panther Robotics
2930 1 0 0 0 Sonic Squirrels
2910 1 0 0 0 Jack in the Bot
2471 1 0 0 0 Team Mean Machine
1425 1 0 0 0 Error Code Xero
7034 0.998 0 0 4 2B Determined
2521 0.984 0 2 7 SERT
4131 0.974 0 3 8 Iron Patriots
2046 0.96 0 4 9 Bear Metal
7627 0.942 0 5 10 Bearcat Robotics
3636 0.942 0 5 10 Generals
955 0.921 1 6 11 CV Robotics
3711 0.84 4 9 14 Iron Mustang
6465 0.732 8 13 18 Mystic Biscuit
2147 0.71 9 14 19 CHUCK
5920 0.67 11 16 21 VIKotics
753 0.652 12 17 22 High Desert Droids
5975 0.652 12 17 22 Beta Blues
4915 0.652 12 17 22 Spartronics
2522 0.634 13 18 23 Royal Robotics
4061 0.615 14 19 24 SciBorgs
9036 0.576 16 21 26 Ramen Robotics
4911 0.555 17 22 27 CyberKnights
1778 0.555 17 22 27 Chill Out
6343 0.532 18 23 28 Steel Ridge Robotics
2990 0.532 18 23 28 Hotwire
9023 0.526 45 50 55 Future Martians
3674 0.51 19 24 29 CloverBots
4682 0.465 21 26 31 CyBears
4512 0.465 21 26 31 Otter Chaos
8896 0.449 50 55 60 Wolf Tech Robotics
488 0.443 22 27 32 Team XBOT
4043 0.423 23 28 33 NerdHerd
4060 0.403 24 29 34 Bearcat Robotics
1318 0.403 24 29 34 Issaquah Robotics Society
8532 0.384 25 30 35 Classified
948 0.379 55 60 65 NRG (Newport Robotics Group)
8386 0.379 55 60 65 Th3_IRON_G0@+$
8303 0.379 55 60 65 G0LDEN_GAT0RS
8302 0.379 55 60 65 Ni-Tro
8051 0.379 55 60 65 TinkRex
8032 0.379 55 60 65 Redshift
7461 0.379 55 60 65 Sushi Squad
6076 0.379 55 60 65 Mustang Mechanica
5937 0.379 55 60 65 MI-Robotics
5827 0.379 55 60 65 Code Purple
5683 0.379 55 60 65 Riverside Robotics
492 0.379 55 60 65 Titan Robotics Club
4918 0.379 55 60 65 The Roboctopi
4692 0.379 55 60 65 Metal Mallards
4579 0.379 55 60 65 RoboEagles
4469 0.379 55 60 65 R.A.I.D. (Raider Artificial Intelligence Division)
4180 0.379 55 60 65 Iron Riders
3876 0.379 55 60 65 Mabton LugNutz
3786 0.379 55 60 65 Chargers
3663 0.379 55 60 65 CPR - Cedar Park Robotics
3588 0.379 55 60 65 the Talon
3393 0.379 55 60 65 Horns of Havoc
3219 0.379 55 60 65 TREAD (Trojan Robotics, Engineering, Art, & Design)
3070 0.379 55 60 65 Team Pronto
2929 0.379 55 60 65 Emeral Ridge JAGBOTS
2907 0.379 55 60 65 Lion Robotics
2906 0.379 55 60 65 Sentinel Prime Robotics
2557 0.379 55 60 65 SOTAbots
1983 0.379 55 60 65 Skunk Works Robotics
1294 0.379 55 60 65 Pack of Parts
6443 0.348 27 32 37 AEMBOT
2980 0.348 27 32 37 Whidbey Island Wildcats
2811 0.348 27 32 37 StormBots
4681 0.331 28 33 38 Murphys law
360 0.314 29 34 39 The Revolution
2976 0.314 29 34 39 The Spartabots
5468 0.297 30 35 40 Chaos Theory
4450 0.297 30 35 40 Olympia Robotics Federation
4173 0.297 30 35 40 IMVERT (Mount Vernon Robotics Team)
2926 0.297 30 35 40 Robo Sparks
1595 0.297 30 35 40 The Dragons
1899 0.28 31 36 41 Saints Robotics
997 0.264 32 37 42 Spartan Robotics
3826 0.264 32 37 42 Sequim Robotics Federation “SRF”
2374 0.264 32 37 42 Crusader Bots
5295 0.232 34 39 44 Aldernating Current
957 0.217 35 40 45 SWARM - South and West Albany Robotics Maniacs
5941 0.217 35 40 45 Cast Iron Orcas
2898 0.203 36 41 46 Flying Hedgehogs
2635 0.203 36 41 46 Lake Monsters
4513 0.19 37 42 47 Circuit Breakers
3673 0.19 37 42 47 C.Y.B.O.R.G. Seagulls
3049 0.177 38 43 48 BremerTron
2928 0.166 39 44 49 Viking Robotics
8248 0.155 40 45 50 ChainLynx
5588 0.155 40 45 50 Reign Robotics
4089 0.155 40 45 50 Stealth Robotics
568 0.136 42 47 52 Nerds of the North
2550 0.136 42 47 52 Skynet
1432 0.128 43 48 53 Metal Beavers
4980 0.121 44 49 54 Canine Crusaders
3712 0.114 45 50 55 RoboCats
1359 0.114 45 50 55 Scalawags
3681 0.109 46 51 56 Robo-Raiders
2903 0.109 46 51 56 NeoBots
2733 0.109 46 51 56 Pigmice
949 0.103 47 52 57 Wolverine Robotics
847 0.0984 48 53 58 PHRED
4104 0.0984 48 53 58 Error-4104
3268 0.0984 48 53 58 Vahallabots
3024 0.0984 48 53 58 My Favorite Team
2097 0.0892 50 55 60 Phoenix Force Robotics
6350 0.0848 51 56 61 Clawbots
4127 0.028 0 5 10 LoggerBots
6845 0 11 16 21 River Bots
6696 0 23 28 33 Cardinal Dynamics
5970 0 24 29 34 Beavertronics
4662 0 21 26 31 Byte Sized Robotics
2915 0 28 33 38 Pandamonium

And here’s an easier to read version:
2023pnw.html.zip (6.9 KB)

10 Likes
2

Nice, the html link is much more readable.

I think it would be useful to have the amount of district points a team has received from each event listed on the table as well. It makes it a bit easier to guess how much a team needs to improve / avoid falling off at their second event to make DC. Especially early in the season where some teams haven’t played at all.

2 Likes
3

How exactly do you calculate the probabilities? Is it a simulation, a complex equation, or something else?

4

Here’s this week’s results, with the data about how teams have done so far incorporated into the main table.

2023pnw.html.zip (7.0 KB)

3 Likes
5

Basically, the idea is:

  1. Calculate a probability distribution for each team’s total given how many points they have earned so far and how many point-earning events they have left.
  2. Calculate a probability distribution for what the point cutoff might be based on adding those together and how big the district championship is.
  3. For each team, see how often what came from step 1 is above what came from step 2.

For figuring out the odds of making the championship, it’s a similar calculation but you have to make a new estimate for each team that is conditional on them making the district championship event.

The code is available here:

1 Like
6

Will you be doing something similar for other districts like FIM?

7

Yes, I here’s a link:

1 Like
8

will you be updating the projections as weeks/competitions happen, or no?

9

Yes, the intention is to update weekly.

If you want to see the results sooner than I post, the code should be pretty portable if you want to run it. The one caution that I would give you is that it doesn’t really understand what’s going on when an event is half finished. So if there’s an event in progress in whatever district you want to look at the answers will be skewed.

1 Like
10

Awesome, thank you so much!

11

One thing I’m confused about- for example, the 40th team 1318, the IRS, apparently has a 40.3% chance of getting into DCMP. But according to the score cutoff chart, there is a total of 99.5% chance of the score cutoff being at 67 points (IRS’s total points) or lower. How does that work?

12

The reason for the 40.3% odds of 1318 making DCMP was because it was not taking into account their performance at their second event. And the reason that it didn’t take into account their second event is that it had not yet occurred.

Here’s a link to the predictions from a week later:

And you will see that they’re given much better odds (over 99%).

I was going to wait to give out an update for this week until events were done in all districts, but I’ll give you the new PNW results now. And as the uncertainty about the cutoff decreases, my latest estimate is that 1318 is 100% in.

2023pnw.html.zip (6.8 KB)

4 Likes
13

Appreciate the update.

I don’t mean to criticize, but is it really a projection if all the teams who have not completed all their matches are at a disadvantage in the model? It seems like there’s a center “chunk” of teams that are not necessarily low-scoring, but simply have not played their second match yet. Isn’t it possible to duplicate their performance (however that’s calculated in this model) in the first competition somehow and rank them that way? Please correct me if the projection is working as intended.

14

This is not exactly correct, but still a fair criticism. The model is set up to not account for team skill. For all teams with unplayed events, it gives them the same distribution of point totals per remaining event. Thus the estimates of teams with remaining events will be skewed towards average. This is not inherently positive or negative for teams with events left to play because being thought of as average is an underestimate if you’re good, but an overestimate if you’re not.

Going a bit deeper, since the 2023 PNW district has only 50/120 advance this year, being perfectly in the middle is bad for you. But if we were looking at something like Indiana 2022, you’d have 32/51 advancing so looking average nudges you to the right side of the cutoff.

It would certainly be possible to do a calculation that would take skill into account. In fact, I’ve done that before. My older estimates would just categorize teams into five categories of probability essentially based on what’s the worst/best that someone with their district point total and one event has done in their other.

That method is not directly implementable in the current system though. With enough data you could just calculate if a team gets N district points in their first event what does the distribution for their second look like. Maybe I should implement something that would at least tell me what the sample sizes look like these days.

Philosophically, I think there are legitimate arguments about whether or not you want to do this too. On the one hand, treating all teams equally is maximally unbiased. On the other hand, maximizing accuracy would dictate giving some weight to an estimate of skill.

Ideally, you’d have something like this website:

If you don’t mind looking at five year old major junior hockey results, you’ll find that it’ll do predictions for you and there’s just a button that you can push where it will let you choose whether or not you want the remaining games to all be treated as toss ups or with probabilities based on the results from earlier from that season.

If you want to try it out, look towards the upper left where it says “Method:” and below it you can click on either “Weighted” or “50/50”.

But wait! There’s more! Even if there were forecasts customized per team, teams with fewer events played would still get pushed towards the middle because the number of points they might earn is so much bigger than the variance in what the cutoff will be. In other words, once a team has played all its events, this naturally pushes them towards 0% or 100% because most teams don’t end up within the range of where the cutoff might be.

This might be a bit more than you bargained for, but I hope this explanation is useful.

2 Likes
15

Hi, thanks for sharing this analysis. For teams like ours that still have their second event coming up, with almost no time between that event and DCMPS, the projected points needed to qualify for DCMPS informs prioritization. The ROI of a given capability varies in the context of Auburn vs. DCMPS.

1 Like
16

I’m glad you find it useful. Good luck at Auburn.

17

And here’s what the predictions look like after all of the district events have been played.

Note that this does not take into account declines, so if your team is listed as not going you might still want to see if you are close.

2023pnw.html.zip (6.0 KB)

Now, the more interesting part is probably the column about predicting the odds of going to worlds.

4 Likes
18

Thank you! This is helpful.

1 Like
19

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.