OCCRA
Go to Post Maybe Dean Kamen can appear in Levi's advertising? - David Brinza [more]
Home
Go Back   Chief Delphi > ChiefDelphi.com Website > Extra Discussion
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Reply
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 07-15-2017, 11:09 PM
Caleb Sykes's Avatar
Caleb Sykes Caleb Sykes is offline
Registered User
AKA: inkling16
no team
 
Join Date: Feb 2011
Rookie Year: 2009
Location: Minneapolis, Minnesota
Posts: 1,298
Caleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond repute
paper: Miscellaneous Statistics Projects

Thread created automatically to discuss a document in CD-Media.

Miscellaneous Statistics Projects by Caleb Sykes
Reply With Quote
  #2   Spotlight this post!  
Unread 07-15-2017, 11:10 PM
Caleb Sykes's Avatar
Caleb Sykes Caleb Sykes is offline
Registered User
AKA: inkling16
no team
 
Join Date: Feb 2011
Rookie Year: 2009
Location: Minneapolis, Minnesota
Posts: 1,298
Caleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond repute
Re: paper: Miscellaneous Statistics Projects

I frequently work on small projects that I don't believe merit entire threads on their own, so I have decided to upload them here and make a post about them in an existing thread. I also generally want my whitepapers to have instructions sheets so that anyone can pick them up and understand them. However, I don't want to bother with this for my smaller projects.
Reply With Quote
  #3   Spotlight this post!  
Unread 07-24-2017, 08:12 PM
Caleb Sykes's Avatar
Caleb Sykes Caleb Sykes is offline
Registered User
AKA: inkling16
no team
 
Join Date: Feb 2011
Rookie Year: 2009
Location: Minneapolis, Minnesota
Posts: 1,298
Caleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond repute
Re: paper: Miscellaneous Statistics Projects

In this post, Citrus Dad asked for a comparison of my Elo and OPR match predictions for the 2017 season. I have attached a file named "Elo and OPR comparison" that does this. Every qual match from 2017 is listed. Elo projections, OPR projections, and the average of the two, are also shown for each match. The square errors for all projections are shown, and these square errors are averaged together to get Brier scores for the three models.

Here are the Brier score summaries of the results.
Code:
Total Brier scores		
OPR	Elo	Average
0.212	0.217	0.209
		
Champs only Brier scores		
OPR	Elo	Average
0.208	0.210	0.204
The OPR and Elo models have similar Brier scores, with OPR taking a slight edge. This is directly in line with results from other years. However, predictions this year were much less predictive than any year since at least 2009. This is likely due to a combination of the non-linear and step-function-esque aspects of scoring for the 2017 game. My primary prediction method last season actually used a raw average of the Elo predictions and the OPR predictions, which provided more predictive power than either method alone.
Reply With Quote
  #4   Spotlight this post!  
Unread 07-25-2017, 05:02 PM
Citrus Dad's Avatar
Citrus Dad Citrus Dad is offline
Business and Scouting Mentor
AKA: Richard McCann
FRC #1678 (Citrus Circuits)
Team Role: Mentor
 
Join Date: May 2012
Rookie Year: 2012
Location: Davis
Posts: 1,148
Citrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond reputeCitrus Dad has a reputation beyond repute
Re: paper: Miscellaneous Statistics Projects

Quote:
Originally Posted by Caleb Sykes View Post
In this post, Citrus Dad asked for a comparison of my Elo and OPR match predictions for the 2017 season. I have attached a file named "Elo and OPR comparison" that does this. Every qual match from 2017 is listed. Elo projections, OPR projections, and the average of the two, are also shown for each match. The square errors for all projections are shown, and these square errors are averaged together to get Brier scores for the three models.

Here are the Brier score summaries of the results.
Code:
Total Brier scores		
OPR	Elo	Average
0.212	0.217	0.209
		
Champs only Brier scores		
OPR	Elo	Average
0.208	0.210	0.204
The OPR and Elo models have similar Brier scores, with OPR taking a slight edge. This is directly in line with results from other years. However, predictions this year were much less predictive than any year since at least 2009. This is likely due to a combination of the non-linear and step-function-esque aspects of scoring for the 2017 game. My primary prediction method last season actually used a raw average of the Elo predictions and the OPR predictions, which provided more predictive power than either method alone.
Thanks
__________________
Reply With Quote
  #5   Spotlight this post!  
Unread 07-28-2017, 07:01 PM
Caleb Sykes's Avatar
Caleb Sykes Caleb Sykes is offline
Registered User
AKA: inkling16
no team
 
Join Date: Feb 2011
Rookie Year: 2009
Location: Minneapolis, Minnesota
Posts: 1,298
Caleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond repute
Re: paper: Miscellaneous Statistics Projects

I am currently working on a model which can be used to predict who will win the Chairman's Award at a regional or district event. I am not covering district championship Chairman's or Championship Chairman's because of their small sample sizes. The primary inputs to this model are the awards data of each team at all of their previous events, although previous season Elo is also taken into account.

The model essentially works by assigning value to every regional/district award a team wins. I call these points milli-Chairman's Awards, or mCA points. I assigned the value of a Chairman's win in the current season at a base event of 50 teams to have a value of 1000 mCA. Thus, all award values can be interpreted as what percentage of a Chairman's award they are worth. Award values and model parameters were the values found to provide the best predictions of 2015-2016 Chairman's wins. At each event, a logistic distribution is used to map a team's total points to their likelihood of winning the Chairman's Award at that event. Rookies, HOF teams, and teams that won Chairman's earlier in the season are assigned a probability of 0%.

I have attached a file named 2017_Chairman's_predictions.xlsm which shows my model's predictions for all 2017 regional and district events, as well as a sheet which shows the key model parameters and a description of each. The model used for these predictions was created by running from the period 2008-2016, with tuning specifically for the period 2015-2016, so the model did not know any of the 2017 results before "predicting" them.

Key takeaways:
  • The mean reversion value of 19% is right in line with the 20% mean reversion value I found when building my Elo model. It intrigues me that two very different endeavors led to essentially equivalent values.
  • It was no surprise to me that EI was worth 80% of a Chairman's Award. I was a bit surprised though to find that Dean's List was worth 60% of a Chairman's Award, especially because two are given out at each event. That means that the crazy teams that manage to win 2 Dean's List Awards at a single event are better off than a team that won Chairman's in terms of future Chairman's performance.
  • I have gained more appreciation for certain awards after seeing how strongly they predict future Chairman's Awards. In particular, the Team Spirit and Imagery awards.


More work to come on this topic in the next few hours/days.
Reply With Quote
  #6   Spotlight this post!  
Unread 07-29-2017, 07:35 PM
Caleb Sykes's Avatar
Caleb Sykes Caleb Sykes is offline
Registered User
AKA: inkling16
no team
 
Join Date: Feb 2011
Rookie Year: 2009
Location: Minneapolis, Minnesota
Posts: 1,298
Caleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond repute
Re: paper: Miscellaneous Statistics Projects

I have added another workbook named "2018 Chairman's Predictions." This workbook can be used to predict Chairman's results for any set of teams you enter. The model used here has the same base system as the "2017 Chairman's Predictions" model, but some of the parameter values have changed. These parameters were found by minimizing the prediction error for the period 2016-2017.

Also in this book is a complete listing of teams and their current mCA values. The top 100 teams are listed below.
Code:
team	mCA
1718	9496
503	9334
1540	9334
2834	9191
1676	8961
1241	8941
68	8814
548	8531
2468	8112
2974	8092
27	8047
1885	7881
1511	7786
1023	7641
1305	7635
2614	7568
245	7530
1629	7381
2486	7100
66	7027
3132	6748
1816	6742
1086	6551
1311	6482
1710	6263
2648	6241
125	6223
558	6155
141	6083
1519	6082
1983	6060
4039	5985
33	5851
2771	5780
1902	5582
624	5578
1011	5496
118	5470
2137	5461
1218	5424
2169	5390
910	5382
3284	5353
3478	5344
771	5321
75	5306
2557	5291
233	5287
987	5224
1868	5215
3309	5175
1714	5158
932	5147
1986	5144
537	5138
597	5077
604	5068
2056	5059
2996	5054
4613	5042
399	5029
1477	5010
2220	4994
2337	4955
3618	4896
4125	4823
217	4816
1730	4803
359	4784
2655	4714
2500	4706
694	4695
1923	4667
708	4662
1622	4661
1987	4655
2642	4655
1671	4630
4013	4627
772	4626
2415	4622
4063	4604
540	4501
433	4440
4525	4426
384	4412
3476	4384
2485	4333
3008	4325
303	4307
1711	4288
2590	4266
3142	4264
3256	4260
836	4251
3880	4250
1678	4244
2471	4237
230	4230
78	4224
If I make an event simulator again next year, I will likely include Chairman's predictions there.
Reply With Quote
  #7   Spotlight this post!  
Unread 08-04-2017, 01:24 PM
Caleb Sykes's Avatar
Caleb Sykes Caleb Sykes is offline
Registered User
AKA: inkling16
no team
 
Join Date: Feb 2011
Rookie Year: 2009
Location: Minneapolis, Minnesota
Posts: 1,298
Caleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond reputeCaleb Sykes has a reputation beyond repute
Re: paper: Miscellaneous Statistics Projects

I got a question about historical mCA values for a team, so I decided to post the start of season mCA values for all teams since 2009. This can be found in the attached "Historical_mCA" document.
Reply With Quote
Reply


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 06:31 PM.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi