$25 Detroit Prediction Contest

The is the official thread for the $25 Detroit Match Prediction Contest. Here is his Caleb Sykes’s contest for the Houston half-champs, which this is mostly a copy of. Just like the previous contest, anyone can submit their predictions for whichever division(s) they choose to, using the preliminary schedule which was released by FIRST a few days ago. The best overall model will receive a $25 Amazon gift code. I am not able to offer $90 from AndyMark, unfortunately, but I received this card a few days ago and I’m fine parting with it. Rules etc are below:

Submission:

  1. Go to the link for the appropriate division below. Thank you to wjordan for the CSV schedules:
  • Archimedes edit-seems like the CSV for Archimedes has 3 missing matches. I will manually add those in, please wait until I finish. edit2-fixed. Please let me know if anything else looks wrong.
    Wes, if you’re reading this, it looks like your script missed the last match on the first three pages. - Carson
  • Curie
  • Daly
  • Darwin
  • Tesla
  1. Click on File>Make a Copy…
  2. Rename the copied spreadsheet to be “<Name/Username>'s <Division> Predictions”. For example, “Parthiv’s Curie Predictions”
  3. Edit the values in the last column as you see fit.
  4. In the top right corner, click SHARE>Get Shareable Link
  5. Paste the link into a response to this thread. If you are submitting for multiple divisions, please post the link to a Google Drive folder with all of your spreadsheets inside.
  6. In the response, please include a short description of how you arrived at your answers. If you used the same model as you did in Caleb’s contest, a link to your response there should suffice.

Scoring:

  • I will maintain a spreadsheet with all submissions here.
  • I will also use Brier scores to determine the best model. Your prediction will be subtracted from the actual result (red win = 0, blue win = 1, tie = 0.5), and then squared. So if you predict a 85% blue win probability, and blue does win, then your Brier score for that match would be 0.15^2 = 0.0225.
  • The model with the lowest average Brier score, throughout all matches from all eligible divisions (see the 5th bullet point in “Other rules”), will win the contest.
  • You are welcome to not enter all divisions, but your predictions for all divisions that you do not enter will be considered to be 0.5 (tie) for every match, for the sake of finding the best overall model.
  • Winners must send me a PM on Chief Delphi within 1 week of the end of Detroit half-champs to claim the Amazon gift code. Let’s go with 11:59PM on Saturday, May 5.

Other Rules (mostly plagiarized from Caleb, changes bolded):

  • Limit one submission per division per person
  • You are free to make submissions for multiple divisions.
  • I will make copies of your predictions when you publish them, so don’t share the link until your predictions are finalized. I’m not going to go back and re-copy anything if you make changes.
  • Don’t directly copy anyone else’s predictions. I’m sure everyone is fine with you using their predictions as a starting point, but you should really strive to get at least 100 percentage points different than everyone else. I may or may not enforce this.
  • If the final schedule for a division does not end up being equivalent to the preliminary schedule, that division will be ignored in the calculation of the best overall model.
    So if the schedule for Daly changes, the rankings will only be determined by the Brier scores for the other 5 divisions. - Submission deadline is **Wednesday 4/25 12:00PM (noon) Central Time. I realize that this is not a lot of time, so I will accept late submissions, but please let me know before 4/25 @noon Central Time so that I know to stay on the lookout for your submission. **
  • I reserve the right to remove any submission for any reason (e.g. if I find out you are throwing matches to improve your predictions).
  • I reserve the right to change or add any rules at any time.

Good luck, have fun, and I look forward to seeing what everyone comes up with.

@Eugene, I will redeem myself

I will be entering a “coin flip” model and a “tie every match” model as baselines.

Well @AirplaneWins, you’ll beat those at least. #shotsfired

Oh, and I’ll throw in a TBA t shirt for the best model that isn’t mine or Kunal’s (if he enters).

@Eugene maybe you’ll do better if you average the two…

This is already in my top 10 favorite CD threads and we’re only 7 posts in.

You guys really should calm down though, trash-talk isn’t GP imo. It’s also kind of petty to fight so hard over second place don’t you think?

Don’t worry about it. It’s all fun and games, we are friends.

:cool:

Are we all playing a game of chicken to be the last one to submit predictions so we don’t get averaged? :stuck_out_tongue:

@AirplaneWins are you still able to update the game results? I cannot do so as I mentioned in the other thread.

I figured that this was going on :stuck_out_tongue:

Thanks Eugene! At this rate it looks like my tie each match model is going to win me a TBA shirt…

I’m just waiting for Hal to finish training, but that is an interesting point. But then again I could care less. If you win by averaging our base models, then you are validating their value.

I don’t know what you all are doing, but I’m definitely not going to be the first to submit.

Could we just have the honor system and promise not to just take an average of other’s submission. Aside from manually checking the blend ratio for different submissions, there isn’t really much we could do to stop it. One thing we could do is have a master submission that belongs to no one and is simply an average of everyone’s submissions.( does not necessarily have to be an average, we could use XGboost and other , etc) This submission would not be for competition.

I can take the submissions privately and release them after the deadline, if you would like. I have no horse in this race.

Yes I can.

I don’t really care, I was making a joke.
Here are my predictions: https://drive.google.com/open?id=1DXO2GROZGokpxiOw8jNRofy2-1f_2e-z

Added yours to the main spreadsheet.

~12 hours to get your predictions in, or to arrange a late submission.

Here are my predictions: https://drive.google.com/open?id=1KddubYCFevNObTi9yazopouj4j8wbsAM

In the midst of migrating HAL to PyTorch from Keras, HAL is riddled with bugs right now so he was unusable for this competition. So I decided to stick with my standard Kaggle approach. For each alliance I took Caleb’s Elo for each team, Avi’s Elo, my Elo, Caleb’s OPR, My OPR, and ixOPR, and fed them into a bunch of models. I like model diversity, but did not have a lot of time for this competition so I stuck with Random Forests(RF), Extremely Random Forests(XRT), Gradient-Boosting Machines(GBM, a mixture of LightGBM and XGBoost), Deep Neural Nets(DNN and 1 autoencoder because I was bored) and some Generalized Linear Models(GLM).

For all the models except for the forest-based ones, I had some grid searching to try and find the best hyperparameters. Here are the parameters I looked to optimize:

GBM:

score_tree_interval
histogram_type
ntrees
max_depth
min_rows
learn_rate
sample_rate
col_sample_rate
col_sample_rate_per_tree
min_split_improvement

GLM:

alpha

Deep Learning:

epochs
adaptivate_rate
activation
rho
epsilon
input_dropout_ratio
num_hidden_layers
hidden_dropout_ratio

After finding the parameters I liked, I took the best model of each type and input their predictions into an XGBoost model and use that as my final predictions.