Thread created automatically to discuss a document in CD-Media.
Caleb’s Event Simulator 2018 by: Caleb Sykes
This is a workbook which generates statistics and makes predictions during an event.
This workbook is a tool which can be used to simulate current 2018 events, or to calculate statistics for completed events. Data is pulled from the TBA API in the “data import” sheet. Explanations for how to use each sheet can be found in the “instructions and FAQ” sheet.
This is an extremely rough version of my 2018 event simulator. It’s not even worth calling it a Beta. I wouldn’t even be publishing it if I hadn’t promised something by today. I severely under-estimated how much effort it was going to be to convert to 2018 metrics, switch to TBA API v3, add in new things I want, and edit the formatting to be more pleasing to me.
Data import should work fine, everything else has had one rough pass done by me so is likely full of flaws. I didn’t even start on Chairman’s, match, or ranking predictions. Images can’t pull from TBA, and I haven’t even looked over the Instructions/FAQ sheet.
So basically, use at your own risk and don’t place much faith in anything except data import. I’ll be working on this more tomorrow to hopefully get it to where I want it.
I have uploaded a new workbook which I’m classing as a beta release. I expect to find multiple minor to mid level flaws, but hopefully nothing that breaks the entire workbook. Feel free to let me know if you find any bugs. I have hidden the prediction sheets, so all sheets shown should be usable and working. I don’t normally delete anything I publish, but the alpha release was so bad that I can’t stand to leave it out in the open.
Key changes since the alpha include:
Added capability to download images from TBA
Fixed rankings data for lowest ranked team
Switched to 2018 categories on “team lookup” and “match lookup” sheets
Various minor formatting changes
Updated “team lookup” and “match lookup” sheets to have correct 2018 categories
Updated “instructions and FAQ” sheet to include 2018 specific info
Fixed a bug that made calculated contributions incorrect
I’ll post another update tomorrow night based on the feedback I get tomorrow. I wish I could do more testing before releasing but it is what it is. Good luck to everyone competing week 1, and I hope you find this helpful.
thanks for your hard work. we used this last year to validate scouting data. helped get us to Einstein. Sikes-to-Sykes, we have a lot of new tools under development this year for visual match simulation, but not even close to ready to use or share. Bleeding edge hurts…
Another day, another update. This is the first version that I am actually comfortable releasing. I’ve fixed numerous bugs and tested everything out on multiple events’ data at various points throughout the event. It certainly has a ways to go before it gets to the level of my 2017 simulator near the end of the season, but I have laid the groundwork as well as I can at this point.
Key changes since the Beta include:
Fixed the “last completed qualification match” field in “data import”
Various minor formatting changes
Extended filters in “calculated contributions” and “predicted contributions” sheets
Fixed calculated contributions to appear in the correct columns
Fixed predicted contributions which were completely broken for most categories
Eliminated unnecessary zeros in contributions
Added win probabilities to “team lookup” sheet
Save any major flaws being discovered (which is still a real possibility), this will be my last update for week 1. Expect the week 2 update by Wednesday.
Cool, I’m glad you got some use out of it! I hope it is just as useful for you this season.
A week 2 update has been posted. Quite a few additions, but still not as many as I would have liked. In particular, I would have liked to add Chairman’s predictions, but didn’t get around to it. If you want Chairman’s Predictions, you can still get them in my “2018_Chairman’s_predictions v2” workbook here, but you might have to manually remove some teams that won Chairman’s in week 1.
Changes since v2018.0.2 include:
Various minor formatting changes
When pressing “Update” on “data import”, code execution ends immediately if there is no new data available from TBA
Minor bug fixes
Added auto Run rate
Added efficiencies for vault sections
Added endgame rates
Updated seed values
I’m targeting to have the workbook at the level it was at the end of 2017 by next week’s update (all categories filled in, detailed match predictions, and rankings predictions), although I’m always too optimistic about how easy the changes will be.
It seems that the TBA API is down, I’ve tried updating for Milford event on 2 different internet connections and I’m getting “Error: unable to access TBA api, make sure you have internet access”. It doesn’t seem to be related to the recent “When pressing “Update” on “data import”, code execution ends immediately if there is no new data available from TBA” change, as I am getting the error across all 3 versions.
This is really strange. The API works fine for completed events. It also returns correct team lists for upcoming events. However match data and ranking data for upcoming events are returning responses my simulator doesn’t recognize. I’m inclined to blame myself before I blame TBA, but I also get the same error on previous versions. I don’t recall this happening at all last week, did anyone notice this error on my previous simulators before today?
I’ll try to replicate this tonight and try to come up with a workaround. If anyone else has seen this and knows a solution let me know.
My pleasure, I’m glad people find it useful.
I’m going to wait this out until tomorrow. My guess is that it will no longer be an issue once the event starts. I can investigate more this weekend to hopefully determine what is going on. I’ve heard from Eugene Fang that no recent TBA changes should have affected the API endpoints I use, so I’m probably just not handling null responses properly. Before the event starts, all the simulator really does is find all teams’ seed values, which can also be done using my scouting database. So I’m going to refer people there until their event starts.
I have uploaded a week 3 book. It’s mostly just bug fixes, but I did finally add Chairman’s Predictions. I never tested my Chairman’s Prediction model for district championships or world championships, so predictions do not appear for these events. Maybe sometime during the offseason I’ll look into it more, but the small sample size really concerns me. A known but low priority issue with the Chairman’s Predictions is that they show district teams competing at regionals as eligible to win Chairman’s. To me, this as a minor issue that would take a large effort to correct, so I’m just leaving it for now.
Changes since v2018.2.1 include:
Actually doesn’t run if there is no new data
Correctly handles events which return null ranking data before event starts (mimil, njfla)
Various minor bug fixes
Added Chairman’s Predictions
Updated seed values
I was recently asked to eliminate or adjust two rounds for team 3556 at Orlando. It seems they had a software problems that shut them down. The officials would not replay the two matches. I am guessing that their OPR is about 25% low. Surely other folks that are scouting regionals might benefit by being able to ignore outliers. Any suggestions?
A week 4 book has been uploaded. The only major change is to update seed values.
If you send me the matches to ignore I can manually make this adjustment pretty easily. I’ve had the idea to incorporate something like this in the back of my mind for a while, but it was probably around 6th on my priority list of things to add. I’ll move it up a few slots since you requested it, but that’s still no guarantee that it’ll be out this season.
Doesn’t this open up a whole can of worms with people requesting data to be altered? How does this scale beyond a few reports of “robot malfunction?” If you okay with incorporating this “scouting data,” why not just ask for scouting data from teams from all events?
I wasn’t thinking of changing any of my published data, that would certainly be a dangerous road to go down. I just meant that I could send him a modified list of contributions for a single event to use as he would like. I personally think it’s kind of silly to remove matches from the data set for a specific team, since all teams’ ratings would be improved if you ignored their worst matches, but to each his own.
