*For the benefit of those who would like to analyze the scoring data but don’t know how to access the API and/or parse the XML/JSON responses, I’ve posted Excel CSV files here, so you don’t have to “scrape” web pages to get the data you need.
You’re a gift, thank you.
This is amazing! But I have a very newbie question for you. So looking at say MIKE’s qualification match data, how would you use data to build a profile of which matches a team was involved with and the scores from that particular match? Would it be a pretty complicated set of vlookups to build that profile?
Thanks so much for compiling this! And we’ve been trying to look for ways to learn how to interact with the API but the tutorials are pretty thin. If you have any suggestions, that would be awesome.
In the qual match CSV, Columns E thru J contain the Team Numbers r1,r2,r3,b1,b2,b3.
Say you want to find all of Team 469’s matches in Week2 MIKE2 event.
Insert a new column right after Column J. Now you will have a new blank column K.
Put the heading “Team469” in cell K1.
Put the following formula in cell K2:
Copy cell K2 all the way down.
Now each cell in column K contains the value “1” if 469 played in that match, or a “0” if they did not.
Select all the cells in the spreadsheet, and sort by column K descending. All the matches that 469 played in will be displayed at the top of the spreadsheet.
Is that what you were asking?
*There was an error in the original “Week1 Matches raw data”. That file attachment has been removed, and a corrected version uploaded. Sorry for any confusion or inconvenience.
Thank you! Yes, my brother mentioned doing that as well. Thank you for confirming that this is a good way to go about this. I’m trying to use Tableau to compile data and visualize it all.
Thank you for your help!
I’m not so sure it’s a “good” way, but it’s an quick-n-easy way.
In Excel, a better way would be to write a simple macro to pick each match that 469 played in, and copy it to a new sheet.
I’m trying to use Tableau to compile data and visualize it all.
I’ve never used Tableau. Does it have a macro language? If so, you might want to investigate that.
Thank you for your help!
You’re welcome. Feel free to ask questions any time.
***I should have mentioned in the OP:
Hat tip to Phil Lopreiato and Alex Herreid for their patient help.**
I’m trying to use Tableau to compile data and visualize it all
There’s no equivalent of macro calculations to VBA in Excel, though it does have some kind of data blending between different CSV sheets, as well as some flexibility in their table calculations.
But I’ve found the best way is to blend the data in Excel and then import into Tableau, which does require calculations and possibly macros.
Thank you very much.
Nice results! I’ve always appreciated your love of data and am always a fan of more numbers
Not a problem - I’m always happy to help! Glad you were able to find a solution that worked for you!
***Just to let you all know:
I will be posting the complete Week3 raw data* in CSV format within 18 hours after the last playoff match has been completed.**
*qual match schedule with surrogates; qual match final, foul, and auto scores; qual match scoring details; qual match team rankings with wins, losses, ties, and DQ; alliance selection with substitutions; playoff match schedule; playoff match scoring details; and awards
*I wrote a script to scan through all the Scoring Details data (component scores) and pick out the high value in each category for week1 and week2. See attached.
**Week3 Raw Data CSV files have been compiled and posted here:
EDIT: attached is a compilation of hi component scores for week3.
Just posted Week3 District Rankings CSV files here.**
*Week4 raw data CSV files, District Rankings CSV files, and High Scores were uploaded late yesterday evening.
In the high scores for towerEndStrength, the smallest number might be more interesting then the largest, which is just who got the most technical fouls without weakening the tower. It would also be interesting to see how many times a high score was acheived. For example, breach of 20 and capture of 25 has probably been obtained hundreds, while teleopBoulder points of 76 has probably only happened once.
Thanks Joe. Done. (revA posted)
I just added attachments for cumulative Weeks 1 through 4 Qual and Playoff High Scores.
Thanks for providing these. I had quickly developed some capabilities for scouting using the Ranking data between our first and second (last) regional.
I started directly connecting Ranking data to Tableau using a web data connector. The particular connector uses import.io, which would allow even more complex extractions. I had hoped to use this to make everything work in a standalone Tableau workbook, but time and my inexperience made me switch later to more familiar tools.
For scouting, Tableau was pretty powerful and simple to use. Direct from the Ranking data, I plotted a 2x2 graph with Defense and Goal on y-axis and Scale and Auto on x-axis. In a second worksheet, I plotted individual bar charts with team numbers on the x-axis. I liked this worksheet better because hovering over the y-axis allowed for me to sort, showing the top teams easily. I also plotted some calculated fields here, like the sum of Defense and Goal points. Filtering was really simple too, and useful to exclude teams with bad auto scores, etc.
There is some scripting in Tableau, with integration with R. Using it I ran a clustering algorithm, only using four parameters. In the 2x2 plots, I set colors based on cluster number. Here is an example: http://imgur.com/a/QDWpy . I also wrote more on this: https://goo.gl/DTnwjH
From there I wanted to instead calculate team-based scores from the alliance-based scores (basically OPR). I expect this could be done in Tableau with scripting, but I ended up doing this in Excel, building the matrix (by copy/pasting the 3 team alliances for each permutation (1&1,2&2,3&3,1&2,2&1,1&3,3&1,2&3,3&2), and then using a series of countifs to fill the matrix). And then solving Ax=b in Excel for each. I then exported a CSV and opened this in Tableau. This data was useful. Selection seems to follow it rather closely, with a little greater emphasis to goal scoring teams. So in that way it was good, although it doesn’t give much tactical information and it does undervalue low goal scorers and their contribution to tower captures.
I only started looking at match data a few days before our regional. Not having the API/JSON experience, I did have a student begin to compile match data manually. However, by lunch Friday I knew we didn’t have resources to even make that work. For anyone else in that predicament, here is a Google Sheet I have hastily put together today (https://docs.google.com/spreadsheets/d/1wg6XTXfaHswEGhfSj9wWlNl1Dt7xKTWQhL3I0LLVUVU/edit?usp=sharing). Altering in for other competition, mostly involves changing “code2016” to the correct name for the competition on TBA. I’d planned to use pivot tables to sum the data per team, and then exporting to Tableau. Before that I would fix team numbers (I forgot to account for numbers <1000), and clean the data (e.g. making each defense a header and the value be pts from crossing, and using only numeric or boolean data), likely with a macro.
During the offseason, I want to teach some of this to the team members, all the way from the API to the visualization. It is nice to have these files already compiled so that we’ll be able to jump directly into using the data and exploring different ideas. There is a lot of potential to use this data for scouting and also for the team members to learn how to work with programming/analysis tools.