2016 match data

[Ether]

Week4 raw data CSV files, District Rankings CSV files, and High Scores were uploaded late yesterday evening.

[Joe Ross]

In the high scores for towerEndStrength, the smallest number might be more interesting then the largest, which is just who got the most technical fouls without weakening the tower. It would also be interesting to see how many times a high score was acheived. For example, breach of 20 and capture of 25 has probably been obtained hundreds, while teleopBoulder points of 76 has probably only happened once.

[Ether]

Originally Posted by Joe Ross

In the high scores for towerEndStrength, the smallest number might be more interesting then the largest...It would also be interesting to see how many times a high score was achieved

Thanks Joe. Done. (revA posted)

http://www.chiefdelphi.com/media/papers/3243

[Ether]

Originally Posted by Ether

Week4 raw data CSV files, District Rankings CSV files, and High Scores were uploaded late yesterday evening.

I just added attachments for cumulative Weeks 1 through 4 Qual and Playoff High Scores.

http://www.chiefdelphi.com/media/papers/3243

[ngreen]

Thanks for providing these. I had quickly developed some capabilities for scouting using the Ranking data between our first and second (last) regional.

I started directly connecting Ranking data to Tableau using a web data connector. The particular connector uses import.io, which would allow even more complex extractions. I had hoped to use this to make everything work in a standalone Tableau workbook, but time and my inexperience made me switch later to more familiar tools.

For scouting, Tableau was pretty powerful and simple to use. Direct from the Ranking data, I plotted a 2x2 graph with Defense and Goal on y-axis and Scale and Auto on x-axis. In a second worksheet, I plotted individual bar charts with team numbers on the x-axis. I liked this worksheet better because hovering over the y-axis allowed for me to sort, showing the top teams easily. I also plotted some calculated fields here, like the sum of Defense and Goal points. Filtering was really simple too, and useful to exclude teams with bad auto scores, etc.

There is some scripting in Tableau, with integration with R. Using it I ran a clustering algorithm, only using four parameters. In the 2x2 plots, I set colors based on cluster number. Here is an example: http://imgur.com/a/QDWpy . I also wrote more on this: https://goo.gl/DTnwjH

From there I wanted to instead calculate team-based scores from the alliance-based scores (basically OPR). I expect this could be done in Tableau with scripting, but I ended up doing this in Excel, building the matrix (by copy/pasting the 3 team alliances for each permutation (1&1,2&2,3&3,1&2,2&1,1&3,3&1,2&3,3&2), and then using a series of countifs to fill the matrix). And then solving Ax=b in Excel for each. I then exported a CSV and opened this in Tableau. This data was useful. Selection seems to follow it rather closely, with a little greater emphasis to goal scoring teams. So in that way it was good, although it doesn't give much tactical information and it does undervalue low goal scorers and their contribution to tower captures.

I only started looking at match data a few days before our regional. Not having the API/JSON experience, I did have a student begin to compile match data manually. However, by lunch Friday I knew we didn't have resources to even make that work. For anyone else in that predicament, here is a Google Sheet I have hastily put together today (https://docs.google.com/spreadsheets...it?usp=sharing). Altering in for other competition, mostly involves changing "code2016" to the correct name for the competition on TBA. I'd planned to use pivot tables to sum the data per team, and then exporting to Tableau. Before that I would fix team numbers (I forgot to account for numbers <1000), and clean the data (e.g. making each defense a header and the value be pts from crossing, and using only numeric or boolean data), likely with a macro.

During the offseason, I want to teach some of this to the team members, all the way from the API to the visualization. It is nice to have these files already compiled so that we'll be able to jump directly into using the data and exploring different ideas. There is a lot of potential to use this data for scouting and also for the team members to learn how to work with programming/analysis tools.

[Ether]

Originally Posted by ngreen

Thanks for providing these.

I'm glad you've found them to be useful.

@all_readers: if there is any additional available raw data that you would like to analyze and which isn't included in the CSVs I am posting, let me know and I'll see if I can add it.

I wanted to instead calculate team-based scores from the alliance-based scores (basically OPR). I expect this could be done in Tableau with scripting, but I ended up doing this in Excel, building the matrix (by copy/pasting the 3 team alliances for each permutation (1&1,2&2,3&3,1&2,2&1,1&3,3&1,2&3,3&2), and then using a series of countifs to fill the matrix). And then solving Ax=b in Excel for each. I then exported a CSV

There's a much simpler way to get OPRs. Caleb Sykes posts a large XLSX spreadsheet here. And Team 2834 maintains an XLXM scouting database here.

Or with just a little bit of effort using AWK (or Python) and Octave (or Matlab) you can easily create a CSV of OPR values for any raw data that has match-by-match scores:

Here's a complete AWK script that reads an 8-column whitespace-separated plaintext file that contains the fields red1 red2 red 3 blue1 blue2 blue3 redscore bluescore, and outputs the team list column vector T, the alliance scores column vector b, and the sparse binary 2MxN design matrix A (M is number of matches, N is number of teams).

Here's an Octave script that reads the AWK output and computes OPR.

I only started looking at match data a few days before our regional. Not having the API/JSON experience, I did have a student begin to compile match data manually. However, by lunch Friday I knew we didn't have resources to even make that work.

What match data are you referring to, and what were you trying to compile manually? I post all the raw match data as CSV files.

[Ether]

I will post Week5 raw data sometime late evening Apr3 or early morning Apr4. It will not include Western Canada Regional, which does not finish until Apr6.

I will post updated Week5 data, including Western Canada, sometime late evening Apr6 or early morning Apr7.

[Ether]

Originally Posted by Ether

I will post Week5 raw data sometime late evening Apr3 or early morning Apr4. It will not include Western Canada Regional, which does not finish until Apr6.

Posted.

http://www.chiefdelphi.com/media/papers/3243#views

[Ether]

Week5 District Rankings CSV Files have been posted

[CrazyMohawk]

Originally Posted by Ether

Week5 District Rankings CSV Files have been posted

not all of the points teams earned at Auburn are accounted for in the rankings

[Ether]

Originally Posted by CrazyMohawk

not all of the points teams earned at Auburn are accounted for in the rankings

Perhaps there was an internet issue at that venue. It happens. I'll download the rankings again tomorrow and compare them.

[Ether]

Originally Posted by Ether

Perhaps there was an internet issue at that venue. It happens. I'll download the rankings again tomorrow and compare them.

TBA just updated PNW District Rankings for Auburn event. Updated CSV files have been posted.

[Ether]

Originally Posted by Ether

I will post Week5 raw data sometime late evening Apr3 or early morning Apr4. It will not include Western Canada Regional, which does not finish until Apr6.

Originally Posted by Ether

Posted.

http://www.chiefdelphi.com/media/papers/3243#views

PNW Auburn (WAAHS) Awards was not in the dataset at the time it was retrieved. I have added an attachment for it.

[Ether]

Western Canada Regional (ABCA) Raw Data CSV files have been posted.

[Ether]

I will be posting Week6 raw data CSV files as follows:

the morning of April 10^th for those events ending on the 9^th

the morning of April 11^th for those events ending on the 10^th

http://www.chiefdelphi.com/media/papers/3243#views