Tableau at PNW Districts
 

Attachment 20358

Our Team (CPR - #3663) used Tableau to analyze our match scouting data at the Week 2 PNW Glacier Peak Competition. I am hoping that I successfully attached a copy of our Tableau dashboard in this post. Tableau has been a huge factor in our scouting the last two seasons. We were introduced to it by Quinn Schiller on Team 1983, Skunkworks Robotics. It helped us become a captain in our division at World's in 2015 (losing to 1114 and 148 in the Finals) and contributed heavily to our winning Glacier Peak last weekend.

Tableau is a very powerful tool. It takes our dry Excel data and transforms it into beautiful pictures and graphs that help organize and clarify our information. There is a bit of learning curve with it, but their are multiple tutorials on the Tableau site and through FIRST that can help answer any questions.

At Glacier Peak, we used our Tableau dashboard, hopefully attached here, to convince the top seed, Team 2522, to pick us in alliance selections. Prior to our lobbying, they weren't even considering us. We showed them that we were a perfect complement to their team (we are heavy shooters and breachers, they are heavy breachers and they shoot some low goals). Having been alerted to our existence by our compelling data, they then watched us in our next match. We made five high goals and breached heavily, including the Category A defenses (Portcullis and Cheval de Friese). They then watched the number two seed, who had similar data, perform. We drove more aggressively and quickly and adjusted on the fly to our partner's difficulties. They were convinced we were the best choice and picked us. We went on to win the competition.

This brings me to the subject of qualitative scouting. We have eight people dedicated to quantitative scouting (6 match scouters, 1 Excel data entry person, and 1 Tableau analyst). We have two scouters dedicated to qualitative scouting. We use the quantitative data as the basis for all match strategy preparation and then incorporate the qualitative data to make final decisions. The qualitative data helps us evaluate driver skill (critically important in this year's competition), robot speed and consistency, the flexibility of a team to change strategy mid-match if necessary, and how much partners work well together. Quantitative data can suggest this information (consistency in scoring across matches, amount of defenses crossed and points made to suggest speed) but simple words such as "unreliable" and "fast" convey a wealth of information quickly. The qualitative and quantitative data together help provide a complete picture of a team and provide checks and balances for one another.

I am hoping that more teams will use Tableau. I love it when I see teams making informed, wise decisions in match strategy and alliance selections. I think it elevates the level of play for everyone. While not everyone has the staffing available to make up a full scouting team, it's not hard to combine with other teams to pool data. Our team is always willing to work with other teams to share our process and we post our bar charts at competition.

I'd be curious to learn if other teams use Tableau or something similar.

Good luck to everyone at Stronghold!

Re: Tableau at PNW Districts
 

This is the third year 2834 is using Tableau. We like to collect data and we like them shown in a clear and concise way to make good decisions. I can't give all the credit of how we did in the last 3 years all to Tableau but it certainly is one of the enablers.

Re: Tableau at PNW Districts
 

A word of caution: Tableau is not a cure-all. This should go without saying, but you have to know both how to use it well and understand what you ultimately want to get out of it.

One year we had a new mentor who decided he was going to do analysis of our scouting data using Tableau and he made a bunch of interesting graphs. They were interesting ways to look at the data - he was not a dumb guy. Unfortunately, they didn't quite line up with what we wanted to know, and we weren't able to get the data we did want out in time so we ended up totally disregarding all of our scouts' work at an event. This was a couple of years ago. It was unfortunate.

I know that teams have had good luck with it, but be sure that you know what you're doing before you jump in without a backup.

Re: Tableau at PNW Districts
 

I agree. We use qualitative data as a balance check for the Tableau data. Also, as in any data collection scheme, you need to know which variables to scout.

Re: Tableau at PNW Districts
 

I agree 100%. During the build season, the scouting team meet and propose what scouting information to collect. It is then reviewed by the whole team. After it is finalized, programmers update the Android scouting app and the Tableau person prepare the graphs and work with the game strategy team to finalize the graphs.

The other nice thing about Tableau is once you have the data collected, you can look at it other ways on the fly relatively quickly if the graphs you prepared ahead of time is not as useful as you expected.

It works for us because it is a team effort. I hope more teams will try the software. It is great.

Re: Tableau at PNW Districts
 

We've also begun to use Tableau this year with data from our scouting app. It's been hugely helpful not just for alliance selection, but also pre-match strategy, and convincing other of our strategical ideas.
Really helpful tool, and available to all FRC teams, it's really a shame people don't take more advantage of it.
It's really great to hear that other PNW teams are using it, I've been curious about that for a while.

Re: Tableau at PNW Districts
 

Quick question for you guys: One of the possible features for SuperScouter for next year is the ability to export a Tableau Data Extract file with the table generated from the schema. Would any of you find that interesting?

Re: Tableau at PNW Districts
 

I'm really excited that Shockwave is using Tableau! We're pretty sure we've qualified for PNW District Champs so we'll see you there. I'd love for our Tableau student analyst to meet with yours and talk about Tableau.

Re: Tableau at PNW Districts
 

Our team met during build season, too. Our software team was able to make a sample dummy data set of 24 teams with different Stronghold abilities. I had my scouts use Tableau data to strategize for different red and blue alliances like we do in a real competition. They were able to get used to how the data looked in Tableau and we were able to make changes to the categories of information we looked at. We also did sample alliance selections based on the data. Given that we were all pretty familiar with how this year's competition data looked before we even had a real competition, we were able to hit the ground running. We have had to make very few changes to our data collection after two competitions. Again, I will say that qualitative data is a necessary balance to the quantitative data.

Re: Tableau at PNW Districts
 

It's thanks to this former student that Tableau now provides a copy for every team in the Virtual KOP. It's great to see the legacy he left in PNW and beyond.

Re: Tableau at PNW Districts
 

Absolutely, I'd love to meet your analyst, we have a few people using it, but primarily me, so I'd be excited to meet others. See you guys at Dist. Champs!

Re: Tableau at PNW Districts
 

Yes please! What we're planning on doing right now is SuperScouter -> Excel -> Tableau.

Re: Tableau at PNW Districts
 

Awesome!

Re: Tableau at PNW Districts
 

Is there a product key in the KOP? I might ask my leas mentor about downloading it on our Monday meeting right before the IA regional.

I've heard it's super awesome, and with our awesome scouting system, a printer in the pit, and this, that could round off our data. Not to mention look pretty.

Re: Tableau at PNW Districts
 

We've considered using Tableau as well, it's actually on my laptop, I just haven't dedicated the time to learning it proficiently enough for competition use.

Not to mention we do not have enough students to have one specifically set on data input, I did it for two years and found that we could analyze the handwritten sheets much more effectively. It is mostly a tally system with a section for driver skill, so while we may not have the graphs, it is still quantitative.

My first question would be how long do your scouting night meetings generally take with the help of Tableau? Oftentimes when teams are talking about how they scout most efficiently I can't tell if efficient to them is 1, 3, or even 6 hours of hashing out the data.

Our current system in the last two years has been great for us, I'd be interested in seeing what your match scouting sheets look like as well. Would you mind sharing?

Congratulations on the win in Glacier Peak, those finals matches were fun to watch!

Re: Tableau at PNW Districts
 

All students can get a free one year license at the Tableau site by offering proof of being a high school student. Many students upload a picture of a recent report card. There are also 5 free keys for Tableau licenses on TIMS for non-students. It's in their list of freebies. You would need your TIMS administrator to login and give you the key. We use about 3 of the keys for mentors.

You will need some time to get used to Tableau - do the tutorials, perhaps practice with uploading some data - to make Tableau useful. Unfortunately, it does have a learning curve and I wouldn't recommend deciding to use it the week before a competition with no practice or experience.

Re: Tableau at PNW Districts
 

I fully agree with this. I have been using Tableau for a while now and I absolutely LOVE it! We are using it for scouting, data is input into an excel sheet via 6 tablets. Tableau is one amazing tool and the fact that we get 5 FREE seats is so awesome.

If you are an absolute wiz in Excel, ie can write VBA, functions that are miles long, etc with your eyes closed, you will be able to pick up Tableau in a good couple of hours. But I'd definitely not try using it for the first time a week before your competition, but I would encourage you to take your first competition scouting data and playing with it in Tableau to learn its capabilities.

Peace
CBJ

Re: Tableau at PNW Districts
 

Attachment 20380Hi Rachelle,

First of all, I'd like to say that we loved your team at Auburn. We were really bummed when we couldn't pick you, but even if you hadn't become a captain, you probably would have been snapped up well before it came to be our turn again. You were so fast, reliable, and a great defender. Good luck at your next competition!

I uploaded our Scouting Form and our Excel data from Glacier Peak. On the scouting form, the shooting location is on the left, the breaching,shooting, and end game information is in the middle, and the data entry column is on the right. Scouts fill in the middle and left sections, then rewrite their answers on the right, hopefully legibly, for ease of data entry.

The Excel data form includes all of our quantitative data from Glacier Peak. It does not include the qualitative data, which is handwritten. As always, I caution against using quantitative data only. It includes the majority of our information but doesn't completely account for driver skill, smart play, and other important variables. The qualitative data on your team bumped you up considerably on our lists - much more so than if we had just looked at quantitative data. Our analyst, Alli, is planning to fill all of the Excel boxes up with zero next time and our scouts will change the values when there is something to enter. You can upload our data into your own Tableau file to play around with it.

I would be happy to send you our Tableau packaged workbook from Glacier Peak so you can see what it looks like. I wasn't able to upload it here because it is the wrong file type. I could email it to you or your team if you'd like. It's a read-only, but you can make your own based on it.

You mentioned you do a handwritten tally/driver skill sort of data collection. This is what I use for qualitative information. I don't worry too much about getting all of the data - I do a rough approximation since we have quantitative data covered - but I write enough to remind myself of the robot capabilities. I focus on adding many one word comments such as "struggled, died, fast, smart, wandered, hit partner," etc. I think a handwritten approach like you discussed is a nice option for teams with limited scouting resources. You may miss some things, but you get the gist of it.

In terms of our scouting meetings/preliminary alliance selections after the first day of scouting: it takes us about two to three hours. While we could probably do it faster, my goal is to get my analyst team (4 veterans and 2 first-timers) to understand and think about the robots. Alliance selections is a much tougher task in my mind than match strategy. We make our lists first - top 24 of auto, shooting, breaching (static and manipulative defenses), challenge, scaling, defensive - and then go through each list, raising and lowering teams based on qualitative data. This takes the most time. Then we talk about who we would pick, who we would decline, who we want to lobby, and who is on our do not pick list. We also do mock drafts at that time, looking at likely seeding. We end the night by determining lobbying strategies and noting which robots we want to learn more about the next day. The next morning, we send our lobbying team out, we watch more robots and finalize their position, and make our final lists.

This is probably more information than you wanted but I am hoping others might find it helpful, too. Each team needs to do what works best for them and we are always looking to improve.

Good luck to you!

[ATTACH]Attachment 20380[/ATTACH]

Re: Tableau at PNW Districts
 

You will need some time to get used to Tableau - do the tutorials, perhaps practice with uploading some data - to make Tableau useful. Unfortunately, it does have a learning curve and I wouldn't recommend deciding to use it the week before a competition with no practice or experience.[/quote]



Yeah, to put this into context, I spent the majority of build season on learning Tableau and I still don't know how to do lots of things.

Re: Tableau at PNW Districts
 

Thank you, it is awesome to be told that we are being noticed. Random fact about our team, we've always been in the position where we become a captain or don't go to elims at all. So not our first time as that 8th seed captain, it would have ended that streak :D We've been 8th captain from 9th seed, 11th, and now 15th seed.

This isn't too much info at all! I love to see how other teams do their scouting in order to compare our system and see what we can improve and where we are doing well.

Lobbying is probably the one thing we have never really done, obviously this worked well for you to 2522, is this something we should put a little more focus to in your opinion?

Our scouting meeting system is quite similar, we try to start with our "do not picks" first in order to narrow it down. That does mean that once in while a team will get bumped up later on. In the morning we mainly look for major improvements, to see if anyone is breaking, and if there are any broken ones that we could potentially help get fixed in time for an elim match.

One thing I do like about the paper scouting is it helps keeps us moving at 1am during the meeting shuffling papers versus staring at a screen. I'll PM you an email, I would definitely like to see your Tableau setup.

I was also curious if you (or other teams if you're reading) do any pre-competition scouting? We look for performances in the last couple of years, alliances, and other team info. For our second competition I try to add how the teams did in their first event. (A lot of it is condensed from The Blue Alliance).

Thank you for sharing!

Re: Tableau at PNW Districts
 

Attachment 20423Rachelle,

I emailed you our Tableau packaged workbook from Mt. Vernon for you to get an idea what it looks like.

I've also attached a selection of our data here for everyone (as I mentioned in a previous post, I am unable to upload the packaged Tableau workbook on Chief Delphi because it is the wrong file type). In this Word doc, I have 4 different graphs. We used all of these graphs to lobby Team 3238 to pick us for their alliance. We are fairly new to lobbying as well. We tried it without success last year. We're getting better at it and have had two successes now this year. We make a dashboard that contains three or four graphs on it targeted towards how we complement the team we are lobbying.

The first graph is the distribution of points for each team. Team 3238 is clearly the highest in points. We are second with the 2nd highest goal points.

The second graph is a scatterplot of Total Tele Defenses vs. Tele Goal Points. I love scatterplots. They can really put things in perspective. Let's say you were Team 3681 and you were a captain of alliance #5. You have nice goal points but you don't breach much. While 2605 has nice goal points and you might be able to capture the tower with them, you still don't have a breacher. If you think there will be lots of breachers left by the time you get to make your third pick, you can afford to take 2605. If not, you may want to look at 1778, 2907, 4911, etc. You could figure out this without a scatterplot or Tableau, but Tableau makes it easier.

The third graph is one of my favorites. This shows the shooting locations of two of our top rivals for Team 3238's first pick. Team 3238 did not want to have to jockey with its alliance partner for shooting position. We showed them that we like to shoot in their position, but we had 4 places that we routinely shot from, with two of the places being unique to us.

Finally, we used the 4th graph for picking the 3rd robot. Team 3238 wanted a defensive robot but there weren't really any left that had done notable defense in qualification rounds. We knew that we also needed a robot who could cross in auto and challenge on the batter at the end. By the time we got to the 24th pick, Team 568 was the best choice. They did great in eliminations. While they only crossed in autonomous 60% of the time and challenged the batter 40% of the time in qualifications, in eliminations they crossed 100% of the time and challenged 80% of the time. They also did creditable defense.

You asked about preparing for competitions. We went in to Mt. Vernon with data on 30 of the 36 teams we were facing. We used that information until all of the teams had a few matches under their belts. We will do the same at District Championship, video-scouting teams' latest competitions and uploading the data into Tableau. Again, we'll change over to their performance at District Champs solely once we have data from a few matches.

Good luck at your next competition!

Re: Tableau at PNW Districts
 

I would love to see how that would work out! I love the idea!