The Art of the Pick List

Creating an accurate pick list that reflects your team’s priorities for Alliance Selections can be an art form. After working with an experienced mentor on it, this became especially clear.

So my question is, what strategies have worked well for your team when compiling your pick lists? (Ignoring recent FIRST API issues)

7 Likes

Having extensive data is our #1 priority. This season I’ve created an intense Google sheet that is used as data entry for our paper sheets (No FIRST API issues to worry about). All the data from the sheet is put into the form which is a sheet hooked up to a script that either creates a sheet for a team if it hasn’t been created or inputs all of the data points into a row that represents the match.

Each team’s sheets then automatically pulls pit scouting data from a sheet hooked up to a Google form. And then insights are calculated based on match data. Where a team starts in auto, what notes they pick up, where on the field teams are scoring from, notes they attempt and score in auto or teleop, shuttled notes, endgame percentages and success rates. Every metric we can reasonably gather we do.

The most important insights are then put into a master list where the most important data for each team is compiled. Then we have lists in the compiled spreadsheets that rank teams by speaker performance, amp performance, average points added ( our own form of EPA), who shuttles the most, and best defense

Then you just have to decide what is important to your team. We prioritize a speaker bot with good auto first, and then the third bot needs a compatible auto and fills in or aids any area we are lacking.

The more data you have the better you are positioned to pick a surprise bot that others overlook because of not having name recognition or having an unlucky schedule.

4 Likes

I will be posting a more specific version of this post on our OA blog later this week. But here is the quick notes.

At the FIN Columbus event this past weekend, we received quite a bit of shock and awe at our first overall selection. Then we won.

The art of making a pick list is about data. Scouting and having good ways to view the data is huge, but making a pick list isnt just about data. there are many other factors that go into where we place teams such as recent elims history, drive team experience, drive coach experience, drive coach ability, drive trains, and even overall compatibility of drive teams.

My OA post coming up will delve a bit more into this, but yes it is an art, and not always purely data based.

9 Likes

A big thing to consider is to realize what u want on your alliance, and then sort your data by that.

E.x. if your bot is really good at amp, you can sort by which bots score the most in speaker

6 Likes

If it works for you great, :slight_smile: But be warned, sorting through stuff and having faith that the data are correct is another matter.

  1. Having the data that matters is my priority (provided it can be collected easily). It is easy to swim in data and have no idea what you are doing. Going overboard on data (especially things that need marks or button taps while the gameclock is running) tend to result in data quality issues (big buttons and eyes on field = good). See point 5. (e.g. collecting all 27 grid positions last year when 9 (or even 3ish) would have done the trick. Very few teams last year used all those 27 data points. )

  2. A pick list needs to be easily readable, easy to cross off teams as they get picked, and error free.

  3. A pick list can have a flow chart format (if alliance 1 does such and such then do this; or if all defense bots picked, go all in on such and such) this takes practice

  4. You never need to go beyond 24 teams on the list if you are the number 1 seed at a standard event. You do not need to rank everyone.

  5. All your decisions do not need to be made the night before, just get teams into a rough order and see how they play the next day.

This is by no means a complete list and reflects my opinions.

8 Likes

I would echo that having good data is your basis for a good pick list. On our team tries to extensively watch the matches and keep count of the points scored by each robot. This allows us to use google’s looker studio to crunch data and get some lists like this data or this data.

We try to estimate the average points contribution for each task and each robot. That is a very good baseline to start with. Also, our averages work really well to predict matches, so that is a good indicator for picking alliances too.

After you have good data, you need to have a strategy for your alliance. What do you do well and what do you need to complement your team? The first choice is likely going to be easier, especially if you seeded higher. This year we had mostly to figure out which bot was the best speaker shot for us to pick, as we had the amp covered, and watch a few matches to see how they would hold up when it came to tough situations in the playoffs.

For the first pick you can be a bit more involved and look at some matches and add the qualitative feel to it. Understanding how they play and if they have good metrics because they had easy matches or if they will be able to deal with tougher matches and defense.

For the second (or third) pick. You likely want a rule, not specific team numbers. By the time you get to that pick, it is very hard to predict which bots will be left. Again I would go back to figuring out what your elims strategy is. Do you want someone to play defense? Do you want someone to feed notes? Do you want someone to play speaker/amp?

By knowing what you want to do, you can tailor a specific rule that might show you the bots for that job. In our regional, we wanted to keep our speaker zone clean to avoid getting bogged down by defense, so we wanted to have only 2 bots scoring on the speaker. Because of that, for the 2nd pick we wanted someone who could play defense or feed notes to the other 2 bots.

To get that, we looked at two main rules.

  1. For playing defense we wanted a team with a good drive base and a swerve. In our regional we still have a small amount of swerves, so we made sure to have a list of all of them to not miss any opportunities on good bots that might get forgotten.
  2. For feeding notes we wanted someone with lots of cycles, so we created a new list that only ranked robots by total amount of notes scored during teleop, regardless of where.

These two metrics showed us some nice bots. Since we did that on the night before the last day, we had the last morning to go around looking at a few of the possible second picks to be able to know how the main ones played if it came to choosing them.

Another important step is to try to cross out a few teams that might not perform to their fullest during elims. You can list teams who have broken down too often or gotten too many fouls to either avoid or at least have a look at in the next morning and figure out if it is something that can be fixed or not.

A good pick list is something that you can put as much effort as you want, but you have to know when to stop. Last year, me and a few team members who were not in houston pulled an all nighter to watch at least 3 matches for all 30-ish bots who were in our pick list. It worked great and even our 3rd pick was a really good robot, but if you are part of the team that will be playing the next day, I would highly advise against it. You have to be well rested to do your best the next day, so you will have to pick a few rules or metrics to look at during alliance selection and call it a day.

4 Likes

Thus is the reason why we have an “Insights Compiled” spreadsheet. There we collect only the most important details. Averages, Driver Skill, Maxes, Endgame, Points Added, Pieces moved. Teams are then auto-ranked in different categories and for our final list we can deep dive into their individual team sheets for any small things that move teams up and down our lists for each role for our real list.

2 Likes

There are lots of different ways to run a picklisting meeting, and every team needs to find what works for it. (Not to mention the considerable variation on a year-to-year basis.)

That said, the most successful picklisting meetings I’ve run have been inspired by mergesort.

Key idea: students get more engaged, learn more, and you get a better product when discussions are smaller and allow for more back-and-forth. However, you probably have a lot of different people that you want input from. Break your single large discussion into many small discussions with one trusted strategist running each discussion. Have each small discussion be responsible for evaluating a certain list of teams. After they complete that task, bring the trusted, proven strategists together to merge the lists.

2 Likes

So our team worked with team 2471 a few years ago to put together a presentation about what goes into creating an alliance.

and then we had a Q&A sessions

6 Likes

@Brian_Maher and I wrote a 3 part series about it here:

We talk about specifics of how we do things with anecdotes. Brian also gave a talk based on the series and I’ll let him link it.

5 Likes

Lazy pick list tips:

You don’t have to precisely order the teams that you will never get to pick. If you’re ranked 5th, don’t think too hard about where the 4 teams above you are on your pick list unless you really feel you would want to decline one of them.

Concentrate really hard on the part of your list that you think is likely to affect who you get. Create a couple mock drafts where you guess what the other alliance captains are going to do. Figure out which teams you’re likely to get. Imagine alliances with those teams, and truly figure out your order of preference.

If you’re a top alliance captain and your robot is one of the best at the event, rank based on reliability. Your biggest chance of losing is if you or one of your alliance partners has a bad match. In alliance selections you can’t really affect whether your robot breaks, but you can select reliable partners.

If you’re a low alliance captain, get creative. Put together an alliance that could, given the right circumstances, upset the “better” alliances. I’m talking about those #8 alliances in 2012 that won events with triple balancing, that type of thing. Put something together and throw a Hail Mary.

If you’re a middle alliance captain, put together an alliance of robots that have shown inconsistencies or have drawbacks but, if every bot works correctly, will excel. Don’t be afraid to pick low seeded robots. Find the diamonds in the rough that other teams might be wary of picking.

I’ve got a lot more to write about this topic, but I’ll leave it at this for now.

3 Likes

This is a great point. It is easy to drown in data!!!

Most people that I’ve worked with (including myself) operate on the assumption that more data = better. I am here to protest that! There is nothing worse than a 3 hour pick list meeting that leads to 0 decisions and lots of fighting. This tends to be the case when people don’t agree on how to INTERPRET the data and make decisions on it.

The best pick list meetings have had a focus on what we are looking for, and discussion oriented towards results. Save the data analysis for later when there is more time and cooler heads.

4 Likes

Random bit of advice, but your data collected, metrics calculated, picklist dashboards should change throughout the year. Like how the game changes as robots improve, scouting systems should also change. The data required for a week 1 event is going to be different than the championship. What we started doing last year was only collect the data we 1000% know we will need for our first event and then add additional data for future events.

4 Likes

Scouting teams for notes intaked, notes scored (location irrelevant).
Consider weighting the 2 stats and discussing teams with those 2-3 above or below as they appear in the sorted table.
Observe other data to resolve “really similar” teams.

I think driver compatibility is impossible to scout by numbers and that is a conversation for scouters to have with the drive team “Do you guys like Team 1234 The Other Room”.

80% of the pick list should be made the night before, but the last 20% is the difference between good and great.

1 Like
  1. Rank by throughput/performance for whatever alliance role you want to pick for
  2. Don’t pick robots that will get in the way of the other robots on your alliance (and reduce their throughput)

That’s pretty much it.

2 Likes

I’m curious about this. Do you not find it helpful to see if a robot scores in the amp or in the speaker?

2 Likes

On a personal note, our robot can comfortably score in either, so it doesn’t matter for us.

In general we sort according to the game pieces. The scoring location doesn’t matter until we have a team composition, so we have footnotes for Ampbot or Speakerbot (both pit and match scouting data) but that doesn’t frequently affect our order.

Instead of changing the order on the list proper we keep an ear open to hear out our partner to steer us towards an amp or speakerbot of similar value.

1 Like

We use Notion as part of our picklist ordering (not for the actual scouting though) and I really like that it lets you have tags. So aside from the numerical data, we also have tags for things like subwoofer shot, med shot, and wing shot, and then for climbing, trap climb, center climb, and side climb.

Being able to see these in our list means we can quickly go “Hey, we want a robot that is capable of doing a harmony climb with us for more points, which ones have a side climb?”

2 Likes

We use Excel/Google Sheets instead, as it gives us more functionality and formulas when it comes to sorting. Does Notion have any similar built in features?

1 Like

Kind of, but we don’t scout with Notion though (maybe we could), 3128 has developed an excellent scouting app and have organized multiple alliances to scout into a shared data pool. So that is where all the formula stuff comes into play. Then once we’re actually deciding the order, we take the team rank order from that, and then use Notion to drag it around in a different order based on X factor stuff. We can also embed photos of robots in the entry, in case we need a reminder. It’s much more of a database than a spreadsheet.

I think enough time has passed that no one would get mad at me for showing our 2022 PHR picklist from Notion. We have definitely improved our tags and other columns since then, but this shows you the basics

video_11_Anonymous

I’m currently working on API pulls from TBA and Statbotics info so this data entry isn’t so manual (since we don’t have an API to 3128’s scouting app)

5 Likes