Role of team history in picklisting

In Gettin' Picky: A Guide to Alliance Selections (Part 1: The Picklist), @Katie_UPS and I state our belief in using a teams’ prior season results as at most a tiebreaker between otherwise very similarly-valued teams. This point saw a fair bit of disagreement, so I decided to spin it off into its own topic with a few polls about the role of team history in picklisting.

When evaluating teams to make a picklist, how important is a team’s competitive success in previous years?

  • 5 - One of the most important factors
  • 4 - We frequently consider it
  • 3 - A minor factor
  • 2 - Only as a tiebreaker
  • 1 - Never considered

0 voters

When evaluating teams to make a picklist, how important is having past positive experiences with that team?

  • 5 - One of the most important factors
  • 4 - We frequently consider it
  • 3 - A minor factor
  • 2 - Only as a tiebreaker
  • 1 - Never considered

0 voters

When evaluating teams to make a picklist, how important is having past negative experiences with that team?

  • 5 - One of the most important factors
  • 4 - We frequently consider it
  • 3 - A minor factor
  • 2 - Only as a tiebreaker
  • 1 - Never considered

0 voters

Bonus points for anyone who can make a quantitative estimate of the impact of these factors in comparing teams, in terms of cycle count or any other actual metric.

In one of our recent events we let our past experiences with a team overshadow some very important factors, such as consistency in the current event… In the end we lost because of a malfunction on their side, and since then we decided to reduce it to a very minor tiebreaker, after all other factors have been considered.

2 Likes

For positive and negative experiences, it depends on how far back “past” goes - if drive team tells me that they’ve had bad experiences with another drive team at the event, that team moves further down the picklist.

Anything further back than that we generally don’t consider.

1 Like

Working well with a team can be really beneficial to the success of an alliance. I know I personally would take someone I know I can work with over someone who is maybe slightly better but I have less experience with. I know that influenced our picking decision during our first event (the team I was on in 2019) and we ended up taking it to a 4 match final against a very formidable alliance of 133, and 319. We were able to work together well and compliment each other well and I know I’d personally love to work with them again. Negative experiences are slightly less considered, depending on how far back it goes. My personal policy is that I don’t hold the actions of a few against the entire team, but if a team proves over and over again to be a bad alliance partner I’m less likely to want to pick them. I think team relationships are important but should not be the only factor in alliance selection; maybe in off-seasons if you want to have fun with teams you’re friends with. But working with teams you know well can be a really good experience when it’s the best thing for your alliance. Keep in mind this is just my opinion, but as someone whose taken part in the pick list process before this is just my 2 cents.

1293 maintains a special status on their picklist: Does Not Play Well With Others. If the drive team applies that label, that almost certainly pushes them out of the top 24.

Our philosophy is to rank all the way down, though we will establish a baseline where the team is a glorified BLT and simply copy-paste our quantitative rankings. (At Smoky this year, the threshold was 10 points/match. And remember, you get 6 for driving off and back on Level 1.) DNPWWO status usually gets you put just above that group, though sometimes we’ll penalize less depending on the situation.

As for good history, either at the event in a qualification match or in general, it’s a factor. We don’t assign a quantitative benefit, but it would buy a team the fraction of a cycle to get ahead of a team we don’t have the same warm fuzzies with when we look at them side-by-side.

1 Like

Team relations was a big influencer in our team’s picklist at BattleCry. A couple good teams went down on our list for the pure fact that we either didn’t feel comfortable with the team, or because we didn’t want to be treated a certain way. We constantly thought about our relationship with the teams as we were making our picklist. If you don’t work well with the teams on your alliance, there’s more of a chance the match strategy won’t go as planned. I speak from experience.

2 Likes

Honestly, I don’t even remember what happened during the match. Maybe we should start keeping track too. I do remember there was confusion (in one match) on who was to get on the Hab for endgame.

One thing we take into conisderation that may or may not fall under past competitve success is experience of a team in elims. If two teams, one a rookie and one that has been in elims before, are tied in robot performance, then we usually go with the experienced team. We do this because elims is stressful, so having a team that knows how to deal with that is valuable to us.

4 Likes

Something I’m aware of is that some teams have a list, usually fairly short, of other teams who they will not pick for a very long time. Usually, from what I’ve heard, the reason teams are on the list is due to either backstabbing or out-and-out lying or both.

Those teams may also have another list, somewhat longer, of teams they’ve paired with before and have a good relationship with–that would move those teams UP the list by several spots.

1 Like

It’s pretty easy to quickly rank those to "worry about " pre-scouting by actual # count of Blue Banner (competition wins) count. Then there is everyone else. I do this exam of the field, and it usually proves accurate.

People not believing history of of team counts are omitting a key metric

“The class handicapper judges the merits of a horse not by the time of his recent races, but by the type of company in which he has been competing”

So events won also factor in, harder event wins are ranked higher.

I haven’t yet gotten around to quantifying it yet (I’m probably going to be working on that for this season), but in the past we have factored in things like overall team history and the “we (don’t) like working with them” quotient. In 2017 we selected 2370 first overall because we really liked working with them, and we would solidify having two of the best gear runners at the event on our roster (week 2). There is a part of me that says we might have made a different pick (specifically, 1519) if the chemistry between us and 2370 had been weaker. In other events where we’ve been captains, we very often picked teams who had a pedigree to them; teams who were at the very least, competitive every year.

I won’t go farther on the “should you consider team history” thing than to say you should at least let it temper expectations of a team, putting the current results in context. Even sports scouting considers past tendencies to a certain extent. Would I necessarily let it be a primary factor? Absolutely not. Some portion of team history (and history between teams) is a factor, however.

So you would just cast 930 aside because they had not won a banner since 2004?

14 Likes

Our pick list is generally 85-90% data driven and how well we think a team has demonstrated that they can do whatever task on the field we think we need help with.

With that said the 10-15% is definitely history in the sense on how comfortable we have with them (especially the first pick). Usually, we’re the 5-8 seed captain so previous experience with blue banners doesn’t really lend itself since we’re not working with a lot of teams that are powerhouses at that point. An example is team 5572. They’ve never won a blue banner but our comfort level with them is very high. They always have a good robot, a disciplined drive team, & their drive coach is usually on the same page with me. So if we’re torn between them and another team we don’t know well we’d pick them in a heartbeat. We do have a few teams we’ve had bad experiences with but they’re good enough where we know we might need to play with them but we’d choose other people before them if other teams rank our close to them in our scouting session.

But our list is usually in rank order from 1 to 32 (usually a few extras) so we already drew the line in the sand and baked in our history on that list the night before alliance selection.

1 Like

Can you give statistics to show the predictive nature of blue banner counts?

4 Likes

I’ve seen teams go from consistently good playoff performance to not getting picked. My own team had 8 solid years of performance right off the bat, with our best event each of those 8 years resulting in: Highest rookie seed, 3 years as finalists, 2 as winners, and 3 as semifinalists. In fact, the last 2 years in that run was great - 2013 we were ranked 4th at our first event, and the first pick overall, along with the 6th pick at our second event, and in 2014 were alliance captains for a blue banner.

In 2015, we had a terrible robot. There was no way we were going to be in the playoffs, and we weren’t. Looking at the performance of the robot and the team at that event is much more important than looking at what they’ve done in past years.

I will grant, however, that going into Champs you can use the relative performance (and specific event strengths) as a good starting point, and then adjust as the event happens. But lets be realistic here - across both champs there are a total of 96 teams that are alliance captains. Realistically, about double that number need pick lists - 200 or so teams. Even if we double that again to 400 (half of the teams at champs… really?), you’re only talking about ~10% of all FRC teams. As far as pick lists go, you really should be focused on how the team is doing in the day or two before alliance selection, not weeks or years before.

1 Like

I don’t see anyone disagreeing with this. Boltman didn’t say to make the pick list solely based on previous results. He just said ignoring previous results is omitting useful info. Seems reasonable to me.

I actually think Jim Zondag made a post about this a few years back. I did a quick search and didn’t find anything though, so it’s possible I’m making this up. I remember the final generalization being: If a team was good in previous years, they’ll probably be good this year. If a team was bad in previous years, they’ll probably be bad this year.

1 Like

The hottest of hot takes

3 Likes

That’s a softball. Past banners have a very heavy correlation with future banners.

What you need to him to show is that blue banner counts have a stronger predictive nature than other metric that are tracked in-event.

I completely agree with that, I just asked if he had statistics to support his claim.

1 Like

can someone please run a match prediction model using OPR, DPR, elo, and blue banner count to determine the efficacy?