(Seemingly) Irrational alliance selection decisions?

[Brian Maher]

I was looking at alliance selection results from this season and reflected on how things played out at a few events. I noticed that there are quite a few selection decisions that I couldn't figure out the reasoning behind. I understand that teams don't always make the best decision. With a problem anywhere near as challenging as alliance selection, that's to be expected. However, occasionally teams just make strange picks. I'm sure you've all seen such selections.

In a similar vein, I've seen quite a few teams who've been declined and then advanced further than the team who declined them, especially in scorched-earth scenarios (i.e. high alliance captain picks other alliance captains who decline in order to keep them from partnering up). In those cases, it often seems like the declining team would have been better off accepting.

My questions for everyone:

• Why do teams behave (seemingly, to an outsider) irrationally during alliance selection?
• Has anyone noticed any patterns or common fallacies*?
• How frequently do teams make the "right pick" or something close to it? Is there a way to quantify/measure this?

*Aside from a lack of scouting data. I know from first-hand experience that it can cause problems for an alliance captain.

[Jay O'Donnell]

I'll address your second point first. I've seen teams decline for many different reasons. Some teams want the experience of being an alliance captain, some teams want to feel like they're being "strategic" and they overthink what they should do, and I've even seen a team decline what should have been an obvious accept because their representitive saw the 1 seed's last match where they tried high goal shooting for the first time. Most teams decline seeing who's left at that moment, not who will be left to pick when it gets to them.

As for why teams make bad selections, the usual suspect I've found is teams not having scouting or having misleading data. I've seen teams pick based on the fact that a team was consistently average through an event, and not pick teams who started off poor but got better throughout the event.

There's also the simple fact that many teams don't base their picks off of watching matches. I know many teams who pick based off of who they worked well with in a Quals match, or someone who they are friends with. It's also not uncommon for teams who don't know any better to pick the next team down in the rankings, despite them not actually being that good (trust me, I've been the not very good bot picked for that reason before).

Overall scouting is difficult for most teams to pull off effectively due to a lack of students, or just a lack of care by anyone to take data and use it effectively. The best teams have students who genuinely enjoy scouting and the alliance selection process. Personally being a picking team in alliance selections is my favorite part of a robotics competition.

[Zac]

As Jay pointed out, scouting is tough, plain and simple.

A few additional things that I have noticed which play into your three questions...

While teams may have scouting data, even good scouting data, they don't always analyze it well. What I have seen time and time again is that 3 good robots don't always equal a good alliance. Many of these games require the alliance to divide and conquer. For instance this past year, having 3 robots that all score 10 boulders in the high goal consistently then climb looks like a great alliance on paper. But if they all require using the low bar to cycle, and then need the center spot to climb... well thats just a traffic jam. A good alliance needs to be 3 robots that can work well TOGETHER, and bring out the best in one another. I have seen on many occasions an alliance of 3 good/ok robots work together very well and take down seemingly unbeatable alliances made up of awesome teams that just trip over each other the entire match.

One of my favorite alliances to be on from this past year was at the Boston regional. 1768 was with the alliance captain 1058, as well as 5563 (they actually came in as a backup robot early in the eliminations, and were an absolute jackpot). I have never felt so comfortable on an alliance. 1058 did what the were doing best and made sure to get boulders in the low goal for the guaranteed tower strength, this simultaneously gave the rest of the alliance a huge sense of relief. All of a sudden there was so much less stress to score X number of boulders, 1058 has that covered. This meant less stress for the other drivers. 5563 was absolutely awesome, and knocked down the defenses while being simultaneously invisible most of the time, they were cautious in when they crossed as to not disrupt any shots that were being taken, and they always stayed close to the defenses to allow for easy driving around for the rest of the alliance. 5563 also allowed for 1058 and 1768 to climb on the outside positions of the tower, and would then drive up the batter at the last second to avoid bumping either robot during the critical "get the hooks on" phase. It was fantastic. Each robot on the alliance allowed the others to play to their strengths.

This is also important to notice while scouting. I saw several instances this past season where a robot would cross several defenses, and get the check marks on the scouting sheets, but in doing so that robot cut off an alliance member, then crashed into the other alliance member causing them to miss a shot. Performing well while causing your alliance partners to perform poorly isn't an easy thing to quantify on a scouting sheet, and often dent make its way into the notes section.

~Zac

[Jay O'Donnell]

Originally Posted by Zac

This is also important to notice while scouting. I saw several instances this past season where a robot would cross several defenses, and get the check marks on the scouting sheets, but in doing so that robot cut off an alliance member, then crashed into the other alliance member causing them to miss a shot. Performing well while causing your alliance partners to perform poorly isn't an easy thing to quantify on a scouting sheet, and often dent make its way into the notes section.

~Zac

This is something Brendan and I have talked about before, the difference between quantitative and qualitative scouting. Quantitative data is useful for comparing teams that are far apart in ability, or when you are looking for specific skill sets. However it's important for scouts to be able to watch matches and see how each team is doing beyond the numbers and how well they'll work with an alliance. I often combine what the scouts see in terms of smart driving with how the drive team feels with each team that they got to play with.

[NShep98]

The only pick I have personally seen that would come anywhere near "irrational" for me was when one alliance picked the lowest seeded team, then the alliance immediately after that picked a much higher seeded team, where the two teams were the same 4 numbers mixed around. It may have been confusion, it may have been a valid pick, I don't know, but it seemed off to me.

Originally Posted by Zac

One of my favorite alliances to be on from this past year was at the Boston regional. 1768 was with the alliance captain 1058, as well as 5563 (they actually came in as a backup robot early in the eliminations, and were an absolute jackpot). I have never felt so comfortable on an alliance. 1058 did what the were doing best and made sure to get boulders in the low goal for the guaranteed tower strength, this simultaneously gave the rest of the alliance a huge sense of relief. All of a sudden there was so much less stress to score X number of boulders, 1058 has that covered. This meant less stress for the other drivers. 5563 was absolutely awesome, and knocked down the defenses while being simultaneously invisible most of the time, they were cautious in when they crossed as to not disrupt any shots that were being taken, and they always stayed close to the defenses to allow for easy driving around for the rest of the alliance. 5563 also allowed for 1058 and 1768 to climb on the outside positions of the tower, and would then drive up the batter at the last second to avoid bumping either robot during the critical "get the hooks on" phase. It was fantastic. Each robot on the alliance allowed the others to play to their strengths.
~Zac

Meanwhile, on the other side of the glass from this, we were the backup team for 5422 with 125 and 238. After making it into the finals (marking a District event where both finalist alliances had 4 teams, no less), we were told before one of the matches that scouting teams had had their eye on us, and the reason no one was told to pick us in the first place was because they forgot. No disrespect to the alliance captain or the team we ended up replacing, but not a fun position to be in.

[fargus111111111]

Originally Posted by Zac

This is also important to notice while scouting. I saw several instances this past season where a robot would cross several defenses, and get the check marks on the scouting sheets, but in doing so that robot cut off an alliance member, then crashed into the other alliance member causing them to miss a shot. Performing well while causing your alliance partners to perform poorly isn't an easy thing to quantify on a scouting sheet, and often dent make its way into the notes section.

~Zac

This is a very difficult thing to quantify and even notice, but it is something that 343 started to take note of on our scouting sheet this year. We made a row on our scouting sheet that rated the "quality of driving" from "professional" to "very drunk and lost" as much as this is subject to objectivity, making it a row on the sheet drew some attention to it and if a robot was marked "very drunk and lost" they were lowered in our rankings, if they were marked above about halfway on the scale (which was a 1-10 iirc) we didn't pay much attention to that row. I think that even though the an actual number may not really be that helpful, just having a place to give the driving quality a grade makes scouters notice it more. Kind of on the same note I plan to suggest a table to replace this row for the 2017 season that has speed on the x axis and accuracy on the y so we can get a better, more complete, though still objective, answer to what we wanted to know, which was, "how good are this team's drivers?"

As far as seemingly irrational decisions, most of them are due to poor scouting, certainly sometimes a team ranks high on luck and thus are picked or, I have been in this situation, where a team is picked because they have an old team number, so they must be good, right, but I have seen a few (very few) that were the result of very good scouting and those alliances, though they seem to be a very rag-tag bunch manage to do very well because they work well together and complement each other.

[jijiglobe]

In my experience the bulk of "irrational" alliance selection decisions occur when teams don't really talk to their potential picks before alliance selections. I think one of the clearest examples I've seen of this was this year's New York City regional.

First of all, I believe that my team (694) was the strongest robot at the New York City regional, but we only managed to seed 8th given the fact that we spend almost two entire matches not moving. It's not irrational to not want to pick a team that had these kinds of issues, however, I was surprised by how few teams asked us what actually happened.

The second strongest robot at the regional almost definitely team 3419, who seeded third. Going into alliance selections, I personally think that the most rational picks for the first seed (team 375) would be first team 3419, knowing they would most likely reject, then to pick us, who would almost be forced to accept given our low seed.

The reason I think that this would be optimal is that that I predicted that team 375 would be either eliminated by us, or by team 3419 (I admit that some of this prediction was due to my own conceit).

The key is that all of this is only optimal given the fact that our dead matches were not for repeatable reasons, and even then, it's still very much arguable that this wouldn't be optimal, as we could have more issues in eliminations.

As it turns out, our two consecutive matches of non-action were due to

1. Forgetting to revert to competition code after pit testing
2. leaving our breaker switch loose

Both issues were fixed completely, and I think if team 375 had known this, they might have picked us. Instead, however, we were picked by the fourth alliance and went on to win the regional without a single loss in eliminations.

Disclaimer: I don't know that 375 didn't have this information, only that they didn't ask me for it, or ask anyone in the pit while I was there. In fact, we may have talked to members of their team about it, but it is sometimes difficult to find the right person to talk to when negotiating about alliance selections.

Basically, I think that a lot of teams fail to make "rational" decisions are acting perfectly rational based off of the information they had at the time.

[Dominick Ferone]

Originally Posted by jijiglobe

In my experience the bulk of "irrational" alliance selection decisions occur when teams don't really talk to their potential picks before alliance selections. I think one of the clearest examples I've seen of this was this year's New York City regional.

First of all, I believe that my team (694) was the strongest robot at the New York City regional, but we only managed to seed 8th given the fact that we spend almost two entire matches not moving. It's not irrational to not want to pick a team that had these kinds of issues, however, I was surprised by how few teams asked us what actually happened.

The second strongest robot at the regional almost definitely team 3419, who seeded third. Going into alliance selections, I personally think that the most rational picks for the first seed (team 375) would be first team 3419, knowing they would most likely reject, then to pick us, who would almost be forced to accept given our low seed.

The reason I think that this would be optimal is that that I predicted that team 375 would be either eliminated by us, or by team 3419 (I admit that some of this prediction was due to my own conceit).

The key is that all of this is only optimal given the fact that our dead matches were not for repeatable reasons, and even then, it's still very much arguable that this wouldn't be optimal, as we could have more issues in eliminations.

As it turns out, our two consecutive matches of non-action were due to

1. Forgetting to revert to competition code after pit testing
2. leaving our breaker switch loose

Both issues were fixed completely, and I think if team 375 had known this, they might have picked us. Instead, however, we were picked by the fourth alliance and went on to win the regional without a single loss in eliminations.

Disclaimer: I don't know that 375 didn't have this information, only that they didn't ask me for it, or ask anyone in the pit while I was there. In fact, we may have talked to members of their team about it, but it is sometimes difficult to find the right person to talk to when negotiating about alliance selections.

Basically, I think that a lot of teams fail to make "rational" decisions are acting perfectly rational based off of the information they had at the time.

I would like to build upon this with the NYC Regional.
By the end of the first day we won all but 2 matches, the first being us and another robot having technical problem.(which we found out was caused by a bad controller) The second match was something silly, besides the point. We went back to the hotel and examined out scouting data, and we looked over all the bots. The ones that were very similar we put in a maybe pile, while our biggest concerns were people who complimented us. We did lowbar and A defenses in under a second so we needed someone to take care of B and D.

Our top picks included 3419, 694 and 5016. We went up to 3419 with the intentions of creating one of the strongest alliances in our opinions, and they informed us they were watching us. Something that hurt our chances, was up until that point we were still trying to do vision tracking and not succeeding( 694 and 1796 were in the same boat). So we wanted to prove we could shoot high and relied on our vision, instead of mounting our flashlight like 694 had. So the scouting report shows we weren't very successful with high goals.

Once we saw 694 we had to pick them, and we were initially planning on grabbing a defense bot since we assumed 5016 would get grabbed. When we they were avaliable we changed our plan to full offensive attack. I know during alliance selections 694 was hesitant about this, but in the end it helped prevail.

That and that our scouting report had a feature that told us the most optimal defense to put against the opposing alliance based a grading of success.

[Bluman56]

Originally Posted by Dominick Ferone

I would like to build upon this with the NYC Regional.
By the end of the first day we won all but 2 matches, the first being us and another robot having technical problem.(which we found out was caused by a bad controller) The second match was something silly, besides the point. We went back to the hotel and examined out scouting data, and we looked over all the bots. The ones that were very similar we put in a maybe pile, while our biggest concerns were people who complimented us. We did lowbar and A defenses in under a second so we needed someone to take care of B and D.

Our top picks included 3419, 694 and 5016. We went up to 3419 with the intentions of creating one of the strongest alliances in our opinions, and they informed us they were watching us. Something that hurt our chances, was up until that point we were still trying to do vision tracking and not succeeding( 694 and 1796 were in the same boat). So we wanted to prove we could shoot high and relied on our vision, instead of mounting our flashlight like 694 had. So the scouting report shows we weren't very successful with high goals.

Once we saw 694 we had to pick them, and we were initially planning on grabbing a defense bot since we assumed 5016 would get grabbed. When we they were avaliable we changed our plan to full offensive attack. I know during alliance selections 694 was hesitant about this, but in the end it helped prevail.

That and that our scouting report had a feature that told us the most optimal defense to put against the opposing alliance based a grading of success.

If you come again next year, you'll likely see the same kind of irrational picking. Apart from a few (emphasis on few) teams, NYC historically has one of the worst scouting collectives of any regional. It's really fun to observe year after year. I don't know if it's just me but there seems to be a huge disconnect between many (of course not all) NYC teams and the rest of FIRST. I sometimes wonder about how many of our teams even know of the concept of districts?

I don't mean to sidetrack the thread, I just enjoy a little quip on the NYC regional every once in a while.

[Dominick Ferone]

Originally Posted by Bluman56

If you come again next year, you'll likely see the same kind of irrational picking. Apart from a few (emphasis on few) teams, NYC historically has one of the worst scouting collectives of any regional. It's really fun to observe year after year. I don't know if it's just me but there seems to be a huge disconnect between many (of course not all) NYC teams and the rest of FIRST. I sometimes wonder about how many of our teams even know of the concept of districts?

I don't mean to sidetrack the thread, I just enjoy a little quip on the NYC regional every once in a while.

Oh I know, I've been to the NYC regional for 12,13,and now 16
2013, we were sure we wouldn't get picked and somehow we went to the semis, losing the first match due to field error and the second by a point with a weird alliance to say the least.

[Bluman56]

Originally Posted by Dominick Ferone

Oh I know, I've been to the NYC regional for 12,13,and now 16
2013, we were sure we wouldn't get picked and somehow we went to the semis, losing the first match due to field error and the second by a point with a weird alliance to say the least.

My fault, I should've better paid attention to your signature. Out of curiosity, how did y'all do a 180 after rank 50 in NYC to rank 1 in LI so quickly? (I didn't watch SBPLI that year )

[Dominick Ferone]

Originally Posted by Bluman56

My fault, I should've better paid attention to your signature. Out of curiosity, how did y'all do a 180 after rank 50 in NYC to rank 1 in LI so quickly? (I didn't watch SBPLI that year )

I think a combination of things helped.
Modified the shooter a bit, but more being confident in our own strategy.
When we went into eliminations for NYC we used our strategies which worked well in our favor. We continued on the trend through SBPLI.
The last couple years, the teams I have been on haven't had the best robots for sure. But it's not always about having the best bot as it is playing your alliance effectively.

It helped true the next year as rookies we used our alliance to the most of its potential and finished second, while being a new team with no experience and being a defensive bot.

[Kevin Leonard]

Originally Posted by Dominick Ferone

I think a combination of things helped.
Modified the shooter a bit, but more being confident in our own strategy.
When we went into eliminations for NYC we used our strategies which worked well in our favor. We continued on the trend through SBPLI.
The last couple years, the teams I have been on haven't had the best robots for sure. But it's not always about having the best bot as it is playing your alliance effectively.

It helped true the next year as rookies we used our alliance to the most of its potential and finished second, while being a new team with no experience and being a defensive bot.

I'll be a bit blunt here- 5030 did not "finish second" at TVR 2014, you seeded second. And while it's true that playing defense was probably one of the things your robot did better, that was only because it completely lacked scoring capabilities.

I'd argue that your seeding second was more due to consistently being paired with teams like 1126, 3015, 2228, 1511, 3003, 870, 3044, 20, 4203, and 174 more than it was due to your tremendous strategic chops. Being unable to possess a ball in 2014 made most strategies other than defense difficult, and put elimination alliances at a major disadvantage - being unable to complete three assist cycles.

5030 the next two years was far better, and while your robots failed to look like beautiful powdercoated creations, they did have tremendous value when playing to your strengths.

-----------------------

On the topic of irrational alliance selections, I'd argue that alliance selections are far more complicated than most people think, and that some teams might be looking at risk/reward, different strategies, and perhaps pick irrationally without thinking about how to actually win the event sometimes.

[BrendanB]

Originally Posted by BMSOTM

In a similar vein, I've seen quite a few teams who've been declined and then advanced further than the team who declined them, especially in scorched-earth scenarios (i.e. high alliance captain picks other alliance captains who decline in order to keep them from partnering up). In those cases, it often seems like the declining team would have been better off accepting.

This is always an interesting scenario especially for the team declining. Sometimes I have heard from teams who declined that they felt the field wasn't deep enough for a strong third pick and therefore wanted to be a lower captain to hopefully secure a better third pick than what would be available for the higher alliance.

For every decline that we see on the field there are times when teams get together with the intent to discuss a possible alliance, but the discussion ends with the conclusion that they won't be better off pairing together. Sometimes teams decide to pick lower than what is expected so as to save the other team from having to decline and make things awkward for everyone else (and save some time).

[Liu346]

After experiencing many different alliance drafts I have found why the picks are often strange and/or out of order. For many teams their picks are all precalculated and when it comes to actually selection a team the predicted scenarios do not always pan out. This is very difficult to change on the fly and most teams just go with who they were going to select anyways. Another crucial part of teams picking either deep of out of order is compatibility. This aspect might not be seen easily from the stands but when interacting with other teams it is everything. When these scenarios occur teams will pick strange picks in order to make there alliance strong as a team instead of three separate units. Yes, teams will make incorrect choices but being behind that mic is very intimidating and puts a lot of pressure on high school students. But in most cases I always assume there is a greater plan and that no alliance is to be taken easily.