Highest ranked team not picked?

[jamesmcip]

Originally Posted by lovelj

In a nutshell, was ranking a poor metric of performance this year, more so than other years? I don't recall so many low raked teams being picked so early compared to prior years. What was it about this year's game that skewed ranking from individual performance?

This year definitely seemed like many great teams seeded extremely low for their ability. If your team is stuck with a dead robot (which occurred far more often this year), then you are pretty much screwed for the capture and it can make breaching difficult if they are right on a defense. Losing just a few ranking points due to some unlucky events can be the difference from seeding top 8 to seeding bottom 20.

This year at our first event for the first day we were only with one other functioning robot (or even once by ourselves) more often than we were with 2 working robots capable of at least challenging the tower.

I remember in Boston team 125 suffered from a lot of issues very early on and got rather unlucky so they seeded 28th, however they were a first pick of the second alliance and went on the be finalists at the event.

[Ed Law]

Originally Posted by lovelj

Ed, thanks for posting this. This is some of the best advice I've seen on CD, especially in terms of marketing your team. What is your opinion on the rate of low ranked teams being picked so high this year? I've seen this before, but don't recall it at such extremes in each division. Am I wrong or was there something about this year's game that amplified this effect.

Before I say anything else, I want to say the GDC did a great job. I really like this game.
Ranking is usually not a good indicator of robot ability especially with 10 matches and 75 teams. It is not enough to bubble up the best robots. On top of that the ranking point system is not just based on wins and losses. I really don't mind it too much. It is just a different objective function to try to maximize. It makes the strategy more interesting. However the RP rewards teams for things other than just robot ability.
This year the main reason for the ranking system to fail in bubbling up the best robots is because of the game. In many games, one exceptional robot or two good robots can carry the whole alliance regardless of who their partners are. This year that is not the case. That makes the match schedule the primary determining factor of the outcome. Teams that have a "lucky" draw will rank higher than if they had an average draw. There are many teams who can contribute to the scoring and very few robots can completely dominate the game on their own.

[Whatever]

The delta between 20th and 55th was roughly 5 to 8 RP.
That's two unfortunate matches.

[ANiyaz]

Originally Posted by lovelj

On the inverse, what was the lowest ranked team picked first?

868 was ranked 42nd in Carson after Quals with an OPR of 64.40 (2nd in the divison to 1024). 868 was the first pick of the 6th alliance.

[Brian Maher]

Originally Posted by lovelj

In a nutshell, was ranking a poor metric of performance this year, more so than other years? I don't recall so many low raked teams being picked so early compared to prior years. What was it about this year's game that skewed ranking from individual performance?

I may have had bad luck, but this season I noticed many more robots losing communication and browning out. I believe this plays a big role in this ranking phenomenon. In addition to a win being much harder to attain with two robots, the ranking point from a capture can almost impossible to obtain, depending on where the robot fails (unless you have the crazy pushing power of 25).

On Archimedes, one of our driver's controllers failed during our match with 1986 and 4183, rendering us immobile for the entirety of teleop. While they and our other partner still won us the match, we could not complete a capture. With that extra RP, 1986 would be ranked 13th rather than 16th and 4183 would be 25th instead of 29th. These are small differences for non-picking teams at a 7 team event, but they reflect the impact of failing partners.

On the flip side, one of our partners was a no-show and another lost comms at the beginning of the match. We lost both matches by less than ten points, and even with towers weakened to 0, could not capture. Those almost four RP matches became 1 RP matches. This is a six ranking point difference, which on Archimedes was the difference between ranking 20th and 50th, 10th and 34th, or 1st and 10th.


I've shared 1257's anecdote about not picking based on ranked before. I've quoted it below:

Originally Posted by BMSOTM

The issue with using qualification rank to assess team strength is that strength of schedule and luck with alliance partners can bias results. Just last weekend, 1257 seeded 52nd of 60 teams at MAR CMP. We averaged of five low goals in a match, and maximum of seven, but had bad luck in quals with alliance partners losing comms and such. 225 (who told me their scouting does not consider qualification rank) and 341 picked us for what we could do, and we won the event.

[EricH]

You know, if you're going to go with the highest ranked team that isn't picked, you gotta go with the #1 seed. They're NEVER picked.



Primarily because nobody's in position to try to pick them before they pick someone...

[wjordan]

Originally Posted by Caleb Sykes

Archimedes:
3932 at rank 12

Carson:
1137 at rank 14

Carver:
1262 at rank 12

Curie:
5279 at rank 19

Galileo:
4499 at rank 15

Hopper:
686 at rank 13

Newton:
2035 at rank 15

Tesla:
2522 at rank 13

4/8 (686, 1137, 1262, 5279) are CHS teams... that's a bit odd.

[CVR]

Originally Posted by Steve Compton

We were part of that cohort, I think, and my kids will want to take a look at why were were passed over as they plan for next year.

I would consider what your goals were in the beginning of the season and how you achieved them.

If your goal this year was to win a regional/district and make it to worlds, a low goal bot could easily do that. A good example is Team Dave 3683 at Greater Toronto Central Regional. They were 9-1-0, ranked 1st and pretty easily won all their elimination matches.

If your goal this year was to be extremely competitive (win a DCMP, CMP division, or compete on Einstein), a low goal bot was not really feasible. Of all the robots that touched Einstein's carpet, I don't remember any low-goalers. I might be missing one or two, but I'm fairly certain almost every bot was high goal or defense.

That in mind, most alliance captain's goals in St. Louis is to first make it to Einstein, then be competitive there. I think most captains rightly concluded that you needed at minimum 2 powerful high goal robots, and a defensive bot.

Since you guys were primarily low goal scorers, you most likely didn't meet the criteria of what alliance captains were looking for.

----

If your goal for next year is to be competitive on Einstein, you'll have to identify what robot role will be most desired on Einstein. This year it was high goal scorers. Last year it was robots that could make stacks 6 high and cap it themselves (minimum 2).

I would forwarn you that having an Einstein competitive robot does not guarantee you a trip to worlds. There are many teams this year that could have been competitive on Einstein but never qualified.

A good example is team 120. We were ranked 1st at Buckeye and passed over them because 781 could put more boulders in the tower. Even though 781 was a low goal bot and would get less points, we knew that we would be the only alliance that would be able to reliably capture and the 25 point bonus would almost assure us wins. So we didn't need a high goaler.

I can't speak to 120's other events, but I suspect something similar happened. Their robot wasn't quite what was needed to win regionals. Had they not been a HoF team, they would not have been in St. Louis. But luckily for 330 and 2481 they had an automatic invite, and their fantastic machine was available for selection.

If you look at the finals matches on Einstein, I suspect 2056's defense bot, 1405, was primarily told to shut down 120. This makes the battle between 120 and 1405 the most crucial part of the finals matches and where World champs was won and lost.

However, remember when I said we (4039) passed them up at Buckeye? We thought they weren't what we needed to win the regional, although they were what was needed to win World's.

There is a risk that if you want to be competitive at World's next year that you'll never qualify. So it's not as simple as finding the right robot role and building that robot, you really have to identify what your goals should be.

Usually Einstein-capable robots are highly complex and take a whole season to tune to perfection. The power-house teams (2056, 254, 971) are the exceptions.

----

My advice for if you decide that your goal next year is to compete on Einstein, is to identify how to score the most points per second in next year's game. This year the low goal and high goal took approximately the same amount of time, but one was worth 150% more points. A reliable high goaler was also way, way, way harder to build. But if you did it well, you had a good shot at Einstein.

But please remember Karthik's advice, which is applicable to 95% of teams:

“Reliability and consistency are paramount. If you do ONE thing, every single match, without fail, you will beat the guy who does many different things poorly. Every time.“

You just need to identify what that one thing is, and it likely won't be the same for winning regionals and winning worlds.

[Jscout11]

This reminds me of a similar thread Akash posted a few years ago before my time in FRC

I just wanted to say, as someone familiar with 1391, that from what I observed, you guys really took the next step this year in a lot of ways. Best thing you can do now is learn from both the good and the bad and start working now for next year.

[JamesBrown]

Originally Posted by wjordan

4/8 (686, 1137, 1262, 5279) are CHS teams... that's a bit odd.

1137, 1262, and 5279 are all effective low goal bots. 836, another CHS Low goal bot was a 4th pick. In Chesapeake low goals were enough to seed high and win. At Champs, at least in our Case, we were able to capture because we could put 6 balls in the low goal, this lead to more ranking points. However in eliminations teams could capture with only high goal shooters, diminishing the value of the low goalers.

Basically we were very valuable as a qualifying bot since when partnered with a single bot that could score 4 balls we would capture regardless of who the third bot was. However in elims teams were able to select two high goal bots who could score 10 balls between the two of them or 3 high goalers who could score 10+ combined. It is something we were aware would happen eventually.

Originally Posted by EricLeifermann

Rank =/ robot abilities

Good teams scout and rank doesn't factor in picking teams.

Originally Posted by AquaMorph

Rankings are not a good indicator for skill. Sometimes, good teams have a lower ranking because they had a tough schedule. Other times, the reverse happens and a team gets a higher ranking than they deserve.

This is certainly true, however there is another factor that plays into it, and it is the way the game changes between qualifications and eliminations. 5279 is a good example, I suspect if you replayed the weekend with other random schedules over and over again I suspect we would seed in the same area as we did on average. In this game, low bar bots that could score 6+ balls did well in qualifications, but were less valuable in eliminations because their were enough high goal shooters that could combine for 10+ balls to capture. I suspect that if they had not raised the tower strength to 10 then these low goal bots would have seeded lower, as more high goal bots would have been able to capture in qualifying with out them. On the other hand if they raised it to 12 then the low goal bots would have seeded higher and would have been more valuable to elimination alliances.

It is entirely possible for a robot to both deserve to seed where they did, and not get picked. In our case, while we were very valuable to the average alliance, we were less valuable as a complimentary part of a designed alliance. On the other hand a robot like 1662 could not contribute as much to an average alliance in qualifying as they were a defensive robot designed to beat very good shooters so they were not able to contribute to damaging the tower, however once an alliance had two robots that could capture without contribution from a third partners, a robot like 1662 jumped in value. Us ranking 19th, and 1662 ranking 65th both reflected (to at least some degree) our value to random alliances, also, 1662 being picked in the second round, and us missing eliminations also reflects our value to the alliances as designed by the alliance captains.

[Steven Smith]

“Reliability and consistency are paramount. If you do ONE thing, every single match, without fail, you will beat the guy who does many different things poorly. Every time.“

As a follow up note, this year 3005 went from never having made championships before, and the only regional finalist achievement made as a 2nd pick in 2011, to a 2nd seed in Alamo (regional finalist AC), 2nd seed in Dallas (SF), and AC of the Curie Finalist division... by designing a low goal capable robot with no hang. We could cycle 6-8 low balls per match. When picking our amazing alliance partners, we knew we were weaker than our 1st and 2nd pick, but had a strategy we knew should work.

On the flip side, this was the first year we ever made a practice bot, had our first robot done and functional before week 5, got 8-10 weeks of practice before champs, and never had a single thing break in our robot that took us out of a match (or lost comms).

Strategy + reliability is a winning combination in many years even with a simple robot, and the GDCs decision on how to structure quals this year made it an exceptionally attractive option for many mid-tier teams.

[Citrus Dad]

Originally Posted by CVR

I would consider what your goals were in the beginning of the season and how you achieved them....

There is a risk that if you want to be competitive at World's next year that you'll never qualify. So it's not as simple as finding the right robot role and building that robot, you really have to identify what your goals should be.

Yes, we took that risk last year. We had a bot that could only stack 5. 1671, 701 and 971 easily could have pushed us out of qualifying at each regional if our can grabber strategy had not been successful.

Originally Posted by CVR

Usually Einstein-capable robots are highly complex and take a whole season to tune to perfection. The power-house teams (2056, 254, 971) are the exceptions.

Exceptions to what? I know that 254, 971 and us all had very different robots at Champs than what we started with. Tuning through the competition season is critical.

[Citrus Dad]

Originally Posted by lovelj

Likely. I just ran Archimedes and Carson. 3098 was 53rd and was 1st pick of 8th alliance in Archimedes. 868 was 42nd and 1st pick of 6th alliance in Carson. Our team 3824, was ranked 47 and was 1st pick of 8th alliance.

In a nutshell, was ranking a poor metric of performance this year, more so than other years? I don't recall so many low raked teams being picked so early compared to prior years. What was it about this year's game that skewed ranking from individual performance?

1983 was ranked 40th on Curie, and was the fist pick of the No. 2 alliance (and they talked their way out of being the first overall pick as the combination would not have been high scoring enough.)

[Akash Rastogi]

A note in reference to Corsetto: he's been on other teams before 1678. At the time he was either 114 or 1662 I think?

Originally Posted by Mastonevich

I went to the thread and found the following from Michael Corsetto on 1678. It is very fitting for this thread as they went from the below comments from Mike to 4 times in a row Einstein appearances with one world championship. Hard work and determination pay off.

Haha I actually just went back to read my old thread as well. Funny how things change in a few years.

1391 is a great team and was arguably one of the best in MAR. Luck and who you know is a huge factor in getting picked at champs in the later rounds. For example, 228 is a decent robot with very capable drivers, but as the very last pick on Carver, we know our friends on 359 had to do a bit of persuasion for us to get picked.

Keep working and getting better, 1391 is more than capable

[Michael Corsetto]

Originally Posted by Akash Rastogi

A note in reference to Corsetto: he's been on other teams before 1678. At the time he was either 114 or 1662 I think?

I was actually with 1678 at the time, but took a brief hiatus from mentoring 1678 full-time when I moved to Lodi and mentored 1662 in 2012 and 2013.

This post is a crazy reminder to me of how fortunate I have been to work with 1678 over the last 9 years.

-Mike