1114 the Statistical Outlier
 

So I was doing some calculations, and according to the OPR and CCWM rankings posted by Ed Law, 1114 appears to be a bit of an outlier.

The most striking statistic to me was the fact that as of Week 2, 1114 is 2.67 standard deviations above...

Second place.

:eek:

Re: 1114 the Statistical Outlier
 

They're quick, that's for sure. I expect the gap to tighten up a bit in the coming weeks. Not only will 2056 come in to play and likely give them a run for their money, but many of the other elite teams will have their kinks worked out.

Still, incredibly impressive performance out of the gates from them.

Re: 1114 the Statistical Outlier
 

Their design is literally amazing. However, they're going to do a lot more to win championships. If you were like me and watched Dallas, the scores were much higher than GTRC. 1114 needs to work more on incorporating all members of their alliance to thus max their efficiency. None the less, awesome robot, and I think they will go far.

Re: 1114 the Statistical Outlier
 

1114 isn't a statistical outlier... 1114 defines statistical outlier!

Jason

Re: 1114 the Statistical Outlier
 

They are honestly AMAZING. Well done, team 1114!

Re: 1114 the Statistical Outlier
 

You guys can go ahead and tell yourselves 1114 isn't actually that great, other teams are just as good, they have weakness X or Y... Whatever, if it makes you feel better about your robot, great.

But make no mistake about it, 1114 is the best in the world at this point. It isn't even close. There's nobody in FIRST who can come close to matching their output. Four stacks of six with cans is more than many entire regional winning alliances did, combined. They are outscoring not only multiple robot alliances, but they're outscoring teams with tethered human player stackers - their intake is that good, their stacker is that fast.

The craziest thing is that 1114 is a team that never stops improving, and we've just seen their first event configuration. Compare 1114 at Championship last year to their performance at their first regional - the strategic addition of a goalie pole, refined cycle completion, even better driving. The leaps and bounds 1114 is going to make with this machine should be astounding.

1114 hasn't been this much better than everyone else since 2008 - and that should scare anybody who is seriously gunning for a World Championship title.

Re: 1114 the Statistical Outlier
 

They were considerably better at their first even in 2015 than they were at their first event in 2008. They've never come out of the gate this strong.

I also don't know if this game permits the amount of growth that 1114 showed in 2008, either. At their later regionals, they were outscoring entire alliances by themselves... in autonomous.

Re: 1114 the Statistical Outlier
 

Part of the reason for this is that there were at least 4 very good teams at Dallas, while GTRC had really just 1114

I do not mean to insult the multitude of very good robots that were at both events, I'm merely trying to point out that the upper echelon included more teams at Dallas than at GTRC

Re: 1114 the Statistical Outlier
 

Some odd trivia:

1114's 2008 bot's name: SS

in 2015, their bot is named SideSwipe.

interesting coincidence?

Re: 1114 the Statistical Outlier
 

I can't imagine what they've already done since GTRC.

By the time CMP rolls around, they'll definitely have another Einstein finals robot if their process of "this is what we don't do well, this is how we fix it, and this is how we excel at it," goes smoothly.

Re: 1114 the Statistical Outlier
 

I would in fact agree with this point. However, you have to find a way to utilize every robot in order to succeed. Even if you have, per say, a very weak bin grabber, let that robot do that, even if it takes a little bit longer. You will have more time to stack, and they will cap in time. Just find the strong suits of each robot, even if they're small.

Re: 1114 the Statistical Outlier
 

Let's not get ahead of ourselves here. You can make a very strong argument for 148 as best robot in FRC. They consistently got 4 6-stacks, 3 capped. If I recall correctly, in a couple matches they even made a 5th stack 3 or 4 totes tall. In addition, we haven't seen their robot with its 28pt autonomous yet. I think 1114 and 148 are miles ahead of everyone else, but let's not forget that 1114 is not the only robot in their class.

Re: 1114 the Statistical Outlier
 

To be fair, 1114 was making four capped six stacks when they got their ramp working, and I think, provided their alliance partner brought them the bins (put them right next to them) they could make a fifth. I agree that 148 and 1114 are the top in the world though, I just think 1114 is a teeny bit better right now. Who knows what will happen when they've both improved their robots for world champs?

Re: 1114 the Statistical Outlier
 

I agree that 148 is very good, certainly one of the best, but they were not consistently getting 4 6-stacks with 3 caps, at least in quals. If they were, their OPR should be around 120, but it was actually 84. In contrast, 1114 actually did have an OPR of 120.

Interestingly enough though, their component OPRs for containers and totes are comparable. 1114 was scoring way more litter and auto points than 148 though. I expect 148 will be scoring much more in auto once they get Alfred back up and running.

Re: 1114 the Statistical Outlier
 

We have not yet seen the platform RC wars.

Re: 1114 the Statistical Outlier
 

It will happen sometime before Worlds, I'm rather sure.

Re: 1114 the Statistical Outlier
 

I have a lot of respect for 1114 and I can't think of a robot they have built as long as I have been in the sport that wasn't at least very very good. I never really bought into the whole fangirl movement they gathered over their history for a lot of reasons.

That being said, I knew when they showed off their fully-powdercoated robot, I felt comfortable making these assumptions:
a) They clearly found a design that they liked a LONG time ago
b) They finished that design that they liked a LONG time ago and I would love
c) I bet you would be better off clocking the amount of practice their drive team had with the official bot is better clocked in days instead of hours.

I'm hoping after the season is over someone on 1114 could give a timeline of how they moved through build season, because it looks like they had the robot ready to go by the end of Week 3. Not only is it a great design, they at least give off the appearance that they spent a colossal amount of time practicing with it.

Re: 1114 the Statistical Outlier
 

And just looking at that intake, the amount of iterations they must have gone through is amazing. You're absolutely right. I was scared when I saw that fully powdercoated robot.

Re: 1114 the Statistical Outlier
 

It will happen THIS week (Week 3).

I can feel it.

Re: 1114 the Statistical Outlier
 

Yes, it would be hugely educational and inspirational for the community (and greatly appreciated) if someone from 1114 could describe the various design options they considered, the decisions they made and the reasoning behind their decisions. Some of us have already been speculating on various topics in one of the other threads and would love to know what the real deal is.

Re: 1114 the Statistical Outlier
 

The top 6 teams are way ahead of the field. If the trend holds this will be the most lopsided year for which stats are avalaible. I'll post analysis in a couple weeks after almost all the teams have competed.

Re: 1114 the Statistical Outlier
 

I think Waterloo will be a good place for them to show/not show being able to work on a team - a lot of depth in that field.

Re: 1114 the Statistical Outlier
 

I don't think we need statistics to tell us that 1114 is an outlier:D

Re: 1114 the Statistical Outlier
 

Agreed... and it really shouldn't be that surprising (I'm not surprised, and I'm guessing you're not especially surprised)

This game is one where with good strategy, ingenuity, quality design & iteration, and plenty of practice, the sky is pretty much the limit... and your partners aren't required for high scores... hence why the top teams can put up tremendous scores. It's also a game that relies on acquiring and placing game pieces with care and precision... a huge factor in why it's very hard for low-effort or mediocre teams to have a consequential impact on the field.

Re: 1114 the Statistical Outlier
 

Congrats to 1114 on such a strong opening season.

I suspect it will be similar to 2010 as far as distribution goes. The top scoring team that year could on occasion, get into the 20s. In the next tier were a few robots that coupld regularly contribute in a manner worth about 8-9 pts. The median robot contribution was 0.8 pts/match.

FIRST games historically have a ratio of top teams to median around 6x to 10x depneding on the game. I didn't run the numbers last year, but I heard it was a historically low of about 4X. 2010 was around the 10X. I would expect 2015 to be close to 10X year as well, but likely not much higher as it gets harder to get higher scores as you go due to easy scoring material going first.

As others have said, there are a handful of teams that will likely get OPRs above 100, with a lot more knocking on the door to 100 (around 90-ish) by season end.

Current average is at about 15 with median around 13. I suspect the median will go up to about 14 or 15 by the end of the season, and the average might get to 17.
If the median gets to 15, then 10X is 150 which is doable, but quite tough to do on a regular basis.

Re: 1114 the Statistical Outlier
 

I remember seeing in one of Ed's spreadsheets that in 2008, 1114 was something like 5.67 standard deviation above the mean. That number is even crazier if you look at the p-value for it.

While there are always so, so dominant, I would have to agree. They are putting up points that seem to be about the limit of what one robot can do on its own, at least without a major change in the dynamics of the alliance. While they will continue to absolutely crush, other top tier teams will likely catch up to some degree.

Re: 1114 the Statistical Outlier
 

You missed something else here. This year, the GDC intentionally (or unintentionally, I hope) made a game that was significantly harder than previous years. Stacking totes isn't hard, but stacking totes, then placing containers on TOP of totes for the big scores is very hard. In addition, this is one of the few games in recent memory where a mistake can descore a tremendous amount of points. Make no mistake - this is a very hard game to play.

As a result, you have a game where a handful of teams are doing very, very well. The rest are struggling mightily.... and in some cases even being shuttled off to the side and not participating so they don't interfere with their stratospheric partners.

It's not like last year where even the most basic robot could do an assist. Unfortunate and probably accidental, but that's the game and teams want to win.

Re: 1114 the Statistical Outlier
 

The biggest problem with this game is that the returns grow exponentially. If your robot can just cap stacks, it's worthless without a great, fast stacker. If your robot can just stack six, it's not putting up stellar scores without a capper. If your robot can just stack 3 and cap a 3-stack, you're doing okay, but you can't compete at anywhere near the top level for elims. But if you can stack and cap six, you're putting up 3 times as many points as anyone else. There are HUGE returns for "doing everything" and huge penalties for compromising any part of your design to do "one thing well". Teams who aren't powerhouses had to choose between hoping they get to pick a perfect alliance of specialists, or trying to do everything and over committing. It's no wonder the top teams are so much better.

Re: 1114 the Statistical Outlier
 

Which is why one cannot simply win on their own. It takes coordination with alliance members.

Re: 1114 the Statistical Outlier
 

At the regional level, that isn't the case if you do everything on your own perfectly, like 1114 is clearly capable of doing. The problem is only powerhouses can successfully achieve this herculean list of tasks.

But again, 1114 is definitely capable of working with other great teams - I don't understand why you doubt this is the case, or how you think 1114 is worse than other teams as a result of not needing partners to make their stacks?

Re: 1114 the Statistical Outlier
 

if you look at the scores for GTCR 1114 didn't need partners to win it.

Powerhouse teams pick strategies that allow them to not have to rely on their alliance partners because Quals can be a total crap shoot of who you get teamed up with. Being able to play the game by yourself is key to seeding high and then being able to pick other teams that play the game the way you want to play it.

1114 can and will play with other teams when necessary. It just wasn't (and might not be until Champs) at GTCR last weekend.

As someone said earlier if you need to justify your teams decision to build a robot that requires team work to perform by arguing against those who don't more power too you. Your challenge ins't 1114 its the scheduling deity to give you a qual schedule that pairs you with teams that compliment you so you can hope to score more than 140 points a match, to seed higher than the 1114's and hope they want you to pick them.

Re: 1114 the Statistical Outlier
 

I think you've hit the nail on the head here. An anecdotal bit of support for this statement - 1684 designed a robot capable of capping stacks; any size, we can get a container on them. However, at our first event, we never capped anything above two totes. Part of this problem was drivetrain/field related (deviations in field flatness and an unforgiving drivebase creating traction issues and poor handling - this also hindered our two-bin auto grab), but bigger was the issue that there were no stacks available to cap. Our QA was unfortunately low as a result, and our showing wasn't impressive.

Specialization seems not nearly as viable this year as in previous games, given the very well-matched alliance a specialist needs. The ROI on successfully doing everything is massive, and the ROI for successfully doing one thing very well is not very high. The possible exception to this is a step container grabber, which may turn out to be a viable niche position. Or maybe it won't - maybe the powerhouse teams can fill that role as well. I can envision an alliance with two high-speed step container grabbers and one high-scoring auto bot. We'll see how it pans out over the next few weeks.

Re: 1114 the Statistical Outlier
 

I predict that the winning alliances at worlds will have sub 1 second Recycle Can grab autonomous team members. Will the super can grabber become the coveted second pick or will they remain the 3rd pick. The platform RC's are the key in the coming weeks. Similar to the sub second mini-bots.

Re: 1114 the Statistical Outlier
 

Or it'll just be an Alliance Captain/1st Pick powerhouse that can also do capped 6-stacks and as a Noodle Jesus. There are certainly enough powerhouses attempting to do so, and it's entirely feasible. They have no reason to leave such a determining factor to chance/someone else when they can do it themselves. ...Similar to the sub-second minibots they fielded. It's actually worse this year, because potentially you just need one 3-4 can grabber rather than 2 minibot racers.

Re: 1114 the Statistical Outlier
 

With the speed that 1114 has shown in acquiring totes, I would be only mildly surprised if they managed to pull 2 containers off the center bump then grab 3 yellow totes, all in auto.

Re: 1114 the Statistical Outlier
 

To answer the first question: I was surprised that the top 6 teams are so far separated from the next tier. I expected a tilt in the distribution but not the segmenting.

On the second point: I agree with this assessment as well with the discussion of the multiple point gains for doing multiple tasks in a single robot. Last year spread that across 3 robots. Combining the multiplier with the difficulty that approaches some of the end games has really separated the top tier.

Even though we're in that top tier, I'm not happy with it because it makes the game less interesting and attractive to our participants and target student population. (But I also can't tell our kids to start throwing matches...)

Re: 1114 the Statistical Outlier
 

Not so fast on that point. That's not what is happening in the highest scoring regional playoffs. And the game design encourages that. In fact in a match where we had 1323 off the field while we added the can grabber, our alliance got the 4 pt robot set in auto to score 32 points total, the only perfect auto at the event. The field is so tight that having 3 moving robots while building many stacks becomes risky compared to the payoff.

Re: 1114 the Statistical Outlier
 

I think we can get this topic covered in a fair bit of detail at a potential presentation at the 2015 FIRST Championship Conferences in St. Louis.

http://www.usfirst.org/roboticsprogr...nd-conferences

Re: 1114 the Statistical Outlier
 

AAAAAAAH YES.

In other news - anyone notice that sideswipe's swipey arm is made of un-anodized versaframe?

I forsee a step bin grabber in the near future.

Re: 1114 the Statistical Outlier
 

Perhaps a bin Swiper? ;)

Looks like I need to find a way to get to St. Louis now! Always been in awe of 1114's strategical designs, so I'd love to hear more about the decisions firsthand.

Re: 1114 the Statistical Outlier
 

I don't know if you've seen this already, but it looks like they tried out their BinSwiper already (GTRC Quals 57): https://www.youtube.com/watch?v=r7-NigYFrWo

It looks like their first 2 bins are captured at about 2 seconds (better speed it up 1114 ;), the "chokehold" number seems to be 1/4 sec!) and they have the other 2 at ~12 sec.

*For some reason, Q57 isn't on WFN, so we just have this video of their 4 bin auto... until Waterloo?

Re: 1114 the Statistical Outlier
 

Simbot no swiping!

Re: 1114 the Statistical Outlier
 

Yesss! THANK YOU! Will have to go to Champs even if our team doesn't earn a spot.

Please also consider making a recording available. I am sure those unable to attend Champs would greatly appreciate the opportunity to get insights into your engineering process.

Re: 1114 the Statistical Outlier
 

If you want to learn more about our processes you can also check out the following resources:

Simbot Seminar Series: https://www.youtube.com/playlist?lis...o6sOEVMR821tkV

Podcast with the EWCP describing our processes: http://www.chiefdelphi.com/forums/sh...ad.php?t=96123

Team Created Resources: http://www.simbotics.org/resources

Re: 1114 the Statistical Outlier
 

You see, that's the thing: 1114 is successful because they work at it, tenaciously and incessantly. But it's no secret: Not only do they post reams of information for all to see, Karthik has spoken at CMP about their process for as long as I can remember. Most are recorded and available somewhere.

The way I see it, beating 1114 is easy: Just do what they do, only more better. :rolleyes:

Re: 1114 the Statistical Outlier
 

The key to winning football games is scoring more points than the other team.

Re: 1114 the Statistical Outlier
 

which they did do to a certain extent, but in this game there are two distinct strategies.
The 1st (and sometimes 2nd)-place alliance, which is the two best teams at the event. they both kinda do their own thing while the 3rd robot is off in a corner trying to stay out of the way while being as useful as possible (you'll see this at most events)
The later placed alliance that magically made it to finals: this team has everything down to a science. the all know who does what part of the process and has gotten it down to maximize their points. still usually lose to the first place alliance (as is what happened to my team at PNW Auburn Mountainview)

What I'm trying to say here is that if the first two (or sometimes even one robot) are good enough, the third alliance partner matters not. in other games they make all the difference, and they definitely will at worlds. the teams that prescribe to the first strategy can only make it so far when other alliances are maximizing scores with three quality robots on the field, something that is just so hard to do at a Regional or District level.

Did I actually end up responding to you? Cuz I was trying to say that sometimes the third robot means nothing (even if it usually helps)

Re: 1114 the Statistical Outlier
 

I meant win championships.

Re: 1114 the Statistical Outlier
 

610's tote Auto was used when they got 4 stacks of 6 with a container on top of all of them. (and I think they got a stack too)

Re: 1114 the Statistical Outlier
 

a very similar grabber deployment system to FRC 1318 (completely different way of getting them off that step, however)

Re: 1114 the Statistical Outlier
 

The engineering notebooks from 2013 and 2014 are pretty interesting as well. Are there any plans to release one for this year?

Re: 1114 the Statistical Outlier
 

Even out here in Australia, I heard that 1114 started CADing during week 3, and in previous seasons, normally later. A team that is able to spend that much time dissecting the game is pretty inspiring to say the least.
The one thing elite teams do is not spend all their time trying to make mechanisms work.
We spend a lot of time trying to make our ideas work, thus not being able to afford the time spent to wait that long given our situation and mentor resources.

1114 can correct me if I'm wrong, but I'm sure they can figure out where I heard it from.:]

Re: 1114 the Statistical Outlier
 

Thanks again, Karthik. I watched the "Strategic Design" video on several times in December and kept it and others in the series going as "background music" while doing paperwork at the office. I will have to download the podcast for future study.


I have never attended one of 1114's team meetings but analyzing the results of their efforts leads me to believe that their design process a lot like the one used at best run, most progressive and highest performing company I have worked for (15-20% growth, every year without fail, leading to market dominance, with the same headcount). It is interesting to note that that company also shared their "secrets" with other manufacturers on a regular basis.


The key to winning football games is understanding how one can ensure that one scores more points than the other team.

Re: 1114 the Statistical Outlier
 

This is not true. CAD is often finished by week 3.

Re: 1114 the Statistical Outlier
 

Interesting, and thanks for clarifying Karthik. Awesome!

Re: 1114 the Statistical Outlier
 

This year (as of the Ed Law spreadsheet I looked at before making this post, which was post-Week 2; I haven't run the numbers yet for Wk. 3), they were slightly over 8 standard deviations above the mean.

Re: 1114 the Statistical Outlier
 

After a fair number of teams came somewhat close to 1114's 143pt QA from GTRC... making them look not-quite-so-outlier-defining, it's looking like they will have a QA of around 200pt at Waterloo!

Mind = blown

Re: 1114 the Statistical Outlier
 

Makes sense. that 143 was almost entirely 1114's contribution to the alliance. Now that there are stronger teams, their contribution is the same and their partners are adding to it.

Re: 1114 the Statistical Outlier
 

Their current QA at Waterloo is 215.60
The highest an alliance without 1114 on it has scored is 215 (Dallas F2)

Re: 1114 the Statistical Outlier
 

We're all just small fish swimming in the Simbotics fishtank....

Re: 1114 the Statistical Outlier
 

If they indeed get a 200pt average that would be crazy. Let's not forget that their current average is paired with 2056 twice and 5406 once making this more of an eliminations looking average...

Re: 1114 the Statistical Outlier
 

Note sure if it was Quarterfinals Match 2, Semi-finals Match 2 or Finals Match 1 of Wisconsin Regional, but 2530 stole a can right off 3418s plunger above the step. One of the most epic moves in 2530 history.

The RC wars HAVE begun!:yikes:

Re: 1114 the Statistical Outlier
 

138 pulled a can out of our gripper in GSDE in week 1! ;)

We've been working to counter that move since...

Re: 1114 the Statistical Outlier
 

I'll admit, I haven't seen any video from Wisconsin, but whatever you did there, I have trouble imagining that it's any more epic, or more inconceivable, than this.

Re: 1114 the Statistical Outlier
 

After 4 weeks, based on the "net OPR" (minus Coop OPR), 1114 is now 8.79 standard deviations (SD) above the mean OPR. The previous high was 6.53 for 1114 in 2008. Only one other team has been above 6 at season's end.

But that skew doesn't stop with just one team here's the number of teams more than 5 and 4 SDs above the mean by year. Cumulatively, only 8 teams have ever been above 5 SD before; now we have 6 teams. And in the years with 6 teams above 4 SD, there were either 1 or 0 above 5.

SD 5 4
2015 6 10
2014 0 3
2013 2 3
2012 1 6
2011 0 6
2010 4 3
2009 0 1
2008 1 3

Re: 1114 the Statistical Outlier
 

I suspect that those number of teams above 4 and 5 SDs for 2015 will go down come champs when more high scoring teams start playing more matches together.

Did you use Ed Law's spreadsheets for all of these numbers?

Re: 1114 the Statistical Outlier
 

This is also due to the fact that frc is growing faster than ever and there is a wider possible point spread than many years.

Re: 1114 the Statistical Outlier
 

Good teams playing together should have a very small impact on OPR... it seems about as likely to slightly decrease or stagnate after Week 6 or 7 as it is to increase. Teams can't easily double scoring contribution during auto (if you have two 3-tote-autos in the same qual match) and have to work harder to double teleop scoring... (i.e., have to work to get more cans off the step, easy to eat through the landfill, etc.). Then there's obviously the issue of field congestion.

Regardless, having high scoring teams together shouldn't impact OPR... OPR should 'filter' all that out. Obviously QAs will continue to rise to be higher and higher.

Re: 1114 the Statistical Outlier
 

Yes, used Ed's spreadsheet.

I did these calculations at 2 weeks, but didn't post them, then went back to 4 weeks since all of the strongest teams have played by now. While there might be some compression at Champs, the gap is enormous right now and unlike any previous year.

As to the expansion of FRC, note that the SDs have not been steadily expanding with growth (which makes sense when looking at unitless distribution measures). In fact the years with the next greatest skews are in 2010 and 2008. This whole year looks like its an outlier, not just one team.

Re: 1114 the Statistical Outlier
 

Their average at Waterloo for quals was 183.3. They then formed an alliance with the second team who had an average of 174.4. Scores off the charts. Mind officially blown. :ahh:

Re: 1114 the Statistical Outlier
 

As a visual person, I would love to see overlapping histograms of the OPRs for all of the years that Ed has spreadsheets for. It would be useful to have a sort of "gini index" for different years. The chart should show if there is a trend towards more skewed performances, or if it's a game specific thing. I'd worry that a long-term trend of this kind would start to turn "inspiring" into "discouraging". I'll try to make the chart when I get some time, unless someone beats me to it.

Disclaimer: I am in slobbering love with Simbot Sideswipe, and I'm not calling foul over their accomplishment. I'm just interested in the meta trends of FRC, and their impact on the sustainability of the sport.

Re: 1114 the Statistical Outlier
 

I have a couple of charts done already but I haven't had time to load them up here--it's trickier than on other websites.

The skew appears to be game specific and not a trend. The games that seem to get the highest praise on CD have the lowest skew.

Re: 1114 the Statistical Outlier
 

Try uploading them to a 3rd party site (Flickr, Dropbox, Google drive, imgur) and posting a link. Otherwise, to upload to Chief you have to go here:http://www.chiefdelphi.com/media/photos/upload/

Re: 1114 the Statistical Outlier
 

As you requested. :)

http://www.chiefdelphi.com/forums/sh...89#post1463289

Re: 1114 the Statistical Outlier
 

Thanks, did it and waiting for approval.

Re: 1114 the Statistical Outlier
 

Thank you (and your team) for sharing yet again.

Re: 1114 the Statistical Outlier
 

Here's the 3 charts I posted with OPR distributions for 2008-2015:

http://www.chiefdelphi.com/media/photos/41817

http://www.chiefdelphi.com/media/photos/41818

http://www.chiefdelphi.com/media/photos/41819

A couple of threads off of them:

http://www.chiefdelphi.com/forums/sh...hreadid=136227

http://www.chiefdelphi.com/forums/sh...hreadid=136224