paper: 2010 Team 1114 Championship Scouting Database

[Chris Fultz]

Originally Posted by Karthik

There may be some errors, but I'm confident the data is 97.1114% accurate.

Karthik, is that accuracy in Canadian or American?

Do you know the exchange rate?


Seriously - awesome work and awesome data.
Much appreciated.

[Drivencrazy]

FIRST off Karthik thanks alot. This is by far one of the most proffessional scouting databases I have seen.

Can you tell me where the Calculated Contribution comes from or point me to a forum that does?

Once again thanks and good luck at the champs.

[Karthik]

Originally Posted by Drivencrazy

FIRST off Karthik thanks alot. This is by far one of the most proffessional scouting databases I have seen.

Can you tell me where the Calculated Contribution comes from or point me to a forum that does?

Once again thanks and good luck at the champs.

This is explained in a few other places on these forums, but it's probably best that I put an explanation in this thread.

Going into an event like the Championship, it's impossible to watch video on all 344 teams. So, how can you get an idea of what each team's scoring potential is? Well, one way to do it would be just to look at their average score per match. Nice and simple, but it only tells you a small part of the story. Since FIRST matches involve alliances, an average score does not isolate the individual performance of a given team. So if Team XYZ repeatedly plays matches with great teams, their average score will not necessarily be an accurate indicator of the team's performance.

So, how do we isolate the impact of a single team on a match? Simple, using good old linear algebra. For those of your familiar with advanced basketball statistics, the method I'm about to describe is very similar to "adjusted +/-". Adjusted +/- has become a very popular tool among NBA franchises to try and figure out just how much each player is contributing on the court. NBA teams have long figured out that just looking at how many points a player scores does not always tell you how much they've impacted a team towards victory. There have been some great papers presented about this at the MIT Sloan Sports Analytics Conference.

So, here's how it works. For each alliance during qualifying matches at a regional, we set up an equation. Say Teams i, j, k were on an alliance together and scored s points. Out equation would be

T_i + T_j + T_k = s

where T_i, T_j & T_k are variables representing those teams. So if there are m teams at a regional, and n matches, we now have equations that give us a m x 2n matrix. We then solve this matrix for our variables, and voila you have each team's Calculated Contribution.

Why is Calculated Contribution so valuable this year? Well, consider a team that plays the midfield and is great at supplying balls to the home zone, but rarely scores them. If your scouts are just tracking goals scored, they might get a big zero. But their Calculated Contribution (if run over a large enough sample size) would should a higher value reflecting the points they helped their alliance score.

There are a lot more subtle details as to why Calculated Contribution is a good tool, and even more that expose some shortcomings. Unfortunately it's a bit much to go into on CD post while I'm eating lunch. I'll talk a lot about this at my Effective FIRST Strategy Seminar in Atlanta.

Hope this helped.

[hipsterjr]

This sheet on an ipod touch will be invaluable. Awesome job.
*ekk* (entered my team) looks like the data on us reaffirms our thoughts that we should focus on defense. This program has even aided in scouting my own team and influencing our stratagy! Now that is a testiment to it's effectivness.

[Tom Bottiglieri]

Originally Posted by Tom Bottiglieri

For the latter, do you think the slope can stay steep? Or is this one of those tortoise/hare situations?

I graphed the Calculated Contributions of a few teams over time. I'd like to do this again after champs and compare the results.

http://i44.tinypic.com/1zdryme.jpg

[Racer26]

It would be interesting to plot weighted CC over time from year to year and see if teams like 1114 are consistently getting better at FRC.

[pfreivald]

I like the calculated contributions bit, especially since we built a midfield robot that is designed to work as a part of an overall alliance, and was not designed as a one-robot scoring machine.

Good stuff!

[kjolana1124]

Originally Posted by Tom Bottiglieri

I graphed the Calculated Contributions of a few teams over time. I'd like to do this again after champs and compare the results.

http://i44.tinypic.com/1zdryme.jpg

Interesting. Can you please explain what the x and y axis each represent? I'm a little confused.

[Ian Curtis]

I presume the y axis is the Calculated Contribution, and the x axis is the event. So for you guys, at x=1 is your performance at WPI, and at x=2 is your performance at CT. Excel probably put in the .5s on its own.

[kjolana1124]

Oh, I see. Thanks

[Drivencrazy]

"Hope this helped."

Yes indeed it did.

Thanks again and I look forward to your seminar. I'll be there.

[Lil' Lavery]

edit: Woops, sorry about that. My general theory still applies.

I agree with Karthik's statements that "Calculated Contributions" (aka "Offensive power rankings" and "Defensive power rankings") worked very well in Overdrive given the more "isolated" nature of the game (still by no means an isolated event, though). I also agree with his statement that they should work much better for Breakaway than they did for Lunacy. However, as awesome as they might be, they still aren't telling the full story and should be taken with a grain of salt.

However, they still can be a useful tool. If you find yourself with a chance to be in a picking position Friday night at an event and don't really have much/any (reliable) scouting data, go ahead and run these types of numbers. Pick out some of the higher ranking teams in whichever attributes you're looking for. Have whatever people you can spare (maybe just your "media person," for example) do whatever they can to watch a match or two of these targeted teams the next day and see if their performance on the field backs up what the numbers say, and base your selection choices off of that.

That being said, this database is still one of the most wonderful resources in FRC. 1114 has done another amazing job on it, just like every year. It's tremendously helpful to find quick information about any team, even those not going to Championship. Kudos again to the Simbots for incredible work.

I do have one request, though. Is there any chance you guys can add in the Dean's list to a teams' profile? You list if a team won a WFFA award, so why not Dean's list as well?

[Karthik]

Originally Posted by Lil' Lavery

Hmmmm... 1712 ranked 3rd in the C_DefC in Philadelphia. Interesting, very interesting.
...
However, as awesome as they might be, they still aren't telling the full story and should be taken with a grain of salt. 1712 never once played defense in our qualification matches in Philadelphia (though we did in the eliminations, but those aren't factored in to 1114's CC). It's still quite possible that perhaps some of our strategies were more defensive oriented than other alliances' (though there were at least three matches where our alliance decided to forgo any defensive efforts). These calculated contributions are still subject to hiccups like this and biasing based on strategies.

Sean,

"Average Defensive Score" is actually not one of our advanced metrics. It's simply the average score of all the alliances a team faced in qualifying matches. So, a high Average Defensive Score means that the alliances you faced scored a lot of points. This year we ranked that stat in descending order on the theory that in a game with scoring that doesn't reward defense in the qualifying round, a team with a high defensive score was either consistently playing stronger alliances or consistently scoring for their opponent. Regardless, Average Defensive Score has no link to Calculated Contribution or OPR in anyway.

Originally Posted by Lil' Lavery

I do have one request, though. Is there any chance you guys can add in the Dean's list to a teams' profile? You list if a team won a WFFA award, so why not Dean's list as well?

We wanted to do this, unfortunately there were far too many gaps on the FIRST website. Most events displayed the winner of the award, but did not have team associated with the winner. We decided that it we couldn't list all the winners accurately, it would be better to leave the award out. (There were gaps with the WFFA's as well, but not as many and we were able to fill those with some Googling)

[Vikesrock]

Originally Posted by Karthik

We wanted to do this, unfortunately there were far too many gaps on the FIRST website. Most events displayed the winner of the award, but did not have team associated with the winner. We decided that it we couldn't list all the winners accurately, it would be better to leave the award out. (There were gaps with the WFFA's as well, but not as many and we were able to fill those with some Googling)

You just didn't ask the right people

Between myself and EricH we managed to track down all but one of the gaps for Fantasy FIRST scoring purposes.

[Chris is me]

Originally Posted by Vikesrock

You just didn't ask the right people

Between myself and EricH we managed to track down all but one of the gaps for Fantasy FIRST scoring purposes.

Can us FF people supply all that information to 1114?