View Single Post
  #2   Spotlight this post!  
Unread 07-07-2015, 11:53
Nathan Streeter's Avatar
Nathan Streeter Nathan Streeter is offline
FIRST Fan(atic)
FRC #1519 (Mechanical MAYHEM)
Team Role: Mentor
 
Join Date: Feb 2008
Rookie Year: 2006
Location: Merrimack, NH
Posts: 676
Nathan Streeter has a reputation beyond reputeNathan Streeter has a reputation beyond reputeNathan Streeter has a reputation beyond reputeNathan Streeter has a reputation beyond reputeNathan Streeter has a reputation beyond reputeNathan Streeter has a reputation beyond reputeNathan Streeter has a reputation beyond reputeNathan Streeter has a reputation beyond reputeNathan Streeter has a reputation beyond reputeNathan Streeter has a reputation beyond reputeNathan Streeter has a reputation beyond repute
Re: paper: A Statistical Analysis of the Success of New England FRC Robotics Teams

I think your goal and process here are quite interesting... I think that it's strength of conclusions are probably being limited by the data (self-reported data and a lack of data that describes a 'successful' team).

Regarding using OPR... first, the use of OPR to define success is mediocre, because OPR itself is somewhat flawed (particularly in some games like 2014 where alliance score is highly dependent on partners) and because not all teams have the goal of best on-field success. You acknowledge this, and I do recognize that there aren't many FRC-stats. Perhaps the District points system would be a better point source though? It looks at a more all-encompassing and is actually quite meaningful over the course of a season.

Additionally, you mentioned looking at the OPR from teams from 2008 to 2013... but this data spans a tremendous length of time and a teams success in 2008 or 2009 probably has incredibly little bearing on their success in 2014 or 2015. Teams wax and wane; the best FRC teams now weren't the best teams then... even in NE which has many old teams. Definitely use more recent data, but do use multiple years (2 or 3, probably; although keep in mind the longer the span the further from the current survey snapshot you are), as some teams do just have 'off years' or get 'lucky' with a particular robot's success.

The self-reported stats are hard to really rely on... it means someone filling in data based off what they think you want (which may not be what you want), and that individual's knowledge of the information may be inaccurate (even if they're someone who should know). Just to prove a point though, look at the 'team budget' data... why do half the teams report a budget of less than 5k, when the minimum registration fee is $5k. Are these teams including only direct robot parts? Are some teams excluding registration fees? How about team apparel and team travel? What costs from team travel are included? How much of these items are self-funded my team members (and so, excluded from the budget)? I'm guessing there is an overwhelming amount of variation in these team budgets largely just from what is being included...

Team mentors, parents, and even student numbers can be difficult too. Some team rosters are huge, but are bloated by people that barely participate/contribute, while other fairly large team rosters may still have moderately high contribution from each member.

Anyway, I would strongly recommend implementing this using the NEFIRST district points from 2014 and 2015... I'm not sure what can be done about the survey data.
__________________
"If you want to build a ship, don't drum up men to gather wood, divide the work, or give orders. Instead, teach them to yearn for the vast and endless sea." - Antoine de Saint-Exupery
"The fight is won or lost far away from witnesses - behind the lines, in the gym, and out there on the road, long before I dance under those lights." - Muhammad Ali
"Simplicity is the ultimate sophistication." - Leonardo da Vinci


Student: 2006-2010 (#1519)
Mentor: 2011-Present (#1519)


Reply With Quote