STAMP (Statistical Team Analysis of Match Performance) is off and running!

[buss]

Add one more developer from team 180:
Steven Buss

[Ethulin]

Originally Posted by buss

Add one more developer from team 180:
Steven Buss

Well I can't edit the post but you have been added to the developers page on the site. Welcome aboard Steven!

[Ethulin]

I thought you could do with some teasers, so here is the match input field for STAMP:

STAMP is PHP and mySQL based so you will be able to run it on any webserver. We will even be issueing an easy to install apache+mysql+php suite so all you have to do is click "go"!

[Bill Moore]

Originally Posted by Ethulin

I thought you could do with some teasers, so here is the match input field for STAMP:

STAMP is PHP and mySQL based so you will be able to run it on any webserver. We will even be issueing an easy to install apache+mysql+php suite so all you have to do is click "go"!

Interesting concept, but performance can lie, if you only focus on scores. How do you account for a score of "0", when the best bot at a competition gets pushed over at the beginning of a match? The other bot may get penalized or DQ'd, but that doesn't help the bot laying on its' side unable to perform.

I have seen this every year since 2002, where a bot plays overaggressive penalized defense against one of the best bots at a comp, and the good bot is knocked over. This is where numbers go wrong. The best bot would score nothing for this match, and drop in your standings, but in the playoffs, these actions drawing a penalty would result in a DQ for the offending alliance. So, even though the best bot may not score in the playoffs, they would still be credited with a win. The "quality" of the best bots 0 score in the qualification match is very poor, because it was not earned through play, but by penalty. If the best bot played a full match and was blocked throughout, and was unable to score, then the 0 score would be fully deserved, and the "quality" of that number would be very high.

I can foresee matches this year, where a slower bot who can only score one ball at a time is ignored, to concentrate on stopping a better bot from scoring. The lesser bot, without anyone playing defense scores 15 points; the better bot, with three robots pushing him for the full 2 minutes 10 seconds scores 14 points. Which bot will you fear most come playoffs? At some point, just like all researchers, you have to assess the quality of the data you are collecting.

[Ethulin]

Originally Posted by Bill Moore

Interesting concept, but performance can lie, if you only focus on scores. How do you account for a score of "0", when the best bot at a competition gets pushed over at the beginning of a match? The other bot may get penalized or DQ'd, but that doesn't help the bot laying on its' side unable to perform.

I have seen this every year since 2002, where a bot plays overaggressive penalized defense against one of the best bots at a comp, and the good bot is knocked over. This is where numbers go wrong. The best bot would score nothing for this match, and drop in your standings, but in the playoffs, these actions drawing a penalty would result in a DQ for the offending alliance. So, even though the best bot may not score in the playoffs, they would still be credited with a win. The "quality" of the best bots 0 score in the qualification match is very poor, because it was not earned through play, but by penalty. If the best bot played a full match and was blocked throughout, and was unable to score, then the 0 score would be fully deserved, and the "quality" of that number would be very high.

I can foresee matches this year, where a slower bot who can only score one ball at a time is ignored, to concentrate on stopping a better bot from scoring. The lesser bot, without anyone playing defense scores 15 points; the better bot, with three robots pushing him for the full 2 minutes 10 seconds scores 14 points. Which bot will you fear most come playoffs? At some point, just like all researchers, you have to assess the quality of the data you are collecting.

Hello Bill and thanks for your interest!

We will do as much as we can to account for this type of situations. For example we will discount mathematical outlier, such as the scenario above. We also have a handy dandy "Notes" section for each robot so you know what happened!

We will also calculate many other variables such as if the bot's performance is increasing or decreasing, and if so by what rate. We will calculate the consistency (their standard deviation from their mean score) as well as many other variables. It is no doubt there is still a "human" element. You need some one who knows what their doing to sift through all this data and rankings to really determine what you need to do to win, this just makes it easier and more accurate.

Hope that cleared up some of your questions!

- Erik

[Bill Moore]

Originally Posted by Ethulin

Hello Bill and thanks for your interest!

We will do as much as we can to account for this type of situations. For example we will discount mathematical outlier, such as the scenario above. We also have a handy dandy "Notes" section for each robot so you know what happened!

We will also calculate many other variables such as if the bot's performance is increasing or decreasing, and if so by what rate. We will calculate the consistency (their standard deviation from their mean score) as well as many other variables. It is no doubt there is still a "human" element. You need some one who knows what their doing to sift through all this data and rankings to really determine what you need to do to win, this just makes it easier and more accurate.

Hope that cleared up some of your questions!

- Erik

Thanks!

I am indeed interested in seeing what the program can do.