Project ORB: A superb predictive scouting system!

[microbuns]

This is incredible. I love the fact that I can still pull up data for teams that are not at champs. The data looks pretty darn accurate. I have two questions:

1) Will these be updating as new matches are played during champs?
2) Could you release a top 50/100/3000? I'd be interested to see how your program ranks the robots, and compare it to OPR and my personal favorites. I could (and might) compile this myself, but it'd take a while, and you might have some way of doing it much more quickly.

[weaversam8]

Good morning everybody!

Originally Posted by microbuns

This is incredible. I love the fact that I can still pull up data for teams that are not at champs. The data looks pretty darn accurate. I have two questions:

1) Will these be updating as new matches are played during champs?
2) Could you release a top 50/100/3000? I'd be interested to see how your program ranks the robots, and compare it to OPR and my personal favorites. I could (and might) compile this myself, but it'd take a while, and you might have some way of doing it much more quickly.

We do plan on updating based on matches being played, not sure if it's live this morning, but if it's not, it will be as soon as possible. Also, I'll take note of your request about the top lists, we'll see if we can do it. It might interest you to understand our match scoring system:

Match Scores
For match scores, our system takes the highest skill level of each individual defense and assumes that is the score of the robot crossing the defense, then it uses that for purposes of scoring. Then, it calculates in low and high goal shots, and then calculates the scale/challenge bonus.

Originally Posted by BMSOTM

This is really interesting; I can't wait to see how well it predicts at CMP. In the meantime, is it possible you could release its output for some of the other competitions this season? I'd really appreciate if you could release its evaluation of the teams at MAR CMP in particular so I can see how accurate the results are.

In regard to releasing previous data:
I can do a data dump of whatever teams you like, including the ones at the MAR Championship, but right now, data in the system will be skewed by the fact that the matches at that championship became part of the team's score, which means it could be less truly accurate. We might have the option to release some data from earlier in the season, but we'd have to dedicate training resources for that, which may be one of our lower priorities at the moment. We'll see what we can do.

Coming next:

• We're looking into building a separate match ranking system that only takes this event into account, like how OPR individually examines an event.
• Top X teams list? (You can already kindof do this in a division by sorting)
• Slackbot to interface with the ORB API
• Recommended defenses for a match view
• And more...

[kinganu123]

How is the data represented for the neural network? I was thinking about this the other day, and wasn't too sure how you'd account for individual teams without essentially combining the teams' stats into one "super team" (for a lack of a better phrase).

[TomAwezome]

Howdy!! I'm Tom, Wired Wizards' Alpha Team leader (Robot Code), and I as well have been doing the vast majority of the back end for ORB.

Originally Posted by kinganu123

How is the data represented for the neural network? I was thinking about this the other day, and wasn't too sure how you'd account for individual teams without essentially combining the teams' stats into one "super team" (for a lack of a better phrase).

The data from the neural networks is saved into a database into tables for goals, defenses, challenge, and scale; goals has 4 values (autolow, autohigh, teleoplow, teleophigh), defenses has 9 values (0-8, 0:lowbar, 1ortcullis, 2:cdf, 3:moat, 4:ramparts, 5:drawbridge, 6:sallyport, 7:rockwall, 8:roughterrain), scale and challenge each have 1 value for their respective percentage. The goals values are representative of the quantity of goals, defenses representative of 0-2 crossings (as that is all TBA has per match), scale and challenge as mentioned are stored as a value between 0-1, a decimal percent. As each match is a combination of 3 teams per alliance, pulling the data gives you an idea of how they perform, but of course it is just a showing of how the alliance performs! The magic happens as the networks for each team analyze all of their different combinations, and finds the trends inside the dataset that would indicate the team it's training for. Good question!

[kinganu123]

Originally Posted by TomAwezome

Howdy!! I'm Tom, Wired Wizards' Alpha Team leader (Robot Code), and I as well have been doing the vast majority of the back end for ORB.



The data from the neural networks is saved into a database into tables for goals, defenses, challenge, and scale; goals has 4 values (autolow, autohigh, teleoplow, teleophigh), defenses has 9 values (0-8, 0:lowbar, 1ortcullis, 2:cdf, 3:moat, 4:ramparts, 5:drawbridge, 6:sallyport, 7:rockwall, 8:roughterrain), scale and challenge each have 1 value for their respective percentage. The goals values are representative of the quantity of goals, defenses representative of 0-2 crossings (as that is all TBA has per match), scale and challenge as mentioned are stored as a value between 0-1, a decimal percent. As each match is a combination of 3 teams per alliance, pulling the data gives you an idea of how they perform, but of course it is just a showing of how the alliance performs! The magic happens as the networks for each team analyze all of their different combinations, and finds the trends inside the dataset that would indicate the team it's training for. Good question!

Ok, so my followup question is how are you combining the teams into an alliance? Do you use something like CCWM to weight each team into a single alliance?

[TomAwezome]

Originally Posted by kinganu123

Ok, so my followup question is how are you combining the teams into an alliance? Do you use something like CCWM to weight each team into a single alliance?

The only data it uses is data it has pulled then trained. To compute how a combination of teams adds up as an alliance, it, for each defense, finds the max of the three teams' defense scores for that defense, then multiplies that by either 5 for lowbar and 2.5 for the other defenses. The same for goals, but with the proper point values for auto/teleop high/goal. And again, the same for scale and challenge, with the proper score multipliers. It then adds these all together and the result is that alliance's score. To determine the winner, it sees who is larger.