Go to Post I voted to keep Stop Build Day but only for students. Mentors get to keep working on the robot. - marshall [more]
Home
Go Back   Chief Delphi > Competition > Championship Event
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Reply
Thread Tools Rating: Thread Rating: 3 votes, 5.00 average. Display Modes
  #16   Spotlight this post!  
Unread 28-04-2016, 00:32
asid61's Avatar
asid61 asid61 is offline
Registered User
AKA: Anand Rajamani
FRC #0115 (MVRT)
Team Role: Mechanical
 
Join Date: Jan 2014
Rookie Year: 2013
Location: Cupertino, CA
Posts: 2,215
asid61 has a reputation beyond reputeasid61 has a reputation beyond reputeasid61 has a reputation beyond reputeasid61 has a reputation beyond reputeasid61 has a reputation beyond reputeasid61 has a reputation beyond reputeasid61 has a reputation beyond reputeasid61 has a reputation beyond reputeasid61 has a reputation beyond reputeasid61 has a reputation beyond reputeasid61 has a reputation beyond repute
Re: Project ORB: A superb predictive scouting system!

The website is showing the "loading teams..." message for an awfully long time for me (it never loads the teams ). Should I be worried?

EDIT: Carver loads fine, for some reason, although there is still a delay in opening.
DOUBLEEDIT: Also, what do the numbers mean? Is it all relative measurement?
__________________
<Now accepting CAD requests and commissions>


Last edited by asid61 : 28-04-2016 at 00:46.
Reply With Quote
  #17   Spotlight this post!  
Unread 28-04-2016, 00:34
weaversam8 weaversam8 is offline
Team Captain
AKA: Sam Weaver
FRC #4534 (Wired Wizards)
Team Role: Leadership
 
Join Date: Jan 2014
Rookie Year: 2014
Location: Wilmington, North Carolina
Posts: 137
weaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to behold
Re: Project ORB: A superb predictive scouting system!

Quote:
Originally Posted by weaversam8 View Post
We noticed the error in the logs before you reported it- we're looking into it right now. It seems that we're missing data points for 3 out of 3144 teams, and one of them happens to be in Curie...
We have discovered that we're missing data for Teams 4, 8, and 11 for scaling. We're going to inject artificial data for those teams (just for that single value) tonight, and tomorrow update it with the most accurate data. That's all we can do tonight.
__________________
ORB Haow!

Reply With Quote
  #18   Spotlight this post!  
Unread 28-04-2016, 00:43
Kyler Hagler's Avatar
Kyler Hagler Kyler Hagler is offline
Jack of all trades
FRC #3310 (Black Hawk Robotics)
Team Role: Mentor
 
Join Date: Jan 2011
Rookie Year: 2011
Location: Rockwall-Heath High School
Posts: 99
Kyler Hagler has a spectacular aura aboutKyler Hagler has a spectacular aura aboutKyler Hagler has a spectacular aura about
Re: Project ORB: A superb predictive scouting system!

Looks like it is working now. Great job getting it up. Thanks for the work and dedication this took and allowing it to be usable for everyone.
__________________
Kyler Hagler - 3310 Alumni/Mentor
Black Hawk Robotics Team 3310

2012 Dallas West Regional Winner
2013 Dallas Regional Winner
2014 Colorado Regional Winner
2015 Hub City Regional Winner
2016 Arkansas Regional Winner
2016 Hub City Regional Winner

Http://Team3310.com
Reply With Quote
  #19   Spotlight this post!  
Unread 28-04-2016, 01:23
microbuns's Avatar
microbuns microbuns is offline
Registered User
AKA: Sam Maier
FRC #4917
Team Role: Mentor
 
Join Date: Jan 2015
Rookie Year: 2014
Location: Elmira
Posts: 81
microbuns is an unknown quantity at this point
Re: Project ORB: A superb predictive scouting system!

This is incredible. I love the fact that I can still pull up data for teams that are not at champs. The data looks pretty darn accurate. I have two questions:

1) Will these be updating as new matches are played during champs?
2) Could you release a top 50/100/3000? I'd be interested to see how your program ranks the robots, and compare it to OPR and my personal favorites. I could (and might) compile this myself, but it'd take a while, and you might have some way of doing it much more quickly.
Reply With Quote
  #20   Spotlight this post!  
Unread 28-04-2016, 06:22
GKrotkov's Avatar
GKrotkov GKrotkov is offline
Registered User
AKA: Gabriel Krotkov
FRC #1712 (Dawgma)
Team Role: Leadership
 
Join Date: Mar 2014
Rookie Year: 2014
Location: Ardmore, PA
Posts: 118
GKrotkov has much to be proud ofGKrotkov has much to be proud ofGKrotkov has much to be proud ofGKrotkov has much to be proud ofGKrotkov has much to be proud ofGKrotkov has much to be proud ofGKrotkov has much to be proud ofGKrotkov has much to be proud of
Re: Project ORB: A superb predictive scouting system!

Quote:
Originally Posted by BMSOTM View Post
This is really interesting; I can't wait to see how well it predicts at CMP. In the meantime, is it possible you could release its output for some of the other competitions this season? I'd really appreciate if you could release its evaluation of the teams at MAR CMP in particular so I can see how accurate the results are.
I second this like nobody's business. I would love to compare ORB with 1712's scouting data for MARCMP.
Reply With Quote
  #21   Spotlight this post!  
Unread 28-04-2016, 06:59
weaversam8 weaversam8 is offline
Team Captain
AKA: Sam Weaver
FRC #4534 (Wired Wizards)
Team Role: Leadership
 
Join Date: Jan 2014
Rookie Year: 2014
Location: Wilmington, North Carolina
Posts: 137
weaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to behold
Re: Project ORB: A superb predictive scouting system!

Good morning everybody!

Quote:
Originally Posted by microbuns View Post
This is incredible. I love the fact that I can still pull up data for teams that are not at champs. The data looks pretty darn accurate. I have two questions:

1) Will these be updating as new matches are played during champs?
2) Could you release a top 50/100/3000? I'd be interested to see how your program ranks the robots, and compare it to OPR and my personal favorites. I could (and might) compile this myself, but it'd take a while, and you might have some way of doing it much more quickly.
We do plan on updating based on matches being played, not sure if it's live this morning, but if it's not, it will be as soon as possible. Also, I'll take note of your request about the top lists, we'll see if we can do it. It might interest you to understand our match scoring system:

Match Scores
For match scores, our system takes the highest skill level of each individual defense and assumes that is the score of the robot crossing the defense, then it uses that for purposes of scoring. Then, it calculates in low and high goal shots, and then calculates the scale/challenge bonus.

Quote:
Originally Posted by BMSOTM View Post
This is really interesting; I can't wait to see how well it predicts at CMP. In the meantime, is it possible you could release its output for some of the other competitions this season? I'd really appreciate if you could release its evaluation of the teams at MAR CMP in particular so I can see how accurate the results are.
In regard to releasing previous data:
I can do a data dump of whatever teams you like, including the ones at the MAR Championship, but right now, data in the system will be skewed by the fact that the matches at that championship became part of the team's score, which means it could be less truly accurate. We might have the option to release some data from earlier in the season, but we'd have to dedicate training resources for that, which may be one of our lower priorities at the moment. We'll see what we can do.

Coming next:
  • We're looking into building a separate match ranking system that only takes this event into account, like how OPR individually examines an event.
  • Top X teams list? (You can already kindof do this in a division by sorting)
  • Slackbot to interface with the ORB API
  • Recommended defenses for a match view
  • And more...
__________________
ORB Haow!

Reply With Quote
  #22   Spotlight this post!  
Unread 28-04-2016, 09:53
kinganu123 kinganu123 is offline
Registered User
FRC #1747
Team Role: Mentor
 
Join Date: Jan 2010
Rookie Year: 2008
Location: Piscataway, NJ
Posts: 243
kinganu123 is on a distinguished road
Re: Project ORB: A superb predictive scouting system!

How is the data represented for the neural network? I was thinking about this the other day, and wasn't too sure how you'd account for individual teams without essentially combining the teams' stats into one "super team" (for a lack of a better phrase).
__________________
Reply With Quote
  #23   Spotlight this post!  
Unread 28-04-2016, 10:30
Eugene Fang's Avatar
Eugene Fang Eugene Fang is offline
The Blue Alliance
no team
Team Role: Alumni
 
Join Date: Jan 2007
Rookie Year: 2000
Location: Bay Area, CA -> Pittsburgh, PA
Posts: 769
Eugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond reputeEugene Fang has a reputation beyond repute
Re: Project ORB: A superb predictive scouting system!

I haven't looked at how accurate ORBs predictions have been, but I noticed that it is consistently overestimating scores by a large margin. Any insight as to why this might be occurring?
__________________
Eugene Fang
2010 Silicon Valley Regional Dean's List Finalist

Various FLL Teams - Student (2000-2006), Mentor (2007-2010)
FRC Team 604 - Student (2007-2010), Mentor/Remote Advisor (2011-2015)
FRC Team 1323 - Mentor/Remote Advisor (2011-2014)

The Blue Alliance | TBA GameDay | TBA Android App
Reply With Quote
  #24   Spotlight this post!  
Unread 28-04-2016, 10:38
CrazyMohawk's Avatar
CrazyMohawk CrazyMohawk is offline
Registered User
FRC #2471 (Team Mean Machine)
Team Role: College Student
 
Join Date: Dec 2012
Rookie Year: 2009
Location: Camas,WA
Posts: 42
CrazyMohawk is an unknown quantity at this point
Re: Project ORB: A superb predictive scouting system!

its down
__________________

Reply With Quote
  #25   Spotlight this post!  
Unread 28-04-2016, 10:41
danielrw danielrw is offline
Scouting App, Autonomous
FRC #4534 (Wired Wizards)
Team Role: Programmer
 
Join Date: Feb 2016
Rookie Year: 2013
Location: Wilmington, NC
Posts: 2
danielrw is an unknown quantity at this point
Re: Project ORB: A superb predictive scouting system!

Quote:
Originally Posted by CrazyMohawk View Post
its down
Thanks, we're working on getting it back up.
Reply With Quote
  #26   Spotlight this post!  
Unread 28-04-2016, 10:44
TomAwezome TomAwezome is offline
Robot Code Supervisor
FRC #4534 (Wired Wizards)
Team Role: Programmer
 
Join Date: Jan 2015
Rookie Year: 2015
Location: United States
Posts: 5
TomAwezome is an unknown quantity at this point
Re: Project ORB: A superb predictive scouting system!

Howdy!! I'm Tom, Wired Wizards' Alpha Team leader (Robot Code), and I as well have been doing the vast majority of the back end for ORB.

Quote:
Originally Posted by kinganu123 View Post
How is the data represented for the neural network? I was thinking about this the other day, and wasn't too sure how you'd account for individual teams without essentially combining the teams' stats into one "super team" (for a lack of a better phrase).
The data from the neural networks is saved into a database into tables for goals, defenses, challenge, and scale; goals has 4 values (autolow, autohigh, teleoplow, teleophigh), defenses has 9 values (0-8, 0:lowbar, 1ortcullis, 2:cdf, 3:moat, 4:ramparts, 5:drawbridge, 6:sallyport, 7:rockwall, 8:roughterrain), scale and challenge each have 1 value for their respective percentage. The goals values are representative of the quantity of goals, defenses representative of 0-2 crossings (as that is all TBA has per match), scale and challenge as mentioned are stored as a value between 0-1, a decimal percent. As each match is a combination of 3 teams per alliance, pulling the data gives you an idea of how they perform, but of course it is just a showing of how the alliance performs! The magic happens as the networks for each team analyze all of their different combinations, and finds the trends inside the dataset that would indicate the team it's training for. Good question!
Reply With Quote
  #27   Spotlight this post!  
Unread 28-04-2016, 11:05
danielrw danielrw is offline
Scouting App, Autonomous
FRC #4534 (Wired Wizards)
Team Role: Programmer
 
Join Date: Feb 2016
Rookie Year: 2013
Location: Wilmington, NC
Posts: 2
danielrw is an unknown quantity at this point
Re: Project ORB: A superb predictive scouting system!

Quote:
Originally Posted by danielrw View Post
Thanks, we're working on getting it back up.
We restarted the server and everything seems to be working now.
Reply With Quote
  #28   Spotlight this post!  
Unread 28-04-2016, 11:57
marshall's Avatar
marshall marshall is online now
My pants are louder than yours.
FRC #0900 (The Zebracorns)
Team Role: Mentor
 
Join Date: Jan 2012
Rookie Year: 2003
Location: North Carolina
Posts: 1,251
marshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond reputemarshall has a reputation beyond repute
Re: Project ORB: A superb predictive scouting system!

Source code?
__________________
"La mejor salsa del mundo es la hambre" - Miguel de Cervantes
"The future is unwritten" - Joe Strummer
"Simplify, then add lightness" - Colin Chapman
Reply With Quote
  #29   Spotlight this post!  
Unread 28-04-2016, 12:10
TomAwezome TomAwezome is offline
Robot Code Supervisor
FRC #4534 (Wired Wizards)
Team Role: Programmer
 
Join Date: Jan 2015
Rookie Year: 2015
Location: United States
Posts: 5
TomAwezome is an unknown quantity at this point
Re: Project ORB: A superb predictive scouting system!

Quote:
Originally Posted by Eugene Fang View Post
I haven't looked at how accurate ORBs predictions have been, but I noticed that it is consistently overestimating scores by a large margin. Any insight as to why this might be occurring?
As far as I could tell much earlier in development, it was oddly over in estimation, but it seemed consistent by a certain multiplier (not impacting the predictions for winning, just showing bigger values.) Before, I had multiplied it by about .7310 to get the scores looking more like actual match scores. It is currently not doing that, since as mentioned it has no impact on prediction. The score value is a result of how it's summing it all, as it is trying to follow as close as possible the way that match scoring works in-game, but is not caused by any errors.
Reply With Quote
  #30   Spotlight this post!  
Unread 28-04-2016, 13:50
weaversam8 weaversam8 is offline
Team Captain
AKA: Sam Weaver
FRC #4534 (Wired Wizards)
Team Role: Leadership
 
Join Date: Jan 2014
Rookie Year: 2014
Location: Wilmington, North Carolina
Posts: 137
weaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to beholdweaversam8 is a splendid one to behold
Re: Project ORB: A superb predictive scouting system!

Hope everybody's been having a great day! We have found that there are some issues with retraining teams as matches occur, so we've turned off that feature in order to ensure that the web service remains online. We'll train tonight for all teams at Championships to make sure the data tomorrow is fresh based on how teams played today.

Quote:
Originally Posted by TomAwezome View Post
As far as I could tell much earlier in development, it was oddly over in estimation, but it seemed consistent by a certain multiplier (not impacting the predictions for winning, just showing bigger values.) Before, I had multiplied it by about .7310 to get the scores looking more like actual match scores. It is currently not doing that, since as mentioned it has no impact on prediction. The score value is a result of how it's summing it all, as it is trying to follow as close as possible the way that match scoring works in-game, but is not caused by any errors.
We just did a tweak and they should be slightly more realistic, but we'll be looking into adjusting these values tonight to make them appear more like match scores.

Quote:
Originally Posted by marshall View Post
Source code?
We plan on releasing the source code at some point, it's a bit fragmented at the moment and we'll need to clean it up.
__________________
ORB Haow!

Reply With Quote
Reply


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 12:55.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi