A New Way to Scout

[orangemoore]

Originally Posted by BBray_T1296

While we're at it with the bluetooth stuff, why not make your scouting system managed by a NXT? They can interface with up to 4 devices at once (or many if you chain them) Sure using Lego would be less-than Ideal, but it sure would be funny to see someone using a NXT as a practical device

My team has access to over 20 NXTs that is a great Idea.

[SoftwareBug2.0]

Originally Posted by brennonbrimhall

Regarding publishing data giving away a well-deserved competitive advantage, I have a unique perspective. 20 collaboratively scouted all three of our events, and IRI. Giving data away doesn't magically make teams better -- the analysis of data does, as well as the understanding of what the numbers mean.

That being said, I recognize the predicament. Perhaps teams should consider publishing data after the event on a unified server, in a common format. Thoughts?

I also think this would be great. It would be fun to play around with even if teams waited till after the season.

As for a format, one year we had a scouting program that stored everything in text files in the form:
<match#> <team#> <property name> <value>

So you'd get lines that said something like:

15 2471 climb 10

[yash101]

Originally Posted by EricH

I don't know how current this is (and, BTW, use of a 2009 Getting Started guide might not be the greatest idea, given that we're dealing with 2013/14 here--the system has changed), but at least at one time, the field COULD run on 2.4 GHz. That was the theory, at any rate. IIRC, it was actually tried once, partly due to restrictions on the 5 GHz band at one particular event. Suffice it to say that the odds of all 6 robots on the field actually working were pretty close to nil for the first day or so of that event, and not all that much better later (though there was improvement thanks to the field staff working very hard).



Tl;dr: While the field nominally runs on 5 GHz, it (and the robots) can run on 2.4 GHz if needed, and they don't need any more potential interference there than is already there.

Yep. That kind-of doesn't seem too accurate because this is about the 2006? control system!

[yash101]

Originally Posted by SoftwareBug2.0

I also think this would be great. It would be fun to play around with even if teams waited till after the season.

As for a format, one year we had a scouting program that stored everything in text files in the form:
<match#> <team#> <property name> <value>

So you'd get lines that said something like:

15 2471 climb 10

So, with this data storage, are you using text-based database? If so:

15/2471/1

match/team#/climb code (0=no,1=10,2=20,3=30)
That would be a lot more readable to PHP and easier to implement. However, it would be important to note that text-based isn't so reliable. If this software is running for multiple teams, every time a match ends, the server will get a huge load. Multiple appends to the same file at the same time, many times, means a corrupt file. Instead, MySQL or some other database server built to store information may be the lifesaver!
The only time I used text databases is when I was learning PHP and using it on a site that maxes out at a view per day!

[brennonbrimhall]

Originally Posted by 1306scouting

As I mentioned (apologies for the double post, but this post covers different topics), 1306 has experience doing this, so I thought I should share to offer that viewpoint for anyone it can help:

1. It's really cool to watch teams work together on scouting.
   
2. We've designed most of the server-side application and are beginning to implement. Once I meet with the lead programmer, I'll see about open-sourcing it. We have dedicated server hardware for this, as well as some possible leads into getting redundancy and so forth.
   
   For the techies here, we're planning on using relational databases to store all the data so that we can efficiently pull the data we need while retaining large amounts of metadata (like the team that submitted and timestamps).
   
3. Mycroft Holmes method is fantastic, but it's not infallible - even if your person is amazing, they can be susceptible to biases and so forth. I like having a Mycroft Holmes-type person with data that we can tap into if we need. You are very lucky to have people that can do that.
   
4. In regards to privacy: 1306's general preference is to give all the raw data to everyone that participates and some "processed" data away for free (balanced by algorithms to create lists). 1306 has always given these lists away to rookies, but we think they're valuable to everyone without giving up any competitive advantage to participants.
   
   This is very open for discussion as we move into implementation and so forth. I personally think information should be free for everyone, but I can see why others would not agree.
   
5. We've designed the platform so that we have a CrowdScout API this year that allows us to take data from pretty much any electronic source. We use paper for our scouters (it's reliable and inexpensive, as well as not needing power and allowing the people scanning it in to see the data before it's uploaded to catch human errors immediately), but we should be able to support apps that implement the API as well as pretty much whatever else the community can dream up.
   
   The API is still young - I agree with brennonbrimhall about having different teams track different metrics, and agreeing on them on a per-tournament basis, but there are advantages to standardized metrics.
   
6. Bluetooth PANs work fairly well for localized internet sharing, and shouldn't have any interference issues with the field nor break any rules. That said, I highly recommend setting up ethernet networks if you can - bluetooth can have issues with many hosts; an old router and ethernet will almost always outperform bluetooth.
   
7. Qualitative data is incredibly important, and because every team functions differently, it's hard to share. CrowdScout could be the platform for doing so, if needed, but I've always thought a team-by-team basis was better for that.

I'm very interested in your platform; 20 has been working on our own solution called Gemini. It too isn't designed to replace paper scouting, but to aggregate and calculate statistics. We've also scouted with other teams at WPI, Connecticut, Champs, and IRI, so our system was designed with that in mind. Once the finishing touches get finalized, I'll post the codebase here.

[Kimmeh]

Originally Posted by antimatter_john

Exactly what? There was no exactly to my question. My question is is why is IT SO IMPORTANT TO KEEP RAW DATA KEPT AWAY FOR ONLY CERTAIN TEAMS. (No yelling inteded just want to get my point across). There is no point. Yuo even said your self



If I am missing something please tell me.

My apologies. I was actually agreeing with you. I see no point in keeping raw data away from teams.



Now, I offer a more broad question to those not interested in waiting to make the information public.

As a community, we see nothing wrong with helping a team make their robot more competative at an event. It's considered a good thing. It helps lift the floor of the competition, right? We share all our "best practices" in hopes that it makes others better.

For those of you against this, or even just suggesting we wait until after the competition: why? I don't understand why you have a problem with it, and I'm trying to understand. In my mind, I see it the same way as helping a robot. We're trying to make teams better.

Perhaps I'm understanding this incorrectly (or others are), the goal would be to share the raw data, correct? We're not asking anyone to share how they manipulate the data. Just work together to get the data and let teams do whatever they choose with it. Correct?

[brennonbrimhall]

Originally Posted by Kimmeh

For those of you against this, or even just suggesting we wait until after the competition: why? I don't understand why you have a problem with it, and I'm trying to understand. In my mind, I see it the same way as helping a robot. We're trying to make teams better.

I agree; that's why we've scouted collaboratively before. Not only does it share data that would be hard for smaller, more challenged teams to get, but it also develops great inter-team relationships.

However, not all teams share that view, and that's perfectly fine.

Originally Posted by Kimmeh

Perhaps I'm understanding this incorrectly (or others are), the goal would be to share the raw data, correct? We're not asking anyone to share how they manipulate the data. Just work together to get the data and let teams do whatever they choose with it. Correct?

I think that's a topic that deserves more discussion. It seems we have three platforms that have been brought up, and if a unified system is to be achieved, we should discuss how that's implemented.

[khanh111]

There are arguments to be made for both sides.

As of now, few enough teams are able to collect good data that good data is itself a competitive advantage. That's why I originally thought that data should be kept within teams of the scouting alliance.

In any case, data will eventually become so widespread that good data will be everywhere (think stock market data). When that comes, data analysis (which is already extremely important) will become even more important.

It's up to us as a community to decide when that will be though. I liked 1306scouting's idea of giving out separate lists to teams pause the scouting alliance.

[XaulZan11]

Originally Posted by khanh111

If anything, I think the "Mycroft Holmes" method should be supplemented by some form of collected hard data.

I agree 100%. There is no substitute for hard data, but having someone that actually remembers what the robots look like, how they play,what they do in a match and so on is very benefical. At events for my team, I watch one or two teams a match and write qualitative notes down. At the end of the event I know the basics of every team (and a lot about certain teams) and could make a pretty good pick list. But, I'm always surprised when going through the data to see how well or how poorly some teams actually do (consistency, especially in autonomous, is typically the biggest source for deviation). It simply comes down to not being able to watch every match for every team.

[Nemo]

On the topic of keeping data private versus sharing it with the world:

It's a huge waste of time for every team to dedicate 7+ people to compiling the same exact set of statistics. If a community effort produced the same thing with far less total effort, the teams with the most effective scouting systems could instead put that effort into creating more advanced statistics and improving their qualitative scouting and their methods of collecting and using that qualitative data. Those teams would still have a competitive advantage to show for their efforts.

[khanh111]

Originally Posted by Nemo

On the topic of keeping data private versus sharing it with the world:

It's a huge waste of time for every team to dedicate 7+ people to compiling the same exact set of statistics. If a community effort produced the same thing with far less total effort, the teams with the most effective scouting systems could instead put that effort into creating more advanced statistics and improving their qualitative scouting and their methods of collecting and using that qualitative data. Those teams would still have a competitive advantage to show for their efforts.

I agree that it can be a waste to have 7+ people constantly looking for the same set of statistics. However, some larger teams use that as a way to help teach newer members about scouting/strategy. Every experience helps a younger member.

[Connerd]

Originally Posted by khanh111

However, some larger teams use that as a way to help teach newer members about scouting/strategy. Every experience helps a younger member.

That is what FRC is about, right? Anyway, I wouldn't see a problem if raw data was released, and not the analysis, interpretation, or anything similar. While we might think the game is the primary focus, we're all here to learn.
The only problem with releasing data, is that it'd be hard to get all the data world-wide in a real-time format, with enough time for choosing good alliance partners.
But those are the musings of a 1-year veteran. Shared data would help a lot of teams, esp. those with small or non-existant scouting teams.

[Roger]

Originally Posted by khanh111

If anything, I think the "Mycroft Holmes" method should be supplemented by some form of collected hard data.

Well, if you can't believe Mycroft, who can you trust? (You're just lucky Mycroft is too lazy to prove he's right.)

It's also true that not every team has a "Mycroft", so alternatives have to be made anyway. Our team is also developing a "crowd" type scouting system, which does seem the way to do it -- the more data in, the better. Though in general two people can always have opposite conclusions for the same bit of data.

[Nemo]

Originally Posted by khanh111

I agree that it can be a waste to have 7+ people constantly looking for the same set of statistics. However, some larger teams use that as a way to help teach newer members about scouting/strategy. Every experience helps a younger member.

It's a challenging task that students can learn from, true. But it's still duplicating the same work dozens of times at each event, and that carries an opportunity cost. Think of all of those hours of work put to use in other ways. I think the product on the field would improve.

I think we'll eventually have a system that makes stats from every event freely available, much like results and standings are available from FRC Spyder / TheBlueAlliance / usfirst.org / etc. When that happens, we'll still see people developing their own scouting systems to gather, aggregate, analyze, and report on additional types of scouting data.

[Andrew Schreiber]

Originally Posted by yash101

Yep. That kind-of doesn't seem too accurate because this is about the 2006? control system!

No, all of Erich's information is about the current-ish control system. The 2006 control system was based on a 900mhz system. In other words, what are you talking about?


As for the questions earlier on why you can't run 2.4Ghz at low power - Simple, it's easier to say "No Wifi" than it is to say "No wifi except in this situation" because then you have to police that. It's more work for an already over worked group of folks.

Originally Posted by Kimmeh

Perhaps I'm understanding this incorrectly (or others are), the goal would be to share the raw data, correct? We're not asking anyone to share how they manipulate the data. Just work together to get the data and let teams do whatever they choose with it. Correct?

Nah, I think we should be sharing how we manipulate the data. Data manipulation and visualization is as much engineering as learning CAD even if it is relegated to the back rooms of scouting meetings in FRC. Course, I'm biased…