Web-based scouting program

[Nibbles]

This is a scouting program I have been working on for the last year. It is fully tested and mature, it does statistics, match records, handles multiple events over multiple years (you can switch the database at least), and multiple people can work at the same time, with multiple devices (with a router or wireless router).
It is a PHP program that uses any CGI web server (I use apache2 and mod_php) and MySQL on Gentoo Linux (with no GUI or X11 to keep it fast, MySQL LOVES memory). I have been keeping version control with Git for only the last three months.
I have dedicated much focus to a great user interface that is simple and quick for stuff that needs to be fast (entering match data, looking up teams).

The temporary demo is online (probably) from the laptop at http://labs.redjacket.ws/. Please be kind to it, I haven't put it through any sort of stress testing yet. Select a database first, there are five databases with actual data (frc_2007_whatever), and a sixth with all combined (frc_everything_2007, disabled for now).

A few features: Robot pictures can be viewed (and uploaded), SQL backups, match predicting can be done for all matches (future and completed) by clicking the "MP" link below the table on a match page (it predicts the correct team 90% of the time, and also gives a % certainty), a strength algorithm that shows how much a team contributes to their alliance's score (0 means average for that event, 1000 means 10 points roughly?), search for teams across all databases by number, official name/sponsor (rookie teams), or nickname from the front page. The "query" page lets you execute real SQL (SELECT statements on databases starting with "frc_" only, haha) across the database. Very useful if you want refined statistics. You can sort teams by the official FIRST ranking or my own strength algorithm clicking on the headers on the team list.

I know it is great, please don't waste a post to say that
Please do post if you want to use it at a regional or suggest a feature. I want to have all match data for every regional entered this year, if you are able to record and upload all stats (penalties, per-team scores which there may be with live scoring this year, etc), please make that available.

If you want the source, you will need Git for now: git clone http://labs.redjacket.ws/tools/scouting.git/
If you just want to view the source, try Git-php, linked to on the front page.

[nate118]

Do you think it would be against the spirt of First to try to sell licenses to a scouting program?

[MARS-CJ]

Would there be a way to sort a list of all the teams in the database by certain categories? For example, this year it would probably be useful to see a team's average number of hurdles per match when choosing final alliances. It would make it easier if the program could list them from greatest to least.

[Nibbles]

Originally Posted by nate118

Do you think it would be against the spirt of First to try to sell licenses to a scouting program?

Against the spirit of first to sell licenses? I don't think so, I know Carl Hayden/Falcon Robotics, FRC 842, is selling a networked scouting device to teams, I don't have the link right in front of me but it is somewhere on CD. Granted, it is a physical device, not just software.

I just added the GPL v3 to the project, so it shouldn't matter for me.

Originally Posted by MARS-CJ

Would there be a way to sort a list of all the teams in the database by certain categories? For example, this year it would probably be useful to see a team's average number of hurdles per match when choosing final alliances. It would make it easier if the program could list them from greatest to least.

That is a great idea! There is a show/hide columns link on top of the teams table, an "add a column" drop down box would work, or a number of check boxes that are unchecked by default. I will try a few different methods, see what works best.

[Nibbles]

I posted files with basic install directions at http://labs.redjacket.ws/files/releases/

Install information is in docs/install.txt, you just need a web server with PHP and MySQL (Windows, GNU/Linux, Mac, doesn't matter).
PLEASE do not hesitate PM me if you have trouble setting it up. You can also catch me on IRC at server irc.freenode.net channel #firstrobotics user AAA_awright, if you know how to use IRC.

[proegssilb]

After build season, I'll look at this a little closer. It holds a lot of promise.

I think there's one thing missing, though. My team's primary use for scouting isn't ranking teams, it's providing info on how to approach each match. To that end, a printable and concise page containing each team, that team's notes, a picture, and the most important stats for that year is a good tool to have.

[Nibbles]

I put great thought into the team pages, it was one of the first things I worked on. All the pages are printer friendly, the menus are automaticly hidden, for example, amoung other things that are non-interactive. You can upload photos of the team's robot, and put in notes and enter data on a team.
Sorry there are no examples of robot photos or how to edit the team (I am still working on permissions, so I can't give out an demp login username/password for now).

[SteveGPage]

Originally Posted by Nibbles

... (cut) match predicting can be done for all matches (future and completed) by clicking the "MP" link below the table on a match page (it predicts the correct team 90% of the time, and also gives a % certainty), a strength algorithm that shows how much a team contributes to their alliance's score (0 means average for that event, 1000 means 10 points roughly?), (cut) ....

Can you share the algorithm that you used to do the match predictions? Thanks!

Best regards,

Steve

[Nibbles]

Originally Posted by SteveGPage

Can you share the algorithm that you used to do the match predictions? Thanks!
Best regards,
Steve

For the strength of team T, find all the matches played by team T (set B), find all the teams they played against or with (set C), and all the matches they played (set D), then find the average of their score minus opponents score (set E). Subtract the team T's average match score minus opponents score.

I have refined it several times over the last year, but that is how it works more or less. It also assumes that each team has an equal chance of competing against each other, which was not the case, as rookie teams always played with better teams boosting their score, and in a few cases the same alliance played together, It still fared very good however.

Let me try explaining another way: It takes into account the scores of your opponents and allies.

This is the exact SQL query used to find the strength for each team:

SELECT
  ROUND(AVG(IF(b.teamnum=c.teamnum ,2,-1)*IF(d.alliance="red",e.redscore-e.bluescore,e.bluescore-e.redscore)*IF(b.alliance=c.alliance,1,-1))*100) AS score2
FROM team_match b JOIN team_match c JOIN team_match d JOIN matches e
ON (b.matchid=c.matchid)
  AND (d.teamnum=c.teamnum AND d.matchid!=b.matchid)
  AND (d.matchid=e.matchid)
GROUP BY q.teamnum

This is in pages/strength.act.strength_2.php .

This is how I get the team strengths. Match predicting is a new area for me, I implemented it just a few month ago.
For matches, I found that the sum of the teams strengths on the target alliance, minus the sum of the opponent's alliance, works well enough.
I can calculate the chance a team will win with a cumulative logistic distribution, who's parent function is f(x) = 1/(1+e^(-x)).

My most recent tests show it is only 80% to 85% accurate, based on how I add the strengths together (add the square roots, etc). I don't really care too much, it isn't the most important part of the program.

[sebas2mil]

I am looking at it for 1251 as a scouting system

we are attending the florida regional and the palmetto regional

If it is useful at florida we will deff use it at palmetto

The only thing is that I'm having trouble setting it up

could I use access 2007

would that do the same thing as using that other software you wanted to use

thanks a lot

sebas

[Nibbles]

I'll summarize the PM I replied to for everyone else:
There is no reason Access couldn't work, however I can't vouch for it's standards support, and I have no clue how to set it up. It is possible though, since I use a database library (ADODB) that supports many types of databases.
I think installing a web server package such as XAMPP or WAMP, which contains Apache and MySQL, is much easier to set up and use.

[synth3tk]

Originally Posted by Nibbles

I'll summarize the PM I replied to for everyone else:
There is no reason Access couldn't work, however I can't vouch for it's standards support, and I have no clue how to set it up. It is possible though, since I use a database library (ADODB) that supports many types of databases.
I think installing a web server package such as XAMPP or WAMP, which contains Apache and MySQL, is much easier to set up and use.

I'm using XAMPP for Windows to build a network, so I'll try it out and see what happens. I've been looking for a good scouting database to integrate into our intranet. Hopefully it'll work out, I'll let you know.

[SteveGPage]

Originally Posted by Nibbles

I have refined it several times over the last year, but that is how it works more or less. It also assumes that each team has an equal chance of competing against each other, which was not the case, as rookie teams always played with better teams boosting their score, and in a few cases the same alliance played together, It still fared very good however.

Thanks! I will certainly incorporate this approach into how I do scouting! Thanks for sharing this!

Good luck this season!

Steve

[Nibbles]

I typed up and posted a fairly descriptive tutorial online:
http://docs.google.com/View?docid=dcz67k4q_31gk8m5rhw
As usual, if anyone has any problems, feel free to contact me with whatever means you feel appropriate (PM or IRC works best).
If anyone is interested in helping to edit it, I can add you as an editor if you have a Google account (which is trivial to create) or Gmail, Private Message me with your email.

[David Bliss]

I have created, with help, an alternate data input to this application. This one focuses on getting data in the database "live"-- as quickly as possible from the stands as the competitions are happening. There are two components, (1) a VB application that the user works with to generate data, and (2) a python 2.5 and pymysql script that commits that data to the database. In the zip so far are two sample INI files needed to power the system, and the Python script to commit data. I know the code is a bit sloppy, and there's probably a few bugs, but it seems to work pretty well so far, so I thought I'd post something to let anyone interested know what I was up to.

tournament.ini is generated by a to-be-written script, and the other INI by the VB app. The python script in the ZIP takes the INI to commit as the only command-line argument, and prompts for the MySQL username/password.

Enjoy. If you want windows binaries, those will take a couple days.