FIRST Event Data in XML format

[rsisk]

After parsing First web pages myself for the Regional Twitter Accounts, I now have a newly found respect for what the team at The Blue Alliance has done to gather data

Seemed like every week was a scramble to adapt to something new. And then when I found out that Einstein's data was not posted real time, well I put the NASA feed projected on a wall at home and posted match scores by hand.

What a joy it would be if First offered some way to get to this information besides parsing their web sites. Not 100% sure what I would expect, maybe a web service that made the data available?

The FMSFRC twitter feed came close to offering some data in a real time feed format, and maybe that is the answer. But now it is tough to go back and scrape all that data from twitter pages if you didn't get it during the realtime feed.

[Andrew Schreiber]

Originally Posted by rsisk

What a joy it would be if First offered some way to get to this information besides parsing their web sites. Not 100% sure what I would expect, maybe a web service that made the data available?

The FMSFRC twitter feed came close to offering some data in a real time feed format, and maybe that is the answer. But now it is tough to go back and scrape all that data from twitter pages if you didn't get it during the realtime feed.

Richard, I will be making the entire FRCFMS feed (That twitter still lets me grab) available as soon as I get time to do it. I have a script written to do it. If someone would like to run it I can give instructions on doing it.

Yes FIRST would make all of our lives simpler if they would find a standard and stick to it. Either let us have an API we can make calls to (Published well before kickoff) or at least have a standardized page layout and don't change it without warning us and telling us about the changes. One of the additional reasons for this project is that we have a STANDARD way of accessing data.

If anyone would like to offer assistance feel free to shoot me a PM.

[Nate Smith]

Originally Posted by Andrew Schreiber

Yes FIRST would make all of our lives simpler if they would find a standard and stick to it. Either let us have an API we can make calls to (Published well before kickoff) or at least have a standardized page layout and don't change it without warning us and telling us about the changes. One of the additional reasons for this project is that we have a STANDARD way of accessing data.

On my to-do list depending on how things go....can't say much more than that right now...I might be able to share more info in PM, if you ask nicely...

[Andrew Schreiber]

Nate, I was just grumbling. Im already providing XML information for a couple of the pages and am working on the others.

As an update:

http://schreiaj.ath.cx/share/FRC_Par...alschedule.php will provide the qualification schedules for the regionals that are not bizarre.

http://schreiaj.ath.cx/share/FRC_Parsers/ranking.php will provide the qualification ranking data.

Both pages take the following options:

Event_Code - Event Code from frclinks.com. Since I now use frclinks to find the pages the exact codes given on there are what need to be used.

Year - 2008, 2009 are currently supported.

HTML_Display [true,false] - This decides whether to escape the tags so that they display in the browser. If you are parsing the xml in a script I would suggest leaving this false (or blank). If you plan on copying and pasting the xml anywhere from the browser use true.

Currently I am working on parsing the team history pages and will post that as soon as I am done.

[Greg Marra]

Originally Posted by Andrew Schreiber

http://schreiaj.ath.cx/share/FRC_Par...alschedule.php will provide the qualification schedules for the regionals that are not bizarre.

I believe that bizarre page formatting comes from the pages being opened in Microsoft Word, edited, and saved again. Some result pages are full of Word HTML markup.

[Andrew Schreiber]

Originally Posted by Greg Marra

I believe that bizarre page formatting comes from the pages being opened in Microsoft Word, edited, and saved again. Some result pages are full of Word HTML markup.

Don't get me started on Word and HTML

On an unrelated note, in the spirit of open source all the code is available http://schreiaj.ath.cx/share/FRC_Parsers/ and the current versions I am working on at the moment are at http://schreiaj.ath.cx/share/FRC_Parsers/Parsers_Beta/

[Nibbles]

Where did you find frclinks.com? That's a nifty idea.

I don't know how well one parsing method works against others, a regex will work as long as they don't add a new table to the document, and don't add non-numerical data. Likewise, an HTML parser will, and will also properly handle entities like < , but any change in structure will not work (though that is a simple parameter change telling it the new path to the data). I just use the DOM and SimpleXML parsers in PHP, Python (eewww, Python) must have something similar.

I have an initiative to standardize how FIRST data is published, XML Interchange format. An example that mixes the rankings and schedule:

<event season="2009" code="GLR">
        <team number="67" game:rank="1" game:win="12" game:lost="0" game:tie="0" game:plays="0" game:qs="24.00" game:rs="51.75" game:mp="117" />
        ...

        <match type="qualification" number="1" time="11:45">
            <alliance name="red">
                <team position="1" number="1940"/>
                <team position="2" number="216"/>
                <team position="3" number="123"/>
            </alliance>
            <alliance name="blue">
                <team position="1" number="1896"/>
                <team position="2" number="468"/>
                <team position="3" number="894"/>
            </alliance>
        </match>
        ...
</event>

Where game is some XML namespace (if you like).

As for licensing, as a rule of thumb, if the code is shorter then the license would be, I put it in public domain.