|
|
|
![]() |
|
|||||||
|
||||||||
![]() |
| Thread Tools | Rate Thread | Display Modes |
|
#16
|
|||
|
|||
|
Re: FRC Event Data in XML Format
Minor changes;
Set the cache time to 1 minute. I really like Dave's idea and will look at implementing that next time I get a chance to really muck with it. I also defaulted the year to 2009, come January I need to remember to default it to 2010. There is also now some text at the landing page telling you the options. This might get a little nicer over time but is by no means a priority. |
|
#17
|
|||||
|
|||||
|
Re: FRC Event Data in XML Format
Just a thought...since you're looking to transition it in January, why not code it to use the current year as the default?
|
|
#18
|
|||||
|
|||||
|
Re: FRC Event Data in XML Format
http://frcfeed.appspot.com/2009/ct/q...ations/results
Code:
Traceback (most recent call last):
File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 507, in __call__
handler.get(*groups)
File "/base/data/home/apps/frcfeed/3.337367731534012182/FRCFeed.py", line 160, in get
self.get_results(request)
File "/base/data/home/apps/frcfeed/3.337367731534012182/FRCFeed.py", line 129, in get_results
self.response.out.write(toXML(self,data[request["When"].capitalize()+' Matches'],'Match'))
KeyError: 'Qualifications Matches'
Would it be possible to produce JSON instead of XML? It's easier to parse in many situations. [s]Also, how frequently is this data refreshed? Do you generate it live off FIRST's servers, or are you caching it?[/s] I see there's a one minute cache time. Thanks! -Greg Last edited by Greg Marra : 29-10-2009 at 17:21. |
|
#19
|
|||
|
|||
|
Re: FRC Event Data in XML Format
Quote:
Here is my gift for you http://frcfeed.appspot.com/2009/detroit/rankings/json The request system has been made a little more robust. It still doesn't report what you are missing but it will dump you to the instruction page. You can request it in xml, json, or human. Human just makes it easier to read for a human, best way is to just try it out and see yourself. It really shouldn't be used for stuff, it is mostly for debugging on my part. If you find any bugs just let me know. Just for reference, the URL is now: frcfeed.appspot.com/{year/}event/{qualification or elimination/}what{/format} Last edited by Andrew Schreiber : 29-10-2009 at 22:08. |
|
#20
|
|||||
|
|||||
|
Re: FRC Event Data in XML Format
Quote:
|
|
#21
|
|||
|
|||
|
Re: FRC Event Data in XML Format
If their server supports it, you could turn the cache down to one or two minutes and send an If-Modified-Since header, in which case the server will tell you if nothing has changed and not send a response. In addition, for games in years past, why not increase the cache to an hour or even a day? The score probably won't change.
A few notes on syntax, looking at /2009/Greater Toronto/awards: Spaces are not allowed in tag names - it is parsed as an attribute without a value. There is a leading nonbreaking space ( ) in many of the tag values (Home town, name, team). is not a predefined entity, you will need a DTD declaration if you want to define it. Or just replace them with a standard space. It looks like FIRST doesn't use any entity other than , but you will need to be careful if in the future they do, that has to get parsed correctly. The ampersands need to be escaped, too, into "&" . As for URL parsing, are you using a regular expression? Code:
(/(\d{4}))?(/(.+))(/(qualification|elimination))?(/(awards|schedule|rankings|results))(/(json|xml|human))?
|
|
#22
|
|||
|
|||
|
Re: FRC Event Data in XML Format
Quote:
I will get to doing the escaping stuff today, I'll replace the spaces in the tag name with a _ and pull out the characters. I will also take care of ampersand and quotes while I am at it. Thanks for your comments. |
|
#23
|
|||
|
|||
|
Re: FRC Event Data in XML Format
Further proving I have no life, I have made the changes to the XML document to replace the spaces with underscores.
Just for fun I also added the ability to slice the results. Somewhere after specifying the event you can add a two numbers with a - in between (1-8 for example) as an element in the request and it will only return elements 1-8. This doesn't work on Awards, but that wouldn't make any sense anyway. For example this new feature can be seen at http://frcfeed.appspot.com/dt/rankings/1-8/human/ which will return the top 8 teams in Detroit. Enjoy. Edit: You don't have to specify the beginning OR the end, you can use -8 to represent the top 8 or you could use 30- to return the bottom ten. Anyone who has ever used Python slices should be familiar with this concept. |
|
#24
|
||||
|
||||
|
Re: FRC Event Data in XML Format
I like the "human" format idea (a lot) for quick data reference. But, since the data is already in an XML format, would it be possible to make the "human" page list the data in a table (with the XML tags as column titles, and such) instead of just the plain-text XML? It's not a crucial addition, but an idea nonetheless.
Last edited by SamC : 01-11-2009 at 03:16. |
|
#25
|
|||||
|
|||||
|
Re: FRC Event Data in XML Format
It also occurs to me that releasing the source code to this project would probably be highly beneficial for the FIRST community.
Any chance of a Google Code project so that people can check the code out themselves and play with it? Hopefully you'll even get improvements contributed back! [edit] Also, any chance of getting "attending teams" on here? [/edit] Last edited by Greg Marra : 31-10-2009 at 15:33. |
|
#26
|
|||
|
|||
|
Re: FRC Event Data in XML Format
Quote:
I will take a look at grabbing attending teams. Just throwing the parser at it didn't work but I think I might be able to make it work by tweaking some of the options but I haven't done that yet. Again, I'll let you know. As for making the data tabular, using FIRST's own site is probably a better use for that. |
|
#27
|
|||
|
|||
|
Re: FRC Event Data in XML Format
The source code is now available via subversion at http://code.google.com/p/frcfeed/ If you are interested in making changes let me know and we can work something out. I apologize that the code is as sloppy looking as it is I am working on cleaning it up and adding more comments.
A word on the eval(...) statements, I felt this would be the easiest way of handling multiple formats or multiple new options. In the next couple days I will try to put a wiki page together on adding new formats and new options. http://code.google.com/p/frcfeed/wiki/HowToUse Link to Wiki on how to use. Expect this to be fleshed out more over the next couple days. If you find parts that are hard to read just let me know and I will try to reword them. Last edited by Andrew Schreiber : 03-11-2009 at 15:53. |
|
#28
|
|||
|
|||
|
Re: FRC Event Data in XML Format
There have been some minor updates:
The request can come in any order now, as long as you specify an What and an Event and the When where applicable it will do its best to figure out what you meant. Human format has been tweaked to be tabular. XML option should be a little more standard. I am now using an XML generator instead of just string concatenation. Got rid of all evals in the python code. I also cleaned up the code to use one function in the place of 4. Also, as a small project I decided to put together some a Javascript object for getting data. I forgot how ugly Javascript's object stuff is. I have included it below. Code:
function Event(eventCode,year){
this.eventCode = eventCode;
this.year = year;
}
Event.prototype.getRankings = function(){
myEvent = this
url = "http://frcfeed.appspot.com/"+myEvent.year+"/"+myEvent.eventCode+"/rankings/json"
new AJAX_Request(url,function(){myEvent.Rankings = eval("("+xmlhttp.responseText+")")},myEvent)
return myEvent
}
Event.prototype.getAwards = function(){
myEvent = this
url = "http://frcfeed.appspot.com/"+myEvent.year+"/"+myEvent.eventCode+"/awards/json"
new AJAX_Request(url,function(){myEvent.Awards = eval("("+xmlhttp.responseText+")")},myEvent)
return myEvent
}
Event.prototype.getQualificationSchedule = function(){
myEvent = this
url = "http://frcfeed.appspot.com/"+myEvent.year+"/"+myEvent.eventCode+"/qualification/schedule/json"
new AJAX_Request(url,function(){myEvent.Qualification_Schedule = eval("("+xmlhttp.responseText+")")},myEvent)
return myEvent
}
Event.prototype.getEliminationSchedule = function(){
myEvent = this
url = "http://frcfeed.appspot.com/"+myEvent.year+"/"+myEvent.eventCode+"/elimination/schedule/json"
new AJAX_Request(url,function(){myEvent.Elimination_Schedule = eval("("+xmlhttp.responseText+")")},myEvent)
return myEvent
}
Event.prototype.getQualificationResults = function(){
myEvent = this
url = "http://frcfeed.appspot.com/"+myEvent.year+"/"+myEvent.eventCode+"/qualification/results/json"
new AJAX_Request(url,function(){myEvent.Qualification_Results = eval("("+xmlhttp.responseText+")")},myEvent)
return myEvent
}
Event.prototype.getEliminationResults = function(){
myEvent = this
url = "http://frcfeed.appspot.com/"+myEvent.year+"/"+myEvent.eventCode+"/elimination/results/json"
new AJAX_Request(url,function(){myEvent.Elimination_Results = eval("("+xmlhttp.responseText+")")},myEvent)
return myEvent
}
// Utility Functions
function AJAX_Request(url,handlerFunction,newEvent){
if(window.XMLHttpRequest){
xmlhttp = new XMLHttpRequest();
}
else{
xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
}
url = url
xmlhttp.onreadystatechange=function() {
if (xmlhttp.readyState==4) {
handlerFunction(newEvent)
}
}
xmlhttp.open("GET",url,true);
xmlhttp.send(null);
}
Code:
kettering = new Event("Kettering",2009);
kettering.getAwards();
kettering.Awards["Regional Winner #1"]["Team"]
|
|
#29
|
|||
|
|||
|
Re: FRC Event Data in XML Format
Due to the fact that I continue to have little to no life I started a complete rewrite of the parser. It currently handles:
I don't have any exact numbers because I haven't figured it out yet but the new parser seems to run significantly faster. Before I wrap it up and release it I am asking for some feedback, What additional pages would you like to see? Formats? Options? |
|
#30
|
|||
|
|||
|
Re: FRC Event Data in XML Format
Well, I have placed the new version out a day later than I wanted to but it is out. I added Team_List, Team_Info and Team_History as options. Team_List requires an event and an optional year (year currently defaults to the current year) Team_Info and Team_History require a team number.
So, during testing I decided to see what events had errors. The results are below. Code:
mn-rankings: Good mn-team_list: Good mn-awards: Error mn-results: Good mn-schedule: Good sdc-rankings: Good sdc-team_list: Error sdc-awards: Good sdc-results: Good sdc-schedule: Good in-rankings: Good in-team_list: Good in-awards: Error in-results: Good in-schedule: Good roc-rankings: Good roc-team_list: Good roc-awards: Good roc-results: Good roc-schedule: Good gg-rankings: Good gg-team_list: Good gg-awards: Good gg-results: Good gg-schedule: Good oc-rankings: Good oc-team_list: Good oc-awards: Good oc-results: Good oc-schedule: Good gt-rankings: Good gt-team_list: Good gt-awards: Good gt-results: Error gt-schedule: Error ga-rankings: Good ga-team_list: Good ga-awards: Good ga-results: Good ga-schedule: Good gl-rankings: Good gl-team_list: Good gl-awards: Error gl-results: Good gl-schedule: Good nh-rankings: Good nh-team_list: Good nh-awards: Good nh-results: Good nh-schedule: Good cur-rankings: Good cur-team_list: Good cur-awards: Good cur-results: Good cur-schedule: Good tx-rankings: Good tx-team_list: Good tx-awards: Good tx-results: Good tx-schedule: Good la-rankings: Good la-team_list: Good la-awards: Error la-results: Good la-schedule: Good li-rankings: Good li-team_list: Good li-awards: Good li-results: Good li-schedule: Good or-rankings: Good or-team_list: Good or-awards: Good or-results: Good or-schedule: Good il-rankings: Good il-team_list: Good il-awards: Good il-results: Good il-schedule: Good new-rankings: Good new-team_list: Good new-awards: Good new-results: Good new-schedule: Good ny-rankings: Good ny-team_list: Good ny-awards: Error ny-results: Good ny-schedule: Good mn2-rankings: Good mn2-team_list: Good mn2-awards: Good mn2-results: Good mn2-schedule: Good ut-rankings: Error ut-team_list: Error ut-awards: Error ut-results: Error ut-schedule: Error va-rankings: Good va-team_list: Good va-awards: Good va-results: Good va-schedule: Good da-rankings: Error da-team_list: Good da-awards: Error da-results: Error da-schedule: Error dt-rankings: Good dt-team_list: Good dt-awards: Good dt-results: Good dt-schedule: Good mo-rankings: Good mo-team_list: Good mo-awards: Good mo-results: Good mo-schedule: Good dt1-rankings: Good dt1-team_list: Good dt1-awards: Good dt1-results: Good dt1-schedule: Error md-rankings: Error md-team_list: Error md-awards: Error md-results: Error md-schedule: Error hi-rankings: Good hi-team_list: Good hi-awards: Good hi-results: Good hi-schedule: Good dt-rankings: Good dt-team_list: Good dt-awards: Good dt-results: Good dt-schedule: Good is-rankings: Good is-team_list: Good is-awards: Good is-results: Good is-schedule: Error cmp-rankings: Error cmp-team_list: Good cmp-awards: Good cmp-results: Good cmp-schedule: Good arc-rankings: Good arc-team_list: Good arc-awards: Good arc-results: Good arc-schedule: Good wi-rankings: Good wi-team_list: Good wi-awards: Good wi-results: Good wi-schedule: Good ok-rankings: Good ok-team_list: Good ok-awards: Good ok-results: Good ok-schedule: Good wor-rankings: Good wor-team_list: Error wor-awards: Good wor-results: Good wor-schedule: Good tx-rankings: Good tx-team_list: Good tx-awards: Good tx-results: Good tx-schedule: Good new-rankings: Good new-team_list: Good new-awards: Good new-results: Good new-schedule: Good mn2-rankings: Good mn2-team_list: Good mn2-awards: Good mn2-results: Good mn2-schedule: Good gl-rankings: Good gl-team_list: Good gl-awards: Error gl-results: Good gl-schedule: Good az-rankings: Good az-team_list: Good az-awards: Good az-results: Good az-schedule: Good wc-rankings: Error wc-team_list: Error wc-awards: Error wc-results: Error wc-schedule: Error wa-rankings: Good wa-team_list: Error wa-awards: Error wa-results: Good wa-schedule: Good wi-rankings: Good wi-team_list: Good wi-awards: Good wi-results: Good wi-schedule: Good li-rankings: Good li-team_list: Good li-awards: Good li-results: Good li-schedule: Good gal-rankings: Good gal-team_list: Good gal-awards: Good gal-results: Good gal-schedule: Good on-rankings: Good on-team_list: Error on-awards: Error on-results: Good on-schedule: Good ok-rankings: Good ok-team_list: Good ok-awards: Good ok-results: Good ok-schedule: Good wat-rankings: Good wat-team_list: Good wat-awards: Good wat-results: Good wat-schedule: Good oh-rankings: Good oh-team_list: Good oh-awards: Error oh-results: Good oh-schedule: Good oc-rankings: Good oc-team_list: Good oc-awards: Good oc-results: Good oc-schedule: Good nv-rankings: Error nv-team_list: Good nv-awards: Good nv-results: Good nv-schedule: Good or-rankings: Good or-team_list: Good or-awards: Good or-results: Good or-schedule: Good roc-rankings: Good roc-team_list: Good roc-awards: Good roc-results: Good roc-schedule: Good co-rankings: Good co-team_list: Good co-awards: Good co-results: Good co-schedule: Good co-rankings: Good co-team_list: Good co-awards: Good co-results: Good co-schedule: Good ca-rankings: Good ca-team_list: Error ca-awards: Good ca-results: Good ca-schedule: Good ein-rankings: Error ein-team_list: Error ein-awards: Error ein-results: Error ein-schedule: Error ct-rankings: Error ct-team_list: Good ct-awards: Good ct-results: Good ct-schedule: Good kc-rankings: Good kc-team_list: Good kc-awards: Good kc-results: Good kc-schedule: Good nj-rankings: Good nj-team_list: Good nj-awards: Good nj-results: Good nj-schedule: Good nc-rankings: Error nc-team_list: Error nc-awards: Error nc-results: Error nc-schedule: Error arc-rankings: Good arc-team_list: Good arc-awards: Good arc-results: Good arc-schedule: Good la-rankings: Good la-team_list: Good la-awards: Error la-results: Good la-schedule: Good mi-rankings: Good mi-team_list: Good mi-awards: Good mi-results: Good mi-schedule: Good dc-rankings: Good dc-team_list: Good dc-awards: Good dc-results: Good dc-schedule: Good dt1-rankings: Good dt1-team_list: Good dt1-awards: Good dt1-results: Good dt1-schedule: Error wat-rankings: Good wat-team_list: Good wat-awards: Good wat-results: Good wat-schedule: Good gt-rankings: Good gt-team_list: Good gt-awards: Good gt-results: Error gt-schedule: Error pit-rankings: Good pit-team_list: Good pit-awards: Good pit-results: Good pit-schedule: Good ct-rankings: Error ct-team_list: Good ct-awards: Good ct-results: Good ct-schedule: Good hi-rankings: Good hi-team_list: Good hi-awards: Good hi-results: Good hi-schedule: Good sdc-rankings: Good sdc-team_list: Error sdc-awards: Good sdc-results: Good sdc-schedule: Good ga-rankings: Good ga-team_list: Good ga-awards: Good ga-results: Good ga-schedule: Good gg-rankings: Good gg-team_list: Good gg-awards: Good gg-results: Good gg-schedule: Good cur-rankings: Good cur-team_list: Good cur-awards: Good cur-results: Good cur-schedule: Good ma-rankings: Good ma-team_list: Error ma-awards: Good ma-results: Good ma-schedule: Good ut-rankings: Error ut-team_list: Error ut-awards: Error ut-results: Error ut-schedule: Error mo-rankings: Good mo-team_list: Good mo-awards: Good mo-results: Good mo-schedule: Good mn-rankings: Good mn-team_list: Good mn-awards: Error mn-results: Good mn-schedule: Good mi-rankings: Good mi-team_list: Good mi-awards: Good mi-results: Good mi-schedule: Good wc-rankings: Error wc-team_list: Error wc-awards: Error wc-results: Error wc-schedule: Error on-rankings: Good on-team_list: Error on-awards: Error on-results: Good on-schedule: Good ein-rankings: Error ein-team_list: Error ein-awards: Error ein-results: Error ein-schedule: Error va-rankings: Good va-team_list: Good va-awards: Good va-results: Good va-schedule: Good sac-rankings: Good sac-team_list: Good sac-awards: Error sac-results: Good sac-schedule: Good ma-rankings: Good ma-team_list: Error ma-awards: Good ma-results: Good ma-schedule: Good is-rankings: Good is-team_list: Good is-awards: Good is-results: Good is-schedule: Error in-rankings: Good in-team_list: Good in-awards: Error in-results: Good in-schedule: Good il-rankings: Good il-team_list: Good il-awards: Good il-results: Good il-schedule: Good oh-rankings: Good oh-team_list: Good oh-awards: Error oh-results: Good oh-schedule: Good az-rankings: Good az-team_list: Good az-awards: Good az-results: Good az-schedule: Good pit-rankings: Good pit-team_list: Good pit-awards: Good pit-results: Good pit-schedule: Good nh-rankings: Good nh-team_list: Good nh-awards: Good nh-results: Good nh-schedule: Good nj-rankings: Good nj-team_list: Good nj-awards: Good nj-results: Good nj-schedule: Good da-rankings: Error da-team_list: Good da-awards: Error da-results: Error da-schedule: Error nc-rankings: Error nc-team_list: Error nc-awards: Error nc-results: Error nc-schedule: Error ny-rankings: Good ny-team_list: Good ny-awards: Error ny-results: Good ny-schedule: Good wa-rankings: Good wa-team_list: Error wa-awards: Error wa-results: Good wa-schedule: Good nv-rankings: Good nv-team_list: Good nv-awards: Good nv-results: Good nv-schedule: Good cmp-rankings: Error cmp-team_list: Good cmp-awards: Good cmp-results: Good cmp-schedule: Good sac-rankings: Good sac-team_list: Good sac-awards: Error sac-results: Good sac-schedule: Good dc-rankings: Good dc-team_list: Good dc-awards: Good dc-results: Good dc-schedule: Good wor-rankings: Good wor-team_list: Error wor-awards: Good wor-results: Good wor-schedule: Good sj-rankings: Good sj-team_list: Good sj-awards: Good sj-results: Good sj-schedule: Good kc-rankings: Good kc-team_list: Good kc-awards: Good kc-results: Good kc-schedule: Good sj-rankings: Good sj-team_list: Good sj-awards: Good sj-results: Good sj-schedule: Good gal-rankings: Good gal-team_list: Good gal-awards: Good gal-results: Good gal-schedule: Good ca-rankings: Good ca-team_list: Error ca-awards: Good ca-results: Good ca-schedule: Good Running all those requests used 1% of my CPU time and 3% of my bandwidth. The good news? This is without any sort of caching so I will not be adding any caching to this version in the foreseeable future unless I start hitting my limits. |
![]() |
| Thread Tools | |
| Display Modes | Rate This Thread |
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| 2009 FRC Kit of Parts (Pro/ENGINEER format) - Supported by PTC | PTC_FRC | CAD | 18 | 18-11-2009 01:02 |
| FIRST Event Data in XML format | Andrew Schreiber | Technical Discussion | 11 | 03-05-2009 04:18 |
| XML Interchange format | Nibbles | Scouting | 25 | 03-10-2008 02:09 |
| How to write image data to binary PGM file format(P5)? | tommy_chai | Programming | 0 | 08-10-2007 08:22 |
| Scouting Data Interchange Format | proegssilb | Scouting | 7 | 06-06-2007 19:29 |