Chief Delphi - FIRST Website Team Pages?

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)

- General Forum (http://www.chiefdelphi.com/forums/forumdisplay.php?f=16)

- - FIRST Website Team Pages? (http://www.chiefdelphi.com/forums/showthread.php?t=79448)

FIRST Website Team Pages?

I am getting ready to update The Blue Alliance to have 2010 data for all teams. However, FIRST has changed their website, and I can't figure out how to find the URL for a team's information page.

For instance:

Team 177 is at https://my.usfirst.org/myarea/index....ils&tpid=28633
Team 195 is at https://my.usfirst.org/myarea/index....ils&tpid=28651

This makes it appear that you can find a FIRST team's page by setting the "tpid" parameter to be equal to "28456 + Team Number". This suggests that Team 111's page should be https://my.usfirst.org/myarea/index....ils&tpid=28567, but this is in fact the page for Team 115.

Is there any way to turn a team number into a USFIRST.org team information page URL without any prior knowledge of how their database is structured? Pat Fairbank seems to have figured it out for frclinks.com, but I am not sure how he has done it.

Thanks!

Re: FIRST Website Team Pages?

Quote:

Originally Posted by Greg Marra (Post 890241)

I am getting ready to update The Blue Alliance to have 2010 data for all teams. However, FIRST has changed their website, and I can't figure out how to find the URL for a team's information page...Is there any way to turn a team number into a USFIRST.org team information page URL without any prior knowledge of how their database is structured? Pat Fairbank seems to have figured it out for frclinks.com, but I am not sure how he has done it.

Why not just use frclinks itself and not worry about reinventing that particular wheel?

Re: FIRST Website Team Pages?

Pat posted how he did it in your previous thread: http://www.chiefdelphi.com/forums/sh...ad.php?t=78368

Re: FIRST Website Team Pages?

Quote:

Originally Posted by Joe Ross (Post 890247)

Pat posted how he did it in your previous thread: http://www.chiefdelphi.com/forums/sh...ad.php?t=78368

Thanks Joe! I hadn't seen these posts.

Quote:

Originally Posted by Alan Anderson (Post 890245)

Why not just use frclinks itself and not worry about reinventing that particular wheel?

FRCLinks uses a Javascript redirect. I am pointing at FRCLinks for links to team pages right now on TBA, but I need to do a full scrape of FIRST's pages to update Team Names to be accurate now. Wget doesn't follow Javascript redirects - I may need to bake up something a bit fancier to either parse these out of FRCLinks, or parse them out of FIRST's team data when I am scraping event attendance.

I wish this were easier.

I was able to get this URL for listing Teams, but it's hardcoded to max out at 250 teams listed. I was hoping to get every single team on the page at once, and then scrape out all the team URLs in one go. This approach won't work on its own.

https://my.usfirst.org/myarea/index....asons_frc=2010

Re: FIRST Website Team Pages?

Attached is a CSV with every team competing in 2010's team number and tpid, which is the number FIRST uses to refer to the team. Hopefully this will be useful to someone in the future.

Re: FIRST Website Team Pages?

Quote:

Originally Posted by Greg Marra (Post 890251)

"window.location.*?=.*?\"(.*)\"" as a regex on the content of the frclinks is a pretty simple way of grabbing Pat's redirect. That is how frcfeed is doing it. Just grab the content of group 1.

In python:

Code:

URL = re.search("window.location.*?=.*?\"(.*)\"",Content).group(1)

Re: FIRST Website Team Pages?

Quote:

Originally Posted by Andrew Schreiber (Post 890386)

"window.location.*?=.*?\"(.*)\"" as a regex on the content of the frclinks is a pretty simple way of grabbing Pat's redirect. That is how frcfeed is doing it. Just grab the content of group 1[/code]

I agree. I talked with Pat and decided it was easiest to just re-scrape the data from FIRST myself, since it required minimal modifications to existing TBA scraping scripts. Pat's service is great, and I'm going to keep using it on TBA where it makes sense. :)

Re: FIRST Website Team Pages?