[TBA]: USFIRST.com changes break compatibility

[Greg Marra]

USFIRST.com has changed the way you access a team's information. While the old page to view Team 177's profile would have been:
https://my.usfirst.org/myarea/index....minfo&team=177

It is now:
https://my.usfirst.org/myarea/index....ils&tpid=20840

You can see the parameter here is tpid=20840. I don't know what that means, but it doesn't have "177" like the old page did.

This breaks all of TBA's "More information about Team xyz" links.

Also, it seems that the events are no longer linked to with a parameter based on the event name, but instead another id.
https://my.usfirst.org/myarea/index....tails&eid=2395

This breaks some of TBA's backend scraping.

[Chris is me]

For the time being you could use http://frclinks.frclinks.com/team/177 to get the same result.

[Greg Marra]

Originally Posted by Chris is me

For the time being you could use http://frclinks.frclinks.com/team/177 to get the same result.

Does anyone know how Pat is doing the frclinks redirects to hit the right USFIRST pages?

[artdutra04]

Originally Posted by Greg Marra

Does anyone know how Pat is doing the frclinks redirects to hit the right USFIRST pages?

I'm guessing a giant lookup table made by crawling through the website and seeing what team each page refers to?

It looks like FIRST's web developers got lazy and stopped using human readable parameters and instead decided to use whatever internal database IDs all the data has.

[lynca]

Originally Posted by artdutra04

I'm guessing a giant lookup table made by crawling through the website and seeing what team each page refers to?

My friend and I were able to make a table of teams. It can be a bit confusing because some of the tpids link up to the same team but different years.

[Greg Marra]

Originally Posted by lynca

My friend and I were able to make a table of teams. It can be a bit confusing because some of the tpids link up to the same team but different years.

I am worried that tpids will change in the future, especially given that they're internal database IDs.

The Blue Alliance's changes this year to use all human-readable parameters made a big difference in finding and linking to things on TBA. It's much easier to go to http://www.thebluealliance.net/tbatv/event/2009ct than something with an arbitrary ID.

Maybe we'll have to build a generic translation system everyone can use?

[Pat Fairbank]

Originally Posted by Greg Marra

Does anyone know how Pat is doing the frclinks redirects to hit the right USFIRST pages?

Pat does. The tpid's seem to be constant per team per year (but are different for the same team in different seasons). I have a database of team->tpid links. If a team is not in the database, Frclinks scrapes the 2009 team list looking for it, and stores any other previously unknown results at the same time (to account for a constantly-expanding list during registration). So once FIRST starts keeping records for 2010, another scraping will likely occur.

[Pat Fairbank]

I should also mention that with some HTML inspection and trial and error I was able to come up with reasonable URL schemes for everything except the teams. So, to access the event details for Midwest 2008, for example, the URL would be https://my.usfirst.org/myarea/index....=2008&event=il.