Quote:
Originally Posted by compwiztobe
...
|
In case anyone's still curious about this...
The query expressions appended to index.lasso only work if you have an active session with the server and if you send the http request with the appropriate headers. These headers should include both a cookie with your session ID, as well as the headers:
Code:
host: my.usfirst.org
referer: https://my.usfirst.org/myarea/index.lasso?event_type=FRC&year=2015
or some variation of these (I didn't rigorously test for what is required, I just mimicked the headers in Chrome's logs until it worked).
Then you can send a request to:
Code:
https://my.usfirst.org/myarea/index.lasso?page=event_details&eid=[Event ID]
with your Event ID. Or you can request any other data. A list of Event IDs that I scraped from the event listing can be found at:
https://netfiles.umn.edu/users/siekm...Cdata/eids.csv
I wrote a python script that scrapes all the event details and builds a table in html. For example, this was scraped at ~11:00am EST on Friday, 9/26:
https://netfiles.umn.edu/users/siekm...ata/table.html
If I can find access to a webserver, I could put up a page that updates this table a few times every hour. However, processing 100+ separate https requests takes nearly a minute, so it's obviously a better idea for usfirst.org to aggregate this data for us and make it available with a single request...