Match Results API for your website

[plnyyanks]

Yeah, this looks very cool and convenient. I'll be sure to check it out when I get a free couple of hours.

[lemiant]

I really like the functionality, but the IMHO the display on your website is a bit tacky. Maybe some rounded corners or AJAX refresh would help.

[quinxorin]

Originally Posted by lemiant

I really like the functionality, but the IMHO the display on your website is a bit tacky. Maybe some rounded corners or AJAX refresh would help.

It DOES have AJAX refresh!
It works by loading a cached copy first, then uses AJAX to replace it with an updated version once the server is done scraping it from FIRST. It then refreshes automatically every 60 seconds. If you click the "Refresh" link (which disappears once the AJAX does it's thing) it will force the page to be updated server-side before being sent to the browser.

It doesn't normally scrape before transmitting due to the several-second lag caused by scraping the data from FIRST.

Also, that very obvious notice about our team's first meeting isn't normally there, obviously. That makes it look rather ugly.

[Andrew Schreiber]

Originally Posted by quinxorin

It DOES have AJAX refresh!
It works by loading a cached copy first, then uses AJAX to replace it with an updated version once the server is done scraping it from FIRST. It then refreshes automatically every 60 seconds. If you click the "Refresh" link (which disappears once the AJAX does it's thing) it will force the page to be updated server-side before being sent to the browser.

It doesn't normally scrape before transmitting due to the several-second lag caused by scraping the data from FIRST.

Also, that very obvious notice about our team's first meeting isn't normally there, obviously. That makes it look rather ugly.

So, you are scraping the data off the FIRST site how often? Are you storing any data on your servers (aside from basic caching)? Basically, are you being a good citizen or not?

[quinxorin]

Originally Posted by Andrew Schreiber

So, you are scraping the data off the FIRST site how often? Are you storing any data on your servers (aside from basic caching)? Basically, are you being a good citizen or not?

Yes - the data is stored on our server, that's what I meant. The system only scrapes the data from our competitions, and most importantly, only when users are actually viewing the page. The data is scraped from FIRST a maximum of once every minute (every hour right now, since it's not competition season). If you use this tool, you can set that number to be whatever you would like, and of course configure it to scrape from your own competitions.

This is a good point, I'm glad you brought it up.

[Andrew Schreiber]

Originally Posted by quinxorin

Yes - the data is stored on our server, that's what I meant. The data is scraped from FIRST a maximum of once every minute (every hour right now, since it's not competition season). It only scrapes the data from our competitions, and only when users are actually viewing the page. If you use this tool, you can set that number to be whatever you would like, and of course configure it to scrape from your own competitions.

This is a good point, I'm glad you brought it up.

I would suggest dumping that lower limit to 6 minutes since that is pretty much the lowest turn around rate for matches. (That is what FRCFeed's caching was set to before Google App Engine popped out of preview and it would have started costing me too much to run it there)

[quinxorin]

Originally Posted by Andrew Schreiber

I would suggest dumping that lower limit to 6 minutes since that is pretty much the lowest turn around rate for matches. (That is what FRCFeed's caching was set to before Google App Engine popped out of preview and it would have started costing me too much to run it there)

Hmmm, now that is a good idea.

[Andrew Schreiber]

Originally Posted by quinxorin

Hmmm, now that is a good idea.

You other option, and one I just never got around to implementing, would be to check the Last-Modified header and see if it has changed since your cached version.

[lemiant]

Originally Posted by quinxorin

It DOES have AJAX refresh!
It works by loading a cached copy first, then uses AJAX to replace it with an updated version once the server is done scraping it from FIRST. It then refreshes automatically every 60 seconds. If you click the "Refresh" link (which disappears once the AJAX does it's thing) it will force the page to be updated server-side before being sent to the browser.

It doesn't normally scrape before transmitting due to the several-second lag caused by scraping the data from FIRST.

Also, that very obvious notice about our team's first meeting isn't normally there, obviously. That makes it look rather ugly.

Ahh, ok. I just clicked the link, saw the page refresh and then assumed.

[quinxorin]

I just updated the system to version 1.3p. The previous discussion about scrape times made me realize (about five minutes ago) that the scrape time limit was per user, so if ten people were viewing the page, then it would scrape ten times as fast, even though each individual user would still see it only as often as the scrape time permitted - an obvious flaw. But it's fixed now, and the system will not scrape data from FIRST if it hasn't been long enough.

I kind of hurried this out, so it hasn't had the proper bug testing. It was a simple change and shouldn't have caused any, but if anything stops working, please tell me.

Another minor change: the "last updated on" notice is now at the bottom, not the top.

There have been fifteen downloads at the time of this posting.

Originally Posted by Andrew Schreiber

You other option, and one I just never got around to implementing, would be to check the Last-Modified header and see if it has changed since your cached version.

Unfortunately, that still means that the connecting has to occur, because the header isn't sent from FIRST unless an http request is sent. The only benefit of using that header, that I can think of, would be to reduce load on the server implementing the API, because then it wouldn't have to process the data. If you look at the function library, it uses a seriously ginormous number of string operations and array manipulations. Am I totally wrong about the usage of that header?

[KeatonM]

This is great!

My question, however, is regarding the number of events you can have.

PHP Code:

'teamevents' => 
      array (
      //Add the event codes for your team here. 
        0 => 'gg',
        1 => 'mi',
        2 => 'gl',
        3 => 'Newton',
      ), 


I added the fourth event, and the page no longer seems to cache and update. I'm not awesome with PHP, so I'm wondering if three events is the max, hardcoded number in the script, or if I should be able to add the fouth event.

Thanks again for coding this!

[quinxorin]

Originally Posted by KeatonM

This is great!

My question, however, is regarding the number of events you can have.
...
I added the fourth event, and the page no longer seems to cache and update. I'm not awesome with PHP, so I'm wondering if three events is the max, hardcoded number in the script, or if I should be able to add the fouth event.

Thanks again for coding this!

Interesting. Yes, you should be able to add all the events you attended. I just tried it, and it worked fine for me. PHP is often set up differently on different servers, however, which means that it behaves differently.

It does work with just three, right?

Try the following:
1. Make sure that your team number is properly defined in the config file. It should read:

PHP Code:

$teamnumber=27; 


2. Try adding the following line to the config file while testing, but be sure to remove it before publication:

PHP Code:

$bypass_scrape_protection=true; 


That line has to do with the scrape protection system that I implemented in the most recent version (see above). It was hurried out, so it still has flaws that I will fix. That line will disable it.
3. Check to make sure the www2.usfirst.org domain is online. They must be doing some upgrades or something, because it seems to be going down for short periods every few days, or at least it did when I was building this system. It got frustrating.
4. If it's still not working, add the following line to the top of the matchresults_config.php file:

PHP Code:

error_reporting(E_ERROR | E_WARNING | E_PARSE); 


then use the system again. That will make PHP spit out any errors that might have occurred. If it's something about the max execution time (or time limit) being exceeded, then use:

PHP Code:

set_time_limit(60); 


to set the max execution time to one minute instead of the default 30 seconds. This means that your server doesn't have a fast enough connection (unlikely) or enough processing power to handle four events in 30 seconds. If it gives an error about a memory limit, use the similar function:

PHP Code:

ini_set('memory_limit', '256M'); 


to increase the memory limit to an outrageous amount.

Normally, you'd use matchresults_viewrecord.php. When testing, try using matchresults_output_show.php. That is the most basic one - it won't use AJAX Refresh or even save to the cache, just get you your data.

[KeatonM]

Yes, it works with just three! I didn't mention that!

Everything was correctly configured, and the page matchresults_output_show.php did show all 4, so my guess is that the cache process doesn't work on my server (shared hosting with Arvixe.com).

My PHP configuration is Memory_limit = 256M, max_execution_time = 600