Anyone else having problems getting info for 2017 Carver through TBA’s API? I’m trying to request the team list using the event code 2017carv which appears to be the expected event code based on the URL for Carver in TBA’s site; also when event info for individual teams in Carver is requested 2017carv is one of the returned events. I’ve e-mailed TBA.
I’ve reproduced your error and passed this on to the TBA Slack group. After some more digging, it looks like this might be a problem for all Houston divisions.
Looks like we’re not the only ones who have had this problem. See https://github.com/the-blue-alliance/the-blue-alliance/issues/1885.
Should be resolved within 24 hours.
Just updating here what’s been covered on the GitHub issue page…
TBA uses extensive caching layers to reduce load and costs. Because there was no indication from FIRST that divisions would be released today, the default cache time (24 hours) was still in place. As a result even though TBA may have the updated data internally, the pages you and I see are cached. It should resolve automatically within 24 hours or so.
Thanks for the replies. I’m not sure the cache issue is the problem here. I’m able to get valid responses for the other divisions. Carver is the only Houston division with a 4 letter abbreviation, probably to distinguish it from Carson in St. Louis, so I wonder if that is the underlying problem. I ended up manually coding the list of teams; all the other TBA requests I used when looping through this list worked without issues.
It’s a cache issue. The other divisions have the same cache timers, but they may not all end at the same time. That’s causing the discrepancy you’re seeing.
Just to clarify a bit more. TBA uses an “edge cache” at the point data leaves Google’s datacenters (or you can think of it at the border of our CDN). If a response has a Cache-Control header that says the data can be cached, then future requests will also have that same content returned without even hitting a TBA server. This is great for keeping costs down, since those request aren’t billed (this matters when you’ve handled 12 million API requests since March 1). However, the downside is that once a response hits this cache, there is no going back until it ages out.
In APIv2, events that are not currently active have a cache expiration of 24 hours (this has been changed in APIv3, so this issue doesn’t appear there). This means that most APIv2 endpoints for CMP divisions were generated at some random point in the last 24 hours and the cached versions are currently being served by the CDN. We can’t flush this cache when new data comes in, since it’s “downstream” of the TBA servers. If we knew about when the divisions are released, then we can manually decrease the cache times (we do this on registration days), but this has to happen some amount of time in advance (at least as long as the cache expiration time). This is a similar approach you might take with DNS TTL values when changing where a domain points.
TBA is designed around knowing when data is “hot” and when it is “cold”. We’re a pretty small operation with a tight budget, so we try and keep costs down where we can. We try to cache cold data as much as possible since it would be wasteful to repeat the same queries over and over for data that isn’t changing (especially when you’re being billed for instance hours and database queries). The assumption breaks down when data unexpectedly changes and near-instant updates are required (like divisions being announced). We can handle one or the other, but not both.
However, we’re working on improving this, as we know it’s a pain point for many people. APIv3 is more efficient under the hood, so we keep all the edge cache expiration at 61 seconds, which means it will update much more quickly than APIv2.
tl;dr: caching is hard
tl;dr2: just use apiv3