Retrieve data from TheBlueAlliance

[stingray27]

Hello Chief Delphi,

I am currently trying to write an android app and I need to pull the team lists from specific regionals in 2013. I am able to parse the CD-Spy xml feed from last year to gain match results, and I am able to parse the match results, awards, and rankings from 2012 from the usFIRST.org site which is in HTML format. But now I would like to try and parse either thebluealliance regional data or the queries on the usfirst site for this upcoming year. I cannot seem to get it right and I am having a whole bunch of trouble. Can anyone explain what web formats are used in these sites or how I should go about trying to parse this information? Thanks!

[Joe Ross]

Did you see the following on the contact us page?


API Access

The new "dotcom" website brings a new API that doesn't require an API key to use. We're not quite ready to launch it, but please contact us if you are interested in using it. Check out our github repository if you'd like to help build it. :D

[stingray27]

You know I went to the contact page but my head was still wrapped up in code world and I wasn't paying attention :o I will get in contact to see if that will be of some interest to me. But as for usfirst.org in the meantime, suggestions would be great.

[Eugene Fang]

But as for usfirst.org in the meantime, suggestions would be great.

What information specifically are you trying to get from the FIRST website?

[stingray27]

What information specifically are you trying to get from the FIRST website?

When you go to the usfirst website, go to events, regional events, and then click on a regional, there is the query for "What teams are registered for this event?" I would like to pull this list of team names and numbers into my app. Only problem with this is that their team name isn't the nickname. But it is by far better than nothing.

[Eugene Fang]

When you go to the usfirst website, go to events, regional events, and then click on a regional, there is the query for "What teams are registered for this event?" I would like to pull this list of team names and numbers into my app. Only problem with this is that their team name isn't the nickname. But it is by far better than nothing.

Some TBA API documentation can be found here. We're constantly working to improve the API so let us know if there are any other calls you'd like to see.
https://github.com/gregmarra/the-blue-alliance/wiki/API-Documentation

You should be able to pull list of team names and numbers with a combination of the calls to ApiEventList (to get all events), ApiEventDetails (to get all teams for an event), and ApiTeamsShow (to get info for a list of teams).

[vinnie]

I've used The FIRST Alliance for a test scouting app that I was working on a little while ago.
Their API is pretty nice.
Here is some sample code, it gets all the teams from a regional and outputs them according to their ranking. It could easily be extended to get the team name.

package org.team3309.scout;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;

import com.google.appengine.labs.repackaged.org.json.JSON Array;
import com.google.appengine.labs.repackaged.org.json.JSON Exception;
import com.google.appengine.labs.repackaged.org.json.JSON Object;

public class FirstAlliance {

private static final String API = "http://www.thefirstalliance.org/api/api.json.php?action=";

public static Event[] getEvents(String teamNumber) {
Event[] competitions = new Event[0];
String url = API + "team-events&team-number=3309";
JSONArray events = getArray(url);
competitions = new Event[events.length()];
try {
for (int i = 0; i < events.length(); i++) {
JSONObject event = events.getJSONObject(i);
competitions[i] = new Event(event);
}
} catch (JSONException ex) {
ex.printStackTrace();
}
return competitions;
}

public static Event getEventById(int id){
String url = API+ "event-details&event-id="+id;
JSONObject json = getJson(url);
return new Event(json);
}

private static JSONArray getArray(String url) {
try {
HttpURLConnection c = (HttpURLConnection) new URL(url)
.openConnection();
BufferedReader reader = new BufferedReader(new InputStreamReader(
c.getInputStream()));
String line;
String input = "";
while ((line = reader.readLine()) != null)
input += line;
JSONObject json = new JSONObject(input);
return json.getJSONArray("data");
} catch (Exception ex) {
ex.printStackTrace();
}
return null;
}

private static JSONObject getJson(String url){
try {
HttpURLConnection c = (HttpURLConnection) new URL(url)
.openConnection();
BufferedReader reader = new BufferedReader(new InputStreamReader(
c.getInputStream()));
String line;
String input = "";
while ((line = reader.readLine()) != null)
input += line;
JSONObject json = new JSONObject(input);
return json.getJSONObject("data");
} catch (Exception ex) {
ex.printStackTrace();
}
return null;
}

public static class Event{

private String name;
private int id;
private JSONArray rankingsJson;
private ArrayList<Ranking> rankings;

private Event(JSONObject json){
this.name = json.optString("name");
this.id = json.optInt("id");
this.rankingsJson = json.optJSONArray("rankings");
}
public String getName(){
return name;
}
public int getId(){
return id;
}
public ArrayList<Ranking> getRankings(){
if(rankings != null)
return rankings;
ArrayList<Ranking> list = new ArrayList<Ranking>();
for(int i=0; i<rankingsJson.length(); i++){
JSONObject json = rankingsJson.optJSONObject(i);
list.add(new Ranking(json.optString("Team"), json.optInt("Rank"), json.optString("Record (W-L-T)")));
}
return list;
}

public String toString(){
return name;
}
}

public static class Ranking{

private String team, record;
private int ranking;

private Ranking(String team, int ranking, String record){
this.team = team;
this.ranking = ranking;
this.record = record;
}

public String getRecord(){
return record;
}

public int getWins(){
return Integer.parseInt(record.substring(0, record.indexOf("-")));
}

public int getLosses(){
return Integer.parseInt(record.substring(record.indexOf("-")+1, record.indexOf("-", record.indexOf("-")+1)));
}

public int getTies(){
return Integer.parseInt(record.substring(record.lastIndex Of("-")+1));
}

public String getTeam(){
return team;
}

public int getRank(){
return ranking;
}
}
}

If you want to get the team names use the method getArray("http://www.thefirstalliance.org/api/api.json.php?action=event-teams&event-id=<your event id>") and then loop through the array and get the field named "name" from each JSONObject. There is also a field called "number" if you want the team number too. If you want to know more about JSON you can go to json.org or look at this code to see how to get the team name.

If you need to find your event id, you can get the data from http://www.thefirstalliance.org/api/api.json.php?action=list-events

Edit: TBA's API looks nice too, if you want to use that you can still use the getJSON method (not getArray) if you don't know how to do HTTP GET in Android

Edit 2: Don't forget to add the INTERNET permission to your manifest. I've done that in probably every app that I've made and I feel more and more stupid each time.

[Brandon_L]

When you go to the usfirst website, go to events, regional events, and then click on a regional, there is the query for "What teams are registered for this event?" I would like to pull this list of team names and numbers into my app. Only problem with this is that their team name isn't the nickname. But it is by far better than nothing.

If you're dead set on using that list, if you can parse the team numbers you can use that in the Blue Alliance API to get the team nicknames.

[Greg Marra]

If you want to fetch the data from FIRST directly, check out what we do in our FIRST Field Management System parser, which gets nicknames by parsing https://my.usfirst.org/frc/scoring/index.lasso?page=teamlist

https://github.com/gregmarra/the-blue-alliance/blob/master/datafeeds/fms_team_list_parser.py

I'm always happy to help people if they want to reach out to me with questions. Sounds like we should make our contact page a bit clearer :D

[stingray27]

Thank you very much for your responses! I was able to get both TFA and TBA up and running. I decided to stick with my html parser for awards, match data, and rankings because it is faster on my android device when refreshing information but as for the team listings, I decided to use The Blue Alliance. For those of you interested, I added my code that I used to develop my methods (in netbeans) before I moved in to Eclipse. So the code was just temporary but it works!

package bluealliance;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Scanner;
import org.json.*;

public class BlueAlliance {

private static final String API = "http://www.thebluealliance.com/api/v1/";

public static void main(String[] args) {
Scanner in = new Scanner(System.in);
String choiceS;
System.out.println("Would you like to view Event(1) or Team(2) information? ");
int choice = in.nextInt();
switch (choice) {
case 1:
System.out.println("Would you like to view a single event(1), or view all events with(2) or without(3) the team listings? ");
choice = in.nextInt();
EventDetails[] events;
String[] team_keys;
switch (choice) {
case 1:
System.out.println("Enter the event code of the 2013 regional: ");
choiceS = in.next();
EventDetails event = getEvent("2013" + choiceS);
team_keys = event.getTeamKeys();
if (event.getOfficiality()) {
System.out.println(event.getName() + "(KN: " + event.getKeyName() + " EC: " + event.getEventCode() + " SN: " + event.getShortName() + ") " + event.getLocation());
for (int j=0; j<team_keys.length; j++) {
System.out.println(team_keys[j]);
}
}
break;
case 2:
System.out.println("Enter the year: ");
choiceS = in.next();
System.out.println("CAUTION: Takes around 60 sec");
events = getAllEventsWithTeams(choiceS);
for (int i=0; i<events.length; i++) {
team_keys = events[i].getTeamKeys();
if (events[i].getOfficiality()) {
System.out.println(events[i].getName() + "(KN: " + events[i].getKeyName() + " EC: " + events[i].getEventCode() + " SN: " + events[i].getShortName() + ") " + events[i].getLocation());
for (int j=0; j<team_keys.length; j++) {
System.out.println(team_keys[j]);
}
}
}
break;
case 3:
System.out.print("Enter the year: ");
choiceS = in.next();
events = getAllEvents(choiceS);
for (int i=0; i<events.length; i++) {
team_keys = events[i].getTeamKeys();
if (events[i].getOfficiality()) {
System.out.println(events[i].getName() + "(KN: " + events[i].getKeyName() + " EC: " + events[i].getEventCode() + " SN: " + events[i].getShortName() + ") " + events[i].getLocation());
}
}
break;
default:
System.out.println("Incorrect Selection");
break;
}
break;
case 2:
System.out.println("Enter the team: ");
choiceS = in.next();
TeamDetails team = getTeam(choiceS);
String[] event_keys = team.getEventKeys();
System.out.println( team.getNickName() + " " + team.getKeyName() + " " + team.getLocation() + " " + team.getWebsite() + "\n" + team.getName());
for (int j=0; j<event_keys.length; j++) {
System.out.println(event_keys[j]);
}
break;
default:
System.out.println("Incorrect Selection");
break;
}
}

public static EventDetails[] getAllEvents(String year) {
EventDetails[] competitions = new EventDetails[0];
String url = API + "events/list?year=" + year;
JSONArray events = getArray(url);
competitions = new EventDetails[events.length()];
try {
for (int i = 0; i < events.length(); i++) {
JSONObject event = events.getJSONObject(i);
competitions[i] = new EventDetails(event);
}
} catch (JSONException ex) {
ex.printStackTrace();
}
return competitions;
}

public static EventDetails[] getAllEventsWithTeams(String year) {
EventDetails[] competitions = getAllEvents(year);
EventDetails[] competitionWithTeams = new EventDetails[competitions.length];
for (int i=0; i<competitions.length; i++) {
competitionWithTeams[i] = getEvent(competitions[i].getKeyName());
}
return competitionWithTeams;
}

public static EventDetails getEvent(String event_key) {
String url = API + "event/details?event=" + event_key;
JSONObject json = getJson(url);
return new EventDetails(json);
}

public static TeamDetails getTeam(String team) {
String url = API + "team/details?team=frc" + team;
JSONObject json = getJson(url);
return new TeamDetails(json);
}

public static TeamDetails[] getEventTeams(String[] teams) {
String teamList = "teams/show?teams=";
for (int i=0; i<teams.length; i++) {
if (i==0)
teamList += teams[i];
else
teamList += "," + teams[i];
}
String url = API + teamList;
System.out.println(url);
TeamDetails[] teamInfo;
JSONArray teamArray = getArray(url);
System.out.println(Integer.toString(teamArray.leng th()));
teamInfo = new TeamDetails[teamArray.length()];
try {
for (int i = 0; i < teamArray.length(); i++) {
JSONObject event = teamArray.getJSONObject(i);
teamInfo[i] = new TeamDetails(event);
}
} catch (JSONException ex) {
ex.printStackTrace();
}
return teamInfo;
}

private static JSONArray getArray(String url) {
try {
HttpURLConnection c = (HttpURLConnection) new URL(url).openConnection();
BufferedReader reader = new BufferedReader(new InputStreamReader(c.getInputStream()));
String line, input = "";
while ((line = reader.readLine()) != null)
input += line;
JSONArray json = new JSONArray(input);
return json;
} catch (Exception ex) {
ex.printStackTrace();
}
return null;
}

private static JSONObject getJson(String url){
try {
HttpURLConnection c = (HttpURLConnection) new URL(url).openConnection();
BufferedReader reader = new BufferedReader(new InputStreamReader(c.getInputStream()));
String line, input = "";
while ((line = reader.readLine()) != null)
input += line;
JSONObject json = new JSONObject(input);
return json;
} catch (Exception ex) {
ex.printStackTrace();
}
return null;
}

private static class EventDetails {

private String key_name, year, name, short_name, location, facebook_eid, start_date, end_date, officiality;
private String[] team_keys;

private EventDetails(JSONObject json) {
this.key_name = json.optString("key");
this.year = json.optString("year");
this.name = json.optString("name");
this.short_name = json.optString("short_name");
this.location = json.optString("location");
this.officiality = json.optString("official");
this.facebook_eid = json.optString("facebook_eid");
this.start_date = json.optString("start_date");
this.end_date = json.optString("end_date");
JSONArray temp = json.optJSONArray("teams");
if (temp!=null) {
String[] temp2 = new String[temp.length()];
try {
for (int i=0; i<temp2.length; i++) {
temp2[i] = temp.getString(i);
}
} catch (Exception e) {
e.printStackTrace();
}
this.team_keys = temp2;
}

}

public String getKeyName(){
return key_name;
}

public int getYear(){
return Integer.parseInt(year);
}

public String getEventCode(){
return key_name.substring(4);
}

public String getName(){
return name;
}

public String getShortName(){
return short_name;
}

public String[] getTeamKeys(){
return team_keys;
}

public String getLocation() {
return location;
}

public boolean getOfficiality() {
if (officiality.equals("true"))
return true;
else
return false;
}

public String getFacebookEID() {
return facebook_eid;
}

public String getStartDate() {
return start_date;
}

public String getEndDate() {
return end_date;
}
}

private static class TeamDetails {

private String key_name, team_number, name, nickname, website, location;
private String[] event_keys;

private TeamDetails(JSONObject json) {
this.key_name = json.optString("key");
this.team_number = json.optString("team_number");
this.name = json.optString("name");
this.nickname = json.optString("nickname");
this.location = json.optString("location");
this.website = json.optString("website");
JSONArray temp = json.optJSONArray("events");
if (temp!=null) {
String[] temp2 = new String[temp.length()];
try {
for (int i=0; i<temp2.length; i++) {
temp2[i] = temp.getString(i);
}
} catch (Exception e) {
e.printStackTrace();
}
this.event_keys = temp2;
}
}

public String getKeyName(){
return key_name;
}

public int getTeamNumber(){
return Integer.parseInt(team_number);
}

public String getName(){
return name;
}

public String getNickName(){
return nickname;
}

public String getWebsite(){
return website;
}

public String[] getEventKeys(){
return event_keys;
}

public String getLocation() {
return location;
}
}
}

[Michael Hill]

I must warn if you try to roll your own, FIRST's website does some really goofy session management. For example, if you want to pull a complete team list, you'll want to pull 250 teams/page rather than the default. However, there's no way to do this through just a URL alone. You must first request a page, extract the session ID they give you from an href field (,the myarea var) then use that session ID when you pull the data. It's really silly the way they did it. They store the number of teams to display/page server side rather than client side.

[Greg Marra]

I must warn if you try to roll your own, FIRST's website does some really goofy session management. For example, if you want to pull a complete team list, you'll want to pull 250 teams/page rather than the default. However, there's no way to do this through just a URL alone. You must first request a page, extract the session ID they give you from an href field (,the myarea var) then use that session ID when you pull the data. It's really silly the way they did it. They store the number of teams to display/page server side rather than client side.

https://github.com/gregmarra/the-blue-alliance/blob/master/datafeeds/datafeed_usfirst.py#L48 if anyone wants an example of doing this. It's a hassle.

[stingray27]

I must warn if you try to roll your own, FIRST's website does some really goofy session management. For example, if you want to pull a complete team list, you'll want to pull 250 teams/page rather than the default. However, there's no way to do this through just a URL alone. You must first request a page, extract the session ID they give you from an href field (,the myarea var) then use that session ID when you pull the data. It's really silly the way they did it. They store the number of teams to display/page server side rather than client side.

That was why I decided to use TBA api because it was too much of a hassle to do that. Parsing from the match results page and pages like that though is really easy.