Archiving Recorded Regionals 2008

[Jonathan Norris]

Regional Season is quickly coming along, its time to do a little organization to see how we are going to get EVERY Regional this year recorded and up on the interwebs. Last year by my rough count we got video footage from 26 regionals, with only about 7-8 of those regionals with at least 90% of the matches uploaded. Which for our community is a great achievement, with a little more organization I believe that we can get 90% of the regionals totally recorded from Match 1 to the last Final match.

I think you all know how great a scouting resource having match footage of different robots is. But it take a lot of effort from volunteers to get all that footage up on the internet. I know that last year I recorded and parsed three regionals (New Hampshire, Waterloo, and GTR) and it took a couple hours to set up the recording and then basically took a day or more for me to go through all the footage and cut it up the upload it for each regional. This year (because I am now in University) I will only be able to record and parse Waterloo. Now that means we need at least 41 other people to volunteer their time to record (the easy part), parse (i.e. cut up the video into matches), and upload them to the different online resources (The Blue Alliance Match Archives, SOAP, Youtube, ect.)

I'll try and keep this list updated of who is volunteering to record what (feel free to have more the one person record a regional! you can split up the hard work of parsing and uploading!)

Regional                                         Volunteer
Week 1
BAE Systems Granite State Regional 
Midwest Regional 
New Jersey Regional 
Oregon Regional 
St. Louis Regional 
Week 2
Arizona Regional 
Finger Lakes Regional                           Ed Patterson (Koko Ed, team 191)
Greater Kansas City Regional
NASA / VCU Regional 
San Diego Regional                              AJ (ay2b, team 980)
Week 3
Boilermaker Regional                            Mike Aalderink (mike AA, team 2015)
Brazil Regional 
Chesapeake Regional 
Connecticut Regional 
Detroit Regional 
Florida Regional 
Peachtree Regional 
Pittsburgh Regional 
Silicon Valley Regional                         AJ (ay2b, team 980)
Wisconsin Regional 
Week 4
Buckeye Regional                                David Thomas (blakcheez, team 964)
Lone Star Regional                              Kevin Sevcik (team 57)
Los Angeles Regional                            AJ (ay2b, team 980)
Microsoft Seattle Regional 
Oklahoma City Regional 
UC Davis Sacramento Regional 
Waterloo Regional                               Jonathan Norris (team 610)
West Michigan Regional                          Mike Aalderink (mike AA, team 2015)
Israel Regional 
Week 5
Bayou Regional 
Boston Regional 
Colorado Regional 
Great Lakes Regional 
Greater Toronto Regional 
Hawaii Regional 
Las Vegas Regional 
Minnesota Regional 
Palmetto Regional 
Philadelphia Regional
SBPLI Long Island Regional 
New York City Regional

(I believe the SOAP people usually will archive the Championship event, i'm sure they will find this thread and let us know if that is happening again)

I'll post some more in-depth tutorials on how to go about recording, parsing, and uploading (and my personal opinion on formating, mpg4 FTW!) later on, but right now its more important to find someone interested in recording each of these regionals. Feel free to ask here or PM me with any questions.

[Jonathan Norris]

For those who don't know the history, the last couple years the folks at team 108 have donated a server to host FIRST match video that any one can access, called SOAP. Much of that video is recorded by the team, and some of it is uploaded by others. In the past year the guys at The Blue Alliance have built a great website taking all this recorded video and put it all into Flash video (like Youtube), and add match scoring information and tagging for teams. This lets you go through and search by team number and come up with a list of all the team's matches. It really is a great resource for scouting, and have made going through matches on SOAP's a lot easier.

So not to say that SOAP doesn't have its place, but I believe the system that The Blue Alliance guys have built for online video is very polished and easy to use. So my personal opinion for this next season of Regionals is to try and use The Blue Alliance as the central hub for match video (I know that I am making a lot of work for you guys, sry ). This is just my opinion, I'm sure I'll hear others with their own ideas about how to centralize match footage.

Youtube... last year we heard from Dean that he wants Youtube flooded with FIRST video, and for FIRST to grow in the media and on the internet this great resource needs to be used... properly. If you go on Youtube today you can find a large amount of FIRST video, however it is spread all over the place and does not do a good enough job at linking back to one another.

To get exposer on Youtube we need to make sure that every new video is properly tagged, title, and linked by channels and community groups. So I propose that we try and find a standard tagging and naming system, and create a large group and channel for all FIRST video. Hopefully we can find a 'standard' and spread the word to all FIRST teams to use this format so we can get better exposer!

Formating... I know in the past the standard video format for all matches has been Windows Media Video (.wmv), but let me try and make the case for MPG4 (.mp4). .wmv files for a long time were a popular standard video file, because basically every one had a PC and it was the best compression format. But these days of mass internet video .mp4 is clearly the new standard and the only .wmv video I have on my computer are old matches (most I have converted). Every type of mass portable media player supports the format (iPods, Zunes, Cell Phones ect.). Even HD DVD's, online movie rentals, ect. use a form of the MPG4 format. In the near future flash video will support MPG4 video (in the form of h.264), so the world is moving towards MPG4 as the standard video format. By using MPG4 as a standard for video it will make it a lot easier for people to put these great match videos where ever they want... on their computer, on their iPod, ect.

[synth3tk]

I can cover Buckeye this year and submit them to TBA, SOAP, and back it up on our site space.

YouTube channel just setup at http://www.youtube.com/profile?user=FRCommunity Contact me by PM, YouTube PM, or media@bedfordrobotics.org to add videos or info.

I agree that there should be a standard tagging convention if we are to do this as a whole community. We can discuss it in this thread.

[Koko Ed]

I can do FLR.

[Kevin Sevcik]

I can pull LSR again, but I think part of the problem last year was SOAP running a little low on server space to handle such a massive amount of footage. I think it'd be polite to give them a heads up that CD is making a concerted effort at archiving all the regionals.

To the suggestion that we just concentrate on TBA as the central hub.... I like their format and layout and etc for the convenience, but it's flash video. It typically looks marginally better than something filmed on a cell phone cam covered in vaseline. If TBA is willing to host better quality WMVs of all the matches in addition to the cheesy flash video, then I'm happy, otherwise, I'd rather find someone who really wants to archive the files.

[synth3tk]

Something to think about as far as bandwidth goes, but how complex would it be to get teams with massive amounts of excess webspace and bandwidth to host their own videos and maybe other area teams, then get these sites to link to them?

[Ellery]

Originally Posted by Koko Ed

I can do FLR.

Ed - Make sure you bring a real VCR and not the mono cheapster one from the site or else the quality is pretty crappy.

FYI - it took me quite a long time to convert 1 regional of all matches so plan on spending a good weekend.

Ellery

[Jonathan Norris]

Originally Posted by blakcheez

Something to think about as far as bandwidth goes, but how complex would it be to get teams with massive amounts of excess webspace and bandwidth to host their own videos and maybe other area teams, then get these sites to link to them?

That is what both the SOAP archives and The Blue Alliance archives (both linked above) are doing for us. They both provide hosting for Regional Match videos and a central location to find all the seasons matches. Its a lot easier to just have a centralized service to host all the footage, then to have the matches spread over a bunch of different servers.

[Jonathan Norris]

Originally Posted by Kevin Sevcik

To the suggestion that we just concentrate on TBA as the central hub.... I like their format and layout and etc for the convenience, but it's flash video. It typically looks marginally better than something filmed on a cell phone cam covered in vaseline. If TBA is willing to host better quality WMVs of all the matches in addition to the cheesy flash video, then I'm happy, otherwise, I'd rather find someone who really wants to archive the files.

On every match on The Blue Alliance you have the option to download it in the higher quality format (most of them right off SOAP last year). Maybe they need to make that more obvious on their website.

[synth3tk]

Originally Posted by Jonathan Norris

That is what both the SOAP archives and The Blue Alliance archives (both linked above) are doing for us. They both provide hosting for Regional Match videos and a central location to find all the seasons matches. Its a lot easier to just have a centralized service to host all the footage, then to have the matches spread over a bunch of different servers.

Well, I understand that, but someone stated above that they started running out of space. It's an idea just in case they start runnning out of space.

[Jonathan Norris]

Originally Posted by blakcheez

Well, I understand that, but someone stated above that they started running out of space. It's an idea just in case they start runnning out of space.

I'll point The Blue Alliance guys over to this thread to see if they have the capabilities to archive all the matches this season.

[Tom Bottiglieri]

MP4 is fine on our side. (In fact, it's much better. Hopefully when flash 9 is polished we can do away with FLV's)

It will take more resources for compression, though. We are battling an h264 encoding problem at the moment. Does anyone know a good way to compress this stuff in real time?

[notaPINKtruck]

h.264 is really processor intensive both on the compression and playback ends, and it's definitely not designed for real-time encoding (at least by anybody other than DirecTV and the likes). I don't think you're going to get good performance unless you buy a specialized encoder card or have a REALLY high end system.

I recommend staying with wmv for now. The encoder performs well in real-time professionally, and is freely available. Plus I would argue it's the more accessible than quicktime MP4.

FLV is great for easy website integration, but not much else.

[synth3tk]

I prefer to disagree to the pro-WMV argument. It is a dying format, more and more players support MP4 well, than Windows Media.

[Mike AA]

I will be able to record via DVD Boilermaker and West Michigan. I will do my best to get the video uploaded for downloads. Also inaddition I have been hosting video archives for the last 3 years. I only have most of last year and part of 2006 online. I am hunting through my DVDs looking for past years video and slowly updating it.

I run www.firstvideoarchive.com I have approximately 1.5TB of storage and 15000TB in bandwidth each month. Feel free to PM me about getting the videos that are recorded onto the web, I will not run out of storage and as my 2.5GB downloaded so far this month would show the videos are relatively easy to find.

Last year SOAP did run out of space and had to delete some videos before championship to be able to record all the fields. I think later in the year they said they had that problem fixed.

-Mike