Archiving Recorded Regionals 2008

Competition Scouting
1

Regional Season is quickly coming along, its time to do a little organization to see how we are going to get EVERY Regional this year recorded and up on the interwebs. Last year by my rough count we got video footage from 26 regionals, with only about 7-8 of those regionals with at least 90% of the matches uploaded. Which for our community is a great achievement, with a little more organization I believe that we can get 90% of the regionals totally recorded from Match 1 to the last Final match. :ahh:

I think you all know how great a scouting resource having match footage of different robots is. But it take a lot of effort from volunteers to get all that footage up on the internet. I know that last year I recorded and parsed three regionals (New Hampshire, Waterloo, and GTR) and it took a couple hours to set up the recording and then basically took a day or more for me to go through all the footage and cut it up the upload it for each regional. This year (because I am now in University) I will only be able to record and parse Waterloo. Now that means we need at least 41 other people to volunteer their time to record (the easy part), parse (i.e. cut up the video into matches), and upload them to the different online resources (The Blue Alliance Match Archives, SOAP, Youtube, ect.)

I’ll try and keep this list updated of who is volunteering to record what (feel free to have more the one person record a regional! you can split up the hard work of parsing and uploading!)


Regional                                         Volunteer
**Week 1**
BAE Systems Granite State Regional 
Midwest Regional 
New Jersey Regional 
Oregon Regional 
St. Louis Regional 
**Week 2**
Arizona Regional 
Finger Lakes Regional                           Ed Patterson (Koko Ed, team 191)
Greater Kansas City Regional
NASA / VCU Regional 
San Diego Regional                              AJ (ay2b, team 980)
**Week 3**
Boilermaker Regional                            Mike Aalderink (mike AA, team 2015)
Brazil Regional 
Chesapeake Regional 
Connecticut Regional 
Detroit Regional 
Florida Regional 
Peachtree Regional 
Pittsburgh Regional 
Silicon Valley Regional                         AJ (ay2b, team 980)
Wisconsin Regional 
**Week 4**
Buckeye Regional                                David Thomas (blakcheez, team 964)
Lone Star Regional                              Kevin Sevcik (team 57)
Los Angeles Regional                            AJ (ay2b, team 980)
Microsoft Seattle Regional 
Oklahoma City Regional 
UC Davis Sacramento Regional 
Waterloo Regional                               Jonathan Norris (team 610)
West Michigan Regional                          Mike Aalderink (mike AA, team 2015)
Israel Regional 
**Week 5**
Bayou Regional 
Boston Regional 
Colorado Regional 
Great Lakes Regional 
Greater Toronto Regional 
Hawaii Regional 
Las Vegas Regional 
Minnesota Regional 
Palmetto Regional 
Philadelphia Regional
SBPLI Long Island Regional 
New York City Regional

(I believe the SOAP people usually will archive the Championship event, i’m sure they will find this thread and let us know if that is happening again)

I’ll post some more in-depth tutorials on how to go about recording, parsing, and uploading (and my personal opinion on formating, mpg4 FTW!) later on, but right now its more important to find someone interested in recording each of these regionals. Feel free to ask here or PM me with any questions.

2

For those who don’t know the history, the last couple years the folks at team 108 have donated a server to host FIRST match video that any one can access, called SOAP. Much of that video is recorded by the team, and some of it is uploaded by others. In the past year the guys at The Blue Alliance have built a great website taking all this recorded video and put it all into Flash video (like Youtube), and add match scoring information and tagging for teams. This lets you go through and search by team number and come up with a list of all the team’s matches. It really is a great resource for scouting, and have made going through matches on SOAP’s a lot easier.

So not to say that SOAP doesn’t have its place, but I believe the system that The Blue Alliance guys have built for online video is very polished and easy to use. So my personal opinion for this next season of Regionals is to try and use The Blue Alliance as the central hub for match video (I know that I am making a lot of work for you guys, sry :p). This is just my opinion, I’m sure I’ll hear others with their own ideas about how to centralize match footage.

Youtube… last year we heard from Dean that he wants Youtube flooded with FIRST video, and for FIRST to grow in the media and on the internet this great resource needs to be used… properly. If you go on Youtube today you can find a large amount of FIRST video, however it is spread all over the place and does not do a good enough job at linking back to one another.

To get exposer on Youtube we need to make sure that every new video is properly tagged, title, and linked by channels and community groups. So I propose that we try and find a standard tagging and naming system, and create a large group and channel for all FIRST video. Hopefully we can find a ‘standard’ and spread the word to all FIRST teams to use this format so we can get better exposer!

Formating… I know in the past the standard video format for all matches has been Windows Media Video (.wmv), but let me try and make the case for MPG4 (.mp4). .wmv files for a long time were a popular standard video file, because basically every one had a PC and it was the best compression format. But these days of mass internet video .mp4 is clearly the new standard and the only .wmv video I have on my computer are old matches (most I have converted). Every type of mass portable media player supports the format (iPods, Zunes, Cell Phones ect.). Even HD DVD’s, online movie rentals, ect. use a form of the MPG4 format. In the near future flash video will support MPG4 video (in the form of h.264), so the world is moving towards MPG4 as the standard video format. By using MPG4 as a standard for video it will make it a lot easier for people to put these great match videos where ever they want… on their computer, on their iPod, ect.

3

I can cover Buckeye this year and submit them to TBA, SOAP, and back it up on our site space.

YouTube channel just setup at http://www.youtube.com/profile?user=FRCommunity Contact me by PM, YouTube PM, or [email protected] to add videos or info.

I agree that there should be a standard tagging convention if we are to do this as a whole community. We can discuss it in this thread.

4

I can do FLR.

5

I can pull LSR again, but I think part of the problem last year was SOAP running a little low on server space to handle such a massive amount of footage. I think it’d be polite to give them a heads up that CD is making a concerted effort at archiving all the regionals.

To the suggestion that we just concentrate on TBA as the central hub… I like their format and layout and etc for the convenience, but it’s flash video. It typically looks marginally better than something filmed on a cell phone cam covered in vaseline. If TBA is willing to host better quality WMVs of all the matches in addition to the cheesy flash video, then I’m happy, otherwise, I’d rather find someone who really wants to archive the files.

6

Something to think about as far as bandwidth goes, but how complex would it be to get teams with massive amounts of excess webspace and bandwidth to host their own videos and maybe other area teams, then get these sites to link to them?

7

Ed - Make sure you bring a real VCR and not the mono cheapster one from the site or else the quality is pretty crappy.

FYI - it took me quite a long time to convert 1 regional of all matches so plan on spending a good weekend.

Ellery

8

That is what both the SOAP archives and The Blue Alliance archives (both linked above) are doing for us. They both provide hosting for Regional Match videos and a central location to find all the seasons matches. Its a lot easier to just have a centralized service to host all the footage, then to have the matches spread over a bunch of different servers.

9

On every match on The Blue Alliance you have the option to download it in the higher quality format (most of them right off SOAP last year). Maybe they need to make that more obvious on their website.

10

Well, I understand that, but someone stated above that they started running out of space. It’s an idea just in case they start runnning out of space.

11

I’ll point The Blue Alliance guys over to this thread to see if they have the capabilities to archive all the matches this season.

12

MP4 is fine on our side. (In fact, it’s much better. Hopefully when flash 9 is polished we can do away with FLV’s)

It will take more resources for compression, though. We are battling an h264 encoding problem at the moment. Does anyone know a good way to compress this stuff in real time?

13

h.264 is really processor intensive both on the compression and playback ends, and it’s definitely not designed for real-time encoding (at least by anybody other than DirecTV and the likes). I don’t think you’re going to get good performance unless you buy a specialized encoder card or have a REALLY high end system.

I recommend staying with wmv for now. The encoder performs well in real-time professionally, and is freely available. Plus I would argue it’s the more accessible than quicktime MP4.

FLV is great for easy website integration, but not much else.

14

I prefer to disagree to the pro-WMV argument. It is a dying format, more and more players support MP4 well, than Windows Media.

15

I will be able to record via DVD Boilermaker and West Michigan. I will do my best to get the video uploaded for downloads. Also inaddition I have been hosting video archives for the last 3 years. I only have most of last year and part of 2006 online. I am hunting through my DVDs looking for past years video and slowly updating it.

I run www.firstvideoarchive.com I have approximately 1.5TB of storage and 15000TB in bandwidth each month. Feel free to PM me about getting the videos that are recorded onto the web, I will not run out of storage and as my 2.5GB downloaded so far this month would show the videos are relatively easy to find.

Last year SOAP did run out of space and had to delete some videos before championship to be able to record all the fields. I think later in the year they said they had that problem fixed.

-Mike

16

I will bring this up at our next meeting, but I am guessing that 108 will be recording the regionals they are attending being New Jersey regional and Florida regional. Please wait for a response from me or Matt Keller in this thread before you put 108 down for any regional or championship. Thanks.

17

I would have to agree that WMV is a dying format, however .h264 recording is just not feasible at this point because of how processor intensive it is (you need a beast of a computer to do live .h264 recording). but quicktime .mp4 is not nearly as processor intensive, i’ll do some tests and research in the next couple weeks and provide some recommendations for recording directly to a computer (hopefully with .mp4).

18

Although if someone CAN get something running .h264 recording please please PLEASE share it with us. I just want to see what a match would look like.

Anyway, what equipment do you all use besides VCR/DVD recording from the main feed? We have a Sony Handycam DRC-DVD300 and a JVC compact VHS camcorder. I’ve got Vegas on a mediocre laptop, and at the school’s computer lab there is a machine with Vegas that’s nicer than mine.

19

I want to echo the standard format/naming conventions. It’s a nightmare to put the videos in a database/retrieval system when you have to rename them. Last year was somewhat-standardized, but there were some lapses (which are time-consuming).

…

Well I can’t speak for folks from TBA or SOAP, but it wouldn’t be too hard to implement a system commonly used for open source software distribution that goes like this: on the back end, any number of web/FTP servers mirror the same file tree, usually through automatic synchronization. On the front-end, knowing who has what files, you use some scripting to randomly send each downloader to a different one of your mirrors.

It’s essentially a form of load-balancing.

There’s also bittorrent to consider, but you’d have to bundle match files and get a good swarm going for it to be effective. That totally eliminates the strength of the current system which allows you to select individual matches, but some do want ALL of them, so it wouldn’t be a bad idea to parallel the web hosting with a torrent tracker for people who want whole regionals.

…

I’d love to help the recording efforts but I’m usually way too busy during competition. I’ll see if I get one of our members or parents to do the HI regional and let you know.

If you ever got a mirroring system up I’d be willing to offer up some of our storage/bandwidth to help out. We have way more than we’ll ever need.

20

That load-balancing type solution you mentioned wouldn’t be a bad idea. If we could get TBA, SOAP, FIRSTvideoArchive.com, along with teams/people willing to pitch in then that would be something to consider. That’s only if it’s really needed. Personally, I think it should be implemented anyway, that way it’s already there if something happens.

As for the Bittorrent, we could offer the seeds by regional. So all of FLR would be one file, Buckeye in another, etc. Championship would be a different story.