Guitar Hero Playing Robot

[flyingcrayons]

hey, does anyone know who owns the gh playing robot that was on display at the pits at CMPs? 1089 is doing a gh3 tourney, and we wanted to know if that robot (or one like it) was commercially available to rent or something like that. it would be totally awesome to have it on display there. if anyone out there has any info on that awesome robot (i beat it tho), it would be greatly appreciated

[Enigma's puzzle]

I am pretty sure it was owned by National Instruments

[Molten]

It looks like there are several of them out there. None I saw were commercially made, but the actual mechanics looked simple. I am just lost as to how the computer takes its input from the screen and translates it into the buttons. Maybe a youtube search will help you find the one you are looking for.

[RyanCahoon]

The system demonstrated on Engineering TV isn't from National Instruments, but it includes NI components. Lots of salesperson speak, but includes some useful tips. If you're looking for a build-your-own approach, many amateur individuals have posted how they made their own on YouTube, as was suggested (a couple were featured on HackedGadgets).

The basic idea isn't too far off from the autonomous programming challenge in FIRST this year. Since the objective of Guitar Hero and other similar games is to press the corresponding buttons when colored markers pass certain points on the screen, the systems typically work by running vision algorithms on those particular portions of the screen. Since each marker is uniquely colored, and is fairly high contrast, algorithms can be written to detect when a marker is passing. From there, it's just an issue of tuning the system to near enough real-time performance that the system can press the buttons on the game controller at the correct time. There are many variations as to how the game controller is actuated (pneumatic and electric solenoids, servos, hacking open the controller and electrically interfacing with the game system) and how the vision system receives the image (whether through a camera or being plugged directly into the game console's video out signal), but the concept is still essentially the same.

Good luck, sounds like a fun off season project.
--Ryan

[flyingcrayons]

thank you guys... i looked on N/I's website, and there was a blog post saying that their robot would be on display at the FIRST championship event... i dont think we'll have much luck persuading them to lend it to us, but making our own is an awesome idea, and a great challenge, i'll mention it to my team, and we'll see what happens...

once again, thanks for all the info...

[Enigma's puzzle]

If you talk to them however they may help you out if you were going to make your own. Especially with the camera reading.

[jspatz1]

I believe the NI guitar hero system at Atlanta used the column position of the moving markers, rather than the color. The dashboard they were showing in the programming area used a black & white image. Each button color is in a specific column, so it would not be necessary to discern color.

[DonRotolo]

GH Players know that sometimes every button on the screen is the same color, and sometimes the button positions are reversed. In both cases it would be easy to note the cahnge and react accordingly.

Here's a mind exercise to get yu started on building your own:

1. Sensors. You need a sensor that will detect "button" from "no button" on each 'string' of the guitar. Start simple with four sensors - you may need more later.

2. Position. Use the same TV each time. Put some masking tape on the screen across the 'strings' and note exactly where the 'buttons' pass. For example, the left-most button (yellow is it?) always passes this spot a split-second before I'm supposed to play that 'note'. Find those spots...

3. Timing. How much time between when you can reliably detect the 'button' on screen to when you have to press that button and strum the guitar? That's your processing speed, make sure your microprocessor has the power to keep up that speed easily.

4. Actuators: One type to press buttons, another to 'strum'. Solenoids? Are you willing to cut open the guitar and wire up directly? How about a guitar-simulator, that replicates the electronics instead?

5. Software: Simple: When 'button' 1 is 'seen', actuate button 1 on the guitar and hit 'strum'. repeat real fast for all strings at the same time.

Start out easy, but eventually you should be able to make a robot that will play any song, at any level, perfectly every time.

Don

[Molten]

Originally Posted by Don Rotolo

Are you willing to cut open the guitar and wire up directly? How about a guitar-simulator, that replicates the electronics instead?

This may just be me, but this would seem to take the visual appeal of the robot. I mean, yes it would play the music. And the signal that the game receives would be the same. But the observer primarily sees the buttons getting pushed. That is what they watch while the whole thing is working. If you take that away, you kind of lose the appeal.

[jspatz1]

Originally Posted by Molten

This may just be me, but this would seem to take the visual appeal of the robot. I mean, yes it would play the music. And the signal that the game receives would be the same. But the observer primarily sees the buttons getting pushed. That is what they watch while the whole thing is working. If you take that away, you kind of lose the appeal.

I agree Molten. I have thought about the various ways one might do this, and to me it is important to have a camera/sensor watching a screen rather than just intercepting the video signal, and to have a mechanical interface operating the guitar rather than interfacing electrical signals. The whole idea is to have a robot that can match what a human oppenent does, so it would greatly distract from it and be unfair for the robot electronics to be directly tied into the game or video in any way. A system like this is for entertainment, or perhaps fundraising, and all the fun is in watching the robot match what a human can do. An all-electronic system would be simpler, but would not be interesting at all to watch.

[jspatz1]

To answer flyingcrayons orginal question....the system was apparently built by a system integrator (application reseller) in CA called Cyth Systems. I assume as a sales gimick to promote themselves and the NI components they sell. NI apparently has acces to it for their own demos, maybe they gave them the compenents with the understanding they could use it. Anyway, based on what some other individuals have done, it looks like they used way more hardware power than they really needed to do this job, just so they could showcase these particular products. From his description of how it works, I would think it could be done with some much lower price range equipment. Not sure if the cheap Axis camera in the KOP is fast enough, but I'll bet the cRio, labview vision tools, and I/O boards that cam in the KOP could handle this.

[DonRotolo]

I, personally, also would not want to cut into the guitar. I'm too cheap.

An on-screen sensor array is trival compared to a vision system to recognize multiple targets. Walk before running.

Don