Cool project idea: LV to play a game or program

[RoboMaster]

I've been thinking about this project idea; what do you think and how could it be done?

You have a LabVIEW VI play a simple computer game, analyzing the simple situations and "clicking" in the right spots to play it. Specifically, I've thought of Facebook's Bejeweled Blitz, because the computer could very quickly find rows to make. Maybe there's something easier to implement out there?

Not only would this be impressive, but it would probably be a good learning opportunity for FRC teams in advanced programing (or just more advanced than they've done so far).



Applying this to Bejeweled Blitz: (gem game where you swap two adjacent gems on a grid to make 3 in a row or more).
LabVIEW could look at the pixels of the screen (or a snapshot of the screen) and analyze the center pixel of each gem position, finding the color of the gem. In this way it would what the grid situation looks like.

To make moves, click events could be generated on gems that needed to be moved. (I'm personally not very knowledgeable about using event structures and generating events, though)

To decide what to do, every gem could be analyzed in a couple of ways:
-See if possible rows are near: look at similar colored gems diagonal from this gem. If there are two diagonals in a similar direction (up, down, right, left) or another similar gem next to the diagonal, move this gem next to the diagonal to make a row.
-See if a row would be made if this gem was moved in X direction: theoretically move the gem, see if two similar gems are next to it in a row.

The sliding down of new gems after a move would have to be considered. They take time to actually get into position and you can't test when gems are moving by. The simplest way would be to wait for a key press from the user signifying all gems have fallen and have stopped making any combos. You could continue analyzing gems that are not above the played row until after a short time limit. This wouldn't take into account any combos, though.


So what do you think? I don't have LabVIEW with me on a computer, so I can't look at what VIs I'd use or go more in depth on how I'd do it. Are there any other strategies for doing what needs to be done?