Computer playing Bejeweled Blitz project

Technical NI LabVIEW
1

As started in this thread, I’m actually working on this project. I have chosen Facebook’s Bejeweled Blitz*, like was first suggested.

I’ve worked on it a bunch so far and have made a main program with the different parts sketched out. I only have the computational part right now, but it finds matches and knows where to move what gem where. It looks at an array of numeric values, which be converted to from the screen.It doesn’t find “gap” matches yet, where there’s two gems in a row with a gap in the middle that you can fill in with a third. It only looks for a row of 2 that it can complete. It does this by looking at a theoretical position to move the current gem being tested. Then it looks around that gem to see if there’s one match with the original. If so, it looks further in the same direction to see if there’s a second match. It counts that as a full match and passes that on the the rest of the program.

Now I need to code up how it will interact with the computer. It was suggested to interact with the Windows API to control the mouse and click in the right spots to move the gems. The Library Call node will do this for you. I looked into it though, and configuring it is beyond me since I don’t know a lot about API commands. Does someone have an example or tutorial on how to use them with LabVIEW? I’ve Googled it and looked at NI’s Dev Zone, but a lot of things expect you to know about Windows API.

Also, anyone have any suggestions for looking at the screen and getting the colors of certain pixels? I know you could do a screen capture somehow and then save the file and analyze it, but I think that might slow and/or a round-about way of doing it. Again, this might take some API work.

Thanks for your help! I’ve attached the VI’s in case you’re interested. You can run them and they’ll work, I’ve put in the test screen shot into the array. Sorry that there’s not a ton of documentation yet.

*An 8x8 playing “board” that has gems you must switch to make 3 in a row or more. Then these gems disappear and more drop from the top, Connect 4 style.

Test_Shot_1.jpg
Main.vi (11.2 KB)
Move.vi (15.6 KB)
M_L1 match test (SubVI).vi (7.85 KB)


Test_Shot_1.jpg
Main.vi (11.2 KB)
Move.vi (15.6 KB)
M_L1 match test (SubVI).vi (7.85 KB)

2

MSDN is your friend :slight_smile:

http://msdn.microsoft.com/en-us/library/ms648394%28VS.85%29.aspx

This function allows you to move the mouse:


BOOL WINAPI SetCursorPos(
  __in  int X,
  __in  int Y
);

it is in the file: User32.dll
X and Y are absolute coordinates on the screen
with 0,0 being the upper left corner of the screen

It returns a boolean value.

It is a similar call to get the cursor’s position


BOOL WINAPI GetCursorPos(
  __out  LPPOINT lpPoint
)

typedef struct {
  LONG x;
  LONG y;
} POINT;

I do not know how to use the labview library call node, but this should help.

3

The clf (Call Library Function) node needs the name and/or path to the library and the function call info to be configured. I believe the example or tutorial retrieves the computer name, and would work fine as a template for your usage. If you are calling the function more than once, it is a good idea to make a subVI wrapper, since the configuration and appearance other ways get annoying.
It is possible to use GDI calls to get the window handle for other apps, and to use bitable to copy a portion of their window into your own, and then to implement GetPixel() or something similar. Possible, but certainly easy. Using a camera to capture the screen will likely have issues with aliasing, lighting, and calibration. Not a cakewalk, but good practice if you want to use the camera on the robot.

Greg McKaskle

4

Wow, that helps a whole bunch more, thanks! That function is one of what I’m looking for. I even poked around, found user32.dll, configured the node, and it worked! Screenshots of configuration attached. It certainly helped to know a little bit about C-like functions for the parameters tab.
I looked at msdn before but couldn’t find anything…thanks for the link. I looked at the other cursor functions but there wasn’t one to click the mouse/cursor, so maybe the user will have to do that quickly when prompted to.

Thanks Greg for the tip about the GDI and GetPixel calls, I’ll look them up on msdn. Yes, I’ve looked at that example you were talking about and some other ones too, but they were still a little over my head.

So let me get this straight for my sake and anyone else’s who might look at this and not know–the Call Library Function Node basically calls/uses a function in a library that you specify. A .dll file is a compressed library with many functions that can be used universally with many systems/programs. And user32.dll is a library that has many/all of the Windows API functions, which can be used by other programs to interact with windows. Aren’t they written in a unique programming language that is very similar to C/Java/most text-based languages?

EDIT: well now that I think about it, of course there wouldn’t really be a function to “click” the mouse; you’re the one that’s clicking it and the software is sensing that. But maybe if there was a function to send a universal “click signal” to all of Windows, or all the programs waiting for a click, or the general windows mouse driver, etc.

Configuration_Call_Library_Function.png
Configuration_Call_Library_Function.png834×512 22.8 KB

2_Configure_Call_Library_Function.png
2_Configure_Call_Library_Function.png870×538 21.2 KB



Configuration_Call_Library_Function.png
Configuration_Call_Library_Function.png834×512 22.8 KB

2_Configure_Call_Library_Function.png
2_Configure_Call_Library_Function.png870×538 21.2 KB

5

There actually is a function listed on MSDN, called ‘mouse_event()’

Note that this function has been superseded(and included by me for reference purposes). You should be using the second function I found, called ‘SendInput()’

Both are housed in ‘User32.dll’ so you’re fine there.

I’m not much of a LabView guy, so I’m not sure how easy it would be to implement either of these API calls, but I hope I could help either way.

6

So let me get this straight for my sake and anyone else’s who might look at this and not know–the Call Library Function Node basically calls/uses a function in a library that you specify. A .dll file is a compressed library with many functions that can be used universally with many systems/programs. And user32.dll is a library that has many/all of the Windows API functions, which can be used by other programs to interact with windows. Aren’t they written in a unique programming language that is very similar to C/Java/most text-based languages?

You seem to have it pretty straight. Libraries are a general concept found on most operating systems. They are a collection of functions that do useful things, and are hopefully documented and grouped according to their functionality. What are they written in? It depends. On the original Mac and Lisa, they were in Pascal and some assembly. C was the language developed for developing unix, and not surprisingly, most of the libraries for linux are still written in C. I believe the NeXT was built around Objective C, BeOS may have had some amount of C++. There were OSes built around Lisp, and of course before C, most OSes were written in Assembly or even lower level.

In reality, the language the functions are written in isn’t important, but the calling conventions are. Notice the dialog for clf asks you the calling conventions? That is because Windows actually has several conventions. These conventions specify how a calling function passes parameters to the called function and retrieves results. This is done differently on different computer architectures and on different OSes.

Given the information you gave LV about the parameters, it takes the wire information and pushes the parameters onto the stack according to the stdcall conventions, executes the call instruction with the address to your function in the library, then retrieves the results and does any stack or register cleanup as specified by the convention. As long as the conventions are followed, any language can reuse code from any other. To flip roles, you can build a DLL from LV so that it can be called from any program capable of calling into a Windows library.

Greg McKaskle

7

Would an application that uses such a dll require the labview runtime engine to run?

8

Yes it would. Technically, we could statically build in all the things it would need for execution and make each one stand-alone, but it isn’t that big a request. It would also help if the datadlow scheduling engine were already on the OS.

Greg McKaskle