Deploying a FRC Java project via command line interface?

[ItzWarty]

Hi,

I was wondering if anyone were aware of how one could deploy code to a robot through another application.
I'm essentially looking for being able to run the following in command prompt, and have compilation and deployment done for me:
> "pathToExecutable" maybe something "pathToRobotCodeNetbeansProject" anything else

Thanks,

[ItzWarty]

So yesterday I had my last final. This morning I started working on this stuff again, and apparently, it's not so complicated.

I used API Monitor to look at the CreateProcessA/CreateProcessW calls Netbeans was making. The maximum captured string size isn't big enough, so you'll have to write another program to read whatever's being printed (or replace the executable with one that just prints out the arguments). I've copy and pasted one at the bottom of this post. It's in C++.

But anyways, the command line arguments for building a project were as follows:
$JAVA_PATH
$JAVAC_PATH
$NB_PROJ_DIR
$NB_PROJ_NAME
$WPILIBJ_DIR
$SUNSPOTFRCSDK_DIR
$ENTRYPOINTJAVAFILE_PATH

If you plan to run any of these commands, ensure your current directory is set to $NB_PROJ_DIR, or the last 2 steps fail.
These calls will probably change whenever "FRC Java" is updated. I didn't include anything on the process of verification... I'm going to look for that stuff now, and update this post accordingly. I'm not sure how badly you can brick the cRIO by screwing up here - I'd advise for people to leave the job of replacing jvm and squawk.out files to Netbeans, which is actually well tested.

Delete $NB_PROJ_DIR\build
Delete $NB_PROJ_DIR\suite
Delete $NB_PROJ_DIR\j2meclasses

Create $NB_PROJ_DIR\build
Create $NB_PROJ_DIR\suite
Create $NB_PROJ_DIR\j2meclasses

$JAVAC_PATH -d "$NB_PROJ_DIR\build" -classpath "$NB_PROJ_DIR\build;$WPILIBJ_DIR\src" -target 1.2 -bootclasspath $SUNSPOTFRCSDK_DIR\lib\squawk_device.jar -g -source 1.3 $ENTRYPOINTJAVAFILE_PATH

This just compiles your project and outputs stuff to $NB_PROJ_DIR\build

$SUNSPOTFRCSDK_DIR\bin\preverify -d j2meclasses -classpath $SUNSPOTFRCSDK_DIR/lib/squawk_device.jar;$SUNSPOTFRCSDK_DIR/lib/WPILibJ/src;;;j2meclasses build

This just compiles your project and outputs stuff to $NB_PROJ_DIR\j2meclasses
I actually have no clue what this does - I'm a C# person, and I sort of don't know the internals of Java, much.

$JAVA_PATH -XX:CompileCommand=exclude,com/sun/squawk/Method.getParameterTypes -XX:CompileCommand=exclude,com/sun/squawk/SymbolParser.getSignatureTypeAt -XX:CompileCommand=exclude,com/sun/squawk/SymbolParser.stripMethods -Xmx256M -classpath $SUNSPOTFRCSDK_DIR\bin\romizer_classes.jar;$SUNSPOTFRCSDK_DIR\bin\squawk.jar;$SUNSPOTFRCSDK_DIR\bin\squawk_device_classes.jar;$SUNSPOTFRCSDK_DIR\bin\translator_classes.jar com.sun.squawk.Romizer -nobuildproperties -nobuildproperties -suitepath:$SUNSPOTFRCSDK_DIR/cRIO -boot:squawk -metadata -lnt -strip:d -cp:suite\$NB_PROJ_NAME_1.0.0.jar -endian:big -o:image "$NB_PROJ_DIR\suite\$NB_PROJ_NAME_1.0.0.jar"

This "ROMizes" your project. I'd assume that means doing whatever precompilations they do (so that you don't have to JIT and cache everything when your robot starts up?)

Netbeans connects to the robot through FTP.
The authentication is anonymous ("USER .." "PASS ..")

RETR /FRC_ImageVersion.ini

Probably to verify the image version of the crio, or some frc firmware.

RETR /ni-rt/system/FRC_JavaVM.out

Probably verifies JVM that is on robot

RETR /ni-rt/system/squawk.out

Probably verifies the compiled squawk vm. No clue what the difference is between this and javavm.

RETR /ni-rt/system/squawk.suite

This has something that is also "ROMified". Squawk's the name of the JVM.
Pretty sure this is just another verification step

STOR /ni-rt/system/robot.suite

$NB_PROJ_DIR/suite/image.suite is uploaded replacing /ni-rt/system/robot.suite

NB then connects to the robot and sends something (i forget the exact string) like "!RUN!", which, I assume, restarts the robot?

The following program was injected into Netbeans to capture whatever CreateProcessA's parameters were.
You'll have to use Microsoft Detours http://research.microsoft.com/en-us/projects/detours/
The program was compiled with Visual Studio 11.

#define DETOURS_X86
#include "Logger.h"
#include <Windows.h>
#include <detours.h>

using namespace ItzWarty;
using namespace std;

// WINBASEAPI
// BOOL
// WINAPI
// CreateProcessA(
//     __in_opt    LPCSTR lpApplicationName,
//     __inout_opt LPSTR lpCommandLine,
//     __in_opt    LPSECURITY_ATTRIBUTES lpProcessAttributes,
//     __in_opt    LPSECURITY_ATTRIBUTES lpThreadAttributes,
//     __in        BOOL bInheritHandles,
//     __in        DWORD dwCreationFlags,
//     __in_opt    LPVOID lpEnvironment,
//     __in_opt    LPCSTR lpCurrentDirectory,
//     __in        LPSTARTUPINFOA lpStartupInfo,
//     __out       LPPROCESS_INFORMATION lpProcessInformation
//     );

typedef BOOL (WINAPI* PFunctionCreateProcessA)(
    LPCSTR lpApplicationName,
    LPSTR lpCommandLine,
    LPSECURITY_ATTRIBUTES lpProcessAttributes,
    LPSECURITY_ATTRIBUTES lpThreadAttributes,
    BOOL bInheritHandles,
    DWORD dwCreationFlags,
    LPVOID lpEnvironment,
    LPCSTR lpCurrentDirectory,
    LPSTARTUPINFOA lpStartupInfo,
    LPPROCESS_INFORMATION lpProcessInformation
);

PFunctionCreateProcessA realCreateProcessA = NULL;
PFunctionCreateProcessA trampCreateProcessA = NULL;

BOOL
WINAPI
MyCreateProcessA(
    LPCSTR lpApplicationName,
    LPSTR lpCommandLine,
    LPSECURITY_ATTRIBUTES lpProcessAttributes,
    LPSECURITY_ATTRIBUTES lpThreadAttributes,
    BOOL bInheritHandles,
    DWORD dwCreationFlags,
    LPVOID lpEnvironment,
    LPCSTR lpCurrentDirectory,
    LPSTARTUPINFOA lpStartupInfo,
    LPPROCESS_INFORMATION lpProcessInformation
);

BOOL APIENTRY DllMain(HMODULE hModule, DWORD ul_reason_for_call, LPVOID lpReserved)
{
	switch (ul_reason_for_call)
	{
		case DLL_PROCESS_ATTACH:
		{
            Logger::Initialize("C:\\dargon.maestro_log.txt");
            Logger::GetOutputStream(LL_NOTICE) << "Dargon.Maestro injected with hModule " << hModule << endl;
            Logger::GetOutputStream(LL_NOTICE) << "Waiting for Kernel32.dll load to hook CreateProcessA " << hModule << endl;

	        HMODULE hModuleKernel32;
	        while((hModuleKernel32 = GetModuleHandleA("Kernel32.dll")) == NULL) Sleep(10);
            Logger::GetOutputStream(LL_NOTICE) << "Kernel32.dll module loaded.  hmodule: " << hModuleKernel32 << endl;
            Logger::GetOutputStream(LL_NOTICE) << "" << hModule << endl;
    
            Logger::GetOutputStream(LL_NOTICE) << "Finding CreateProcessA Address" << endl;
            realCreateProcessA = (PFunctionCreateProcessA)GetProcAddress(hModuleKernel32, "CreateProcessA");
            Logger::GetOutputStream(LL_NOTICE) <<  " -> pCreateProcessA: " << realCreateProcessA << endl;
            
            Logger::GetOutputStream(LL_NOTICE) << "Detouring CreateProcessA " << endl;
            trampCreateProcessA = (PFunctionCreateProcessA)DetourFunction((PBYTE)realCreateProcessA,(PBYTE)&MyCreateProcessA); 
            Logger::GetOutputStream(LL_NOTICE) << "  -> Done!" << endl;
		}
	}
	return true;
}

BOOL WINAPI MyCreateProcessA(LPCSTR lpApplicationName, LPSTR lpCommandLine, LPSECURITY_ATTRIBUTES lpProcessAttributes,
                             LPSECURITY_ATTRIBUTES lpThreadAttributes, BOOL bInheritHandles, DWORD dwCreationFlags,
                             LPVOID lpEnvironment, LPCSTR lpCurrentDirectory, LPSTARTUPINFOA lpStartupInfo,
                             LPPROCESS_INFORMATION lpProcessInformation)
{
    string commandLine(lpCommandLine);
    Logger::GetOutputStream(LL_NOTICE) << "It attempted to launch command line " << commandLine << endl;
    return trampCreateProcessA(lpApplicationName, lpCommandLine, lpProcessAttributes, lpThreadAttributes, bInheritHandles, dwCreationFlags,
                               lpEnvironment, lpCurrentDirectory, lpStartupInfo, lpProcessInformation);
}

It's output is as follows:

Thu Jun 07 14:29:34 2012| NOTICE| Dargon.Maestro injected with hModule 0FCC0000
Thu Jun 07 14:29:34 2012| NOTICE| Waiting for Kernel32.dll load to hook CreateProcessA 0FCC0000
Thu Jun 07 14:29:34 2012| NOTICE| Kernel32.dll module loaded.  hmodule: 74720000
Thu Jun 07 14:29:34 2012| NOTICE| 0FCC0000
Thu Jun 07 14:29:34 2012| NOTICE| Finding CreateProcessA Address
Thu Jun 07 14:29:34 2012| NOTICE|  -> pCreateProcessA: 74731062
Thu Jun 07 14:29:34 2012| NOTICE| Detouring CreateProcessA 
Thu Jun 07 14:29:34 2012| NOTICE|   -> Done!
Thu Jun 07 14:29:39 2012| NOTICE| It attempted to launch command line "C:\Program Files (x86)\Java\jdk1.6.0_21\bin\javac.exe" -d C:\Users\ItzWarty\sunspotfrcsdk\lib\WPILibJ\build -classpath C:\Users\ItzWarty\sunspotfrcsdk\lib\WPILibJ\build -target 1.2 -bootclasspath C:\Users\ItzWarty\sunspotfrcsdk\lib\squawk_device.jar -g -source 1.3 @C:\Users\ItzWarty\AppData\Local\Temp\files6967706139761371308
Thu Jun 07 14:29:42 2012| NOTICE| It attempted to launch command line C:\Users\ItzWarty\sunspotfrcsdk\bin\preverify -d j2meclasses -classpath C:/Users/ItzWarty/sunspotfrcsdk/lib/squawk_device.jar;;;;j2meclasses build
Thu Jun 07 14:29:47 2012| NOTICE| It attempted to launch command line "C:\Program Files\Mercurial\hg.exe" version
Thu Jun 07 14:29:56 2012| NOTICE| It attempted to launch command line "C:\Program Files\Mercurial\hg.exe" status -marduC --repository C:\Robotics2011\RoboticsJavaRepository --cwd C:\Robotics2011\RoboticsJavaRepository C:\Robotics2011\RoboticsJavaRepository\ReboundRumble
Thu Jun 07 14:29:56 2012| NOTICE| It attempted to launch command line "C:\Program Files\Mercurial\hg.exe" resolve -l --repository C:\Robotics2011\RoboticsJavaRepository --cwd C:\Robotics2011\RoboticsJavaRepository C:\Robotics2011\RoboticsJavaRepository\ReboundRumble
Thu Jun 07 14:29:59 2012| NOTICE| It attempted to launch command line "C:\Program Files (x86)\Java\jdk1.6.0_21\bin\javac.exe" -d "C:\Users\ItzWarty\Documents\Visual Studio 11\Projects\Diag1072\NetbeansProjects\DummyProject\build" -classpath "C:\Users\ItzWarty\Documents\Visual Studio 11\Projects\Diag1072\NetbeansProjects\DummyProject\build;C:\Users\ItzWarty\sunspotfrcsdk\lib\WPILibJ\src" -target 1.2 -bootclasspath C:\Users\ItzWarty\sunspotfrcsdk\lib\squawk_device.jar -g -source 1.3 "C:\Users\ItzWarty\Documents\Visual Studio 11\Projects\Diag1072\NetbeansProjects\DummyProject\src\edu\wpi\first\wpilibj\templates\CameraTest.java"
Thu Jun 07 14:30:01 2012| NOTICE| It attempted to launch command line C:\Users\ItzWarty\sunspotfrcsdk\bin\preverify -d j2meclasses -classpath C:/Users/ItzWarty/sunspotfrcsdk/lib/squawk_device.jar;C:/Users/ItzWarty/sunspotfrcsdk/lib/WPILibJ/src;;;j2meclasses build
Thu Jun 07 14:30:02 2012| NOTICE| It attempted to launch command line "C:\Program Files (x86)\Java\jdk1.6.0_21\jre\bin\java.exe" -XX:CompileCommand=exclude,com/sun/squawk/Method.getParameterTypes -XX:CompileCommand=exclude,com/sun/squawk/SymbolParser.getSignatureTypeAt -XX:CompileCommand=exclude,com/sun/squawk/SymbolParser.stripMethods -Xmx256M -classpath C:\Users\ItzWarty\sunspotfrcsdk\bin\romizer_classes.jar;C:\Users\ItzWarty\sunspotfrcsdk\bin\squawk.jar;C:\Users\ItzWarty\sunspotfrcsdk\bin\squawk_device_classes.jar;C:\Users\ItzWarty\sunspotfrcsdk\bin\translator_classes.jar com.sun.squawk.Romizer -nobuildproperties -nobuildproperties -suitepath:C:/Users/ItzWarty/sunspotfrcsdk/cRIO -boot:squawk -metadata -lnt -strip:d -cp:suite\CameraTestProject_1.0.0.jar -endian:big -o:image suite\CameraTestProject_1.0.0.jar

[Slix]

I think what you're looking for is ant. Netbeans uses the ant build system to build and deploy the code to the robot. Netbeans just uses ant commands, which are already defined to do the tasks you mentioned above.

To build and deploy the code from my vim setup, I used ant. I don't remember exactly what shell commands I used because I deleted my setup, but you might be able to figure it out by typing "ant help". I remember that Netbeans' build window prints the ant shell command it's using before executing it.

Also see the project.xml file in the nbproject folder (sample from my team's repo). (There should also be something similar displayed as a GUI in the project settings in Netbeans). It defines the build action as "jar-app" and the run action as "deploy" then "run". So building and deploying the code and watching it run should be as simple as:

ant jar-app
ant deploy
ant run

But I don't remember how team number setting factors in.

I'm impressed that you went to the trouble of injecting code into Netbeans to reverse-engineer the build/deploy process. It's interesting. Did you figure out what it does on FTP via Wireshark?

[ItzWarty]

Originally Posted by Slix

I think what you're looking for is ant. Netbeans uses the ant build system to build and deploy the code to the robot. Netbeans just uses ant commands, which are already defined to do the tasks you mentioned above.

To build and deploy the code from my vim setup, I used ant. I don't remember exactly what shell commands I used because I deleted my setup, but you might be able to figure it out by typing "ant help". I remember that Netbeans' build window prints the ant shell command it's using before executing it.

Also see the project.xml file in the nbproject folder (sample from my team's repo). (There should also be something similar displayed as a GUI in the project settings in Netbeans). It defines the build action as "jar-app" and the run action as "deploy" then "run". So building and deploying the code and watching it run should be as simple as:

ant jar-app
ant deploy
ant run

But I don't remember how team number setting factors in.

I'm impressed that you went to the trouble of injecting code into Netbeans to reverse-engineer the build/deploy process. It's interesting. Did you figure out what it does on FTP via Wireshark?

For some League of Legends reverse engineering, as well as the stuff I did with Netbeans, I use rohitlab's API Monitor - it lets you see calls such as connect, send, recv, etc.

[CatalystV7]

This looks pretty awesome. Does anyone have anything like this for C++?