Custom Vision Processing

[jrmeza514]

Hello,

I am a member of team 2643 and we havedecided to make our own custom vision tool. We were able to get it working marvelously on a regular java project, but as well you may know FRC applications remove some of the java classes and libraries. For example, Math.atan() does no exist. Anyway, our application is dependent on the following classes: java.awt.Image , java.awt.BufferedImage, java.net.URL, java.awt.Graphics, and java.imageio.ImageIO; Here is the problem: none of these classes actually exist in an FRC java application. I have tried fetching those files one by one but the problem is that all that all those files have endless references to other files, making the task impossible. I also the tried extracting all .class files from the .jar files in my JDK directory and decompiling them to .java files and dragging them into the project's SRC directory. This would have worked except it slowed netbeans down to the point that it was inoperable. I guess that what I want is to find a way to upload a library to my FRC application from a .jar file or find an application that can tell me which files are absolutely necessary for my application to work. I anyone can provide me with a solution I
would greatly appreciate it.

-JuanMeza, Team 2643

[muaddib42]

This is not necessarily possible. The Java environment is based off of J2ME, almost any library you will use is going to be based on J2SE. On a side note the squawk MathUtils class has a lot of implementations of the J2SE math functions.

[pblankenbaker]

It is possible to do the vision processing outside of the cRIO (as a stand-alone Java application). You can:


Create a stand-alone Java application which grabs image data directly from the IP camera (it sounds like you have completed this step).
Send the computed information from your Java application back to your robot program running on the cRIO (the step you need to do).


There are several ways to send the information back. Probably the easiest method is to use the NetworkTable objects provided in the FRC plugins. Add the networktables-desktop.jar JAR file to your stand alone Java project. It can be found under the $HOME/sunspotfrcsdk/desktop-lib folder (where $HOME refers to your home directory - something like C:\Users\LOGIN_NAME if you are on a Windows system).

Once the JAR file has been added to your project you should be able to use the NetworkTable objects. Somewhere in your initialization code, you can add code similar to the following (use the IP address of your cRIO):


NetworkTable smartDashboard;

private void setupRobotComm() {
NetworkTable.setClientMode();
// NOTE: Change to the IP address of your cRIO
NetworkTable.setIPAddress("10.8.68.2");

// Get access to the smart dashboard values (allows both put and get)
smartDashboard = NetworkTable.getTable("SmartDashboard");
}


Once your image processing code has information to deliver to the robot, you can use the "put" methods to deliver the information to the robot:


private void sendRobotInfo(boolean leftHot, boolean rightHot) {
smartDashboard.putBoolean("Left Goal Hot", leftHot);
smartDashboard.putBoolean("Right Goal Hot", rightHot);
}


The NetworkTable class should deliver this information to both the robot and the smart dashboard. Within your robot code, you should be able to check the state of these values using methods like the following (make sure your key names match):


public boolean isLeftHot() {
return SmartDashboard.getBoolean("Left Goal Hot");
}


There are some good/bad things related to using this approach:


It allows more rapid image processing code development (you don't have to have a cRIO to develop your code).
It consumes WIFI bandwidth if you run your image processing code on your driver station (there may be a 7Mbps limit this year). Some teams will put a computer on the robot to avoid this - our team has not gone to this extreme yet.
The NetworkTable approach is simple, but introduces it's own delay. If it's too much for your needs, you could alternatively use UDP or TCP to deliver th e information.
You get the diagnostic output on the Smart Dashboard for free.
You will keep a lot of the image processing load off of the cRIO CPU.


Hope that helps.
Paul