Recording with the Camera

[ExarKun666]

Our team was curious on whether or not the 2009 robot's camera could record while it was on to either a TV tape or computer wirlessly, using Wind River. If it is possible, we were wondering on how we would go about doing this. If it is not possible to do it wirelessly is it possible to do it hooked up with ethernet cord, and tether? TY for any help! :]

[R.C.]

Our team was curious on whether or not the 2009 robot's camera could record while it was on to either a TV tape or computer wirlessly, using Wind River. If it is possible, we were wondering on how we would go about doing this. If it is not possible to do it wirelessly is it possible to do it hooked up with ethernet cord, and tether? TY for any help! :]

I really want to say yes, I can't find the post right now (have to go to the shop) but i'm pretty sure you can. I think team 45 used it on their 2009 robot to record video.

-RC

[tdlrali]

I know for a fact that one of the martian teams used the camera to record "video" (couple of pictures per second), and reviewed it after each match. As to whether the "video" was stored on the robot (and downloaded after the match), or recorded on a laptop during the match, I don't know.

[Russ Beavis]

Doesn't the default LabVIEW Dashboard (for the Driver Station "display") show real-time video (regardless of whether you're using LabVIEW or C on the cRIO)? You should be able to modify the LabVIEW Dashboard to store the images and somehow stitch them together later on.

[nathanww]

The simplest way to record camera video is to stream it to a computer and use a motion screen capture utility to grab it. At competitons, or situations where streaming video back to PC is impractical, you can also use the "image write" function to record each frame as a sperate image to the cRIO's flash, and then FTP into it and use a program to batch-convert the images into a video.

[ExarKun666]

Doesn't the default LabVIEW Dashboard (for the Driver Station "display") show real-time video (regardless of whether you're using LabVIEW or C on the cRIO)? You should be able to modify the LabVIEW Dashboard to store the images and somehow stitch them together later on.

I am not sure about this, I have never tried using the Driver Station "display", but I have tried to use the LabVIEW Dashboard, while the robot had Wind River code on it, and the image would not appear on my laptop, not sure if this also means it would not work on the Driver Station.

NOTE: yes I then did try the Dashboard with the robot on LabVIEW code, and it would show on my laptop. However, our team would like to keep it in Wind River, so we can still have all the functions that the robot does, instead of re-imaging to LabVIEW, and only having the default operations.

The simplest way to record camera video is to stream it to a computer and use a motion screen capture utility to grab it. At competitons, or situations where streaming video back to PC is impractical, you can also use the "image write" function to record each frame as a sperate image to the cRIO's flash, and then FTP into it and use a program to batch-convert the images into a video.

What motion screen capture utility would you recommend? Is the "image write" function something you need to write code for, or how would you do this? What do you mean by FTP? [Sorry for my lack of knowledge in this area, but any details would be great :confused: ]

[Joe Ross]

I am not sure about this, I have never tried using the Driver Station "display", but I have tried to use the LabVIEW Dashboard, while the robot had Wind River code on it, and the image would not appear on my laptop, not sure if this also means it would not work on the Driver Station.

NOTE: yes I then did try the Dashboard with the robot on LabVIEW code, and it would show on my laptop. However, our team would like to keep it in Wind River, so we can still have all the functions that the robot does, instead of re-imaging to LabVIEW, and only having the default operations.

Did you use the PCVideoServer class? I think there's an example included with WindRiver.

[ExarKun666]

Did you use the PCVideoServer class? I think there's an example included with WindRiver.

No we didn't try the PCVideoServer class. But now you bring it up, do you use the PCVideoServer class to record or take images too?

[Greg McKaskle]

The PCVideoServer class sends images on a C++ cRIO back to the dashboard. The LV framework had the equivalent code in by default, but because of deadlines, it needed to be added to the C++ code.

Once the images are send to the dashboard, you can modify the LV dashboard code to take the jpgs and add them to an AVI using the vision VIs on the PC.

I'm not at a PC, but I suspect there is an AVI building example if you search for it.

Greg McKaskle

[ExarKun666]

Well thank you for all the input on the topic, unfortunately we can't test it tomorrow because we are in the midst of looking for our dongle, and we are using it this Thursday (that is the camera recording thing), so I have to program or do whatever to make it work all in my head for our presentation on that day, so I have no trial and error leeway (this is to get a team started in FLL).:yikes:

[nathanww]

What motion screen capture utility would you recommend? Is the "image write" function something you need to write code for, or how would you do this? What do you mean by FTP? [Sorry for my lack of knowledge in this area, but any details would be great


I usually use Camstudio--it's free and gets a fairly good frame rate
"Image write" is one of the more obscure FRC vision functions that lets you store an Image object acquired from the camera to a file on flash(you can read more about it in the vision api docs)

FTP stands for File Transfer Protocol. which, as you might expect, is a system for transferring files between computers. the cRIO can act as an FTP server, meaning that it allows a remote computer to accsess it's internal flash memory. This means that if you use the image write function to record frames, you can use FTP to retrieve the images from the cRIO's memory and animate them.

[PaW]

1318 did some recording, but how/when/with what, you'll have to contact username darkmessenger88 directly (he's the programmer).

http://www.chiefdelphi.com/forums/showpost.php?p=853549

[ExarKun666]

BTW, I think I forgot to mention that we wanted to record while in teleop mode, I don't think it would make a difference, but can the recording only work in autonomous, or both autonomous and teleop??

[Greg McKaskle]

The mode has no effect.

Greg McKaskle

[ExarKun666]

Okay I'm not sure about this code in the PCVideoServer, so if you could give it a glance over tell me any fixes that are apparent to you, it would be great. :]

NOTE: Look for the line:
//----------------------INSERTED CAMERA CODE----------------------
Those are the places I put the camera code, I put it in 3 times: Once outside
my infinite while loop [used for a momentary switch], and two times in the infinite while loop once when the button state of the trigger = 1 and once when the button state of the trigger = 0. Please look at the insertion points, and tell me of any corrections that need to be made. [Side Note: I just copied the code out of RobotMain() in the CamertoDashboardExample.


#include "WPILib.h"

#include "vxWorks.h"

#include "AxisCamera.h"
#include "BaeUtilities.h"
#include "FrcError.h"
#include "PCVideoServer.h"
#include "SimpleRobot.h"

class RobotDemo : public SimpleRobot
{
RobotDrive myRobot; // robot drive system

Joystick leftstick; //left joystick initialization

Joystick rightstick; //right joystick initialization

Jaguar roller_left; //Jaguar inittialization for left roller SC

Jaguar roller_right; //Jaguar inittialization for right roller SC

Victor elevator_left; //Victor inittialization for left elevator SC

Victor elevator_right; //Victor initialization for right elevator SC

Servo servo_left; //Servo initializtion for the left servo

Servo servo_right; //Servo intializtion for the right servo



public:
RobotDemo(void):
myRobot(1, 2), // these must be initialized in the same order
leftstick(1),
rightstick(2),
roller_left(3),
roller_right(4),
elevator_left(5),
elevator_right(6),
servo_left(7),
servo_right(8)
{
GetWatchdog().SetExpiration(100);
}


void OperatorControl(void)
{
GetWatchdog().SetEnabled(true);
GetWatchdog().Feed();
// sets all the button states
unsigned char ball_button_state = 1;
unsigned char elevator = 1;

while (IsOperatorControl())
{
GetWatchdog().Feed();
roller_left.Set(1.0);
roller_right.Set(-1.0);
elevator_left.Set(-0.35);
elevator_right.Set(0.35);
//Elevator on off button
//------------------INSERTED CAMERA CODE-----------------------
SetDebugFlag ( DEBUG_SCREEN_ONLY ) ;
/* start camera task */
if (StartCameraTask(13, 0, k320x240, ROT_0) == -1) {
dprintf( LOG_ERROR,"Failed to spawn camera task; exiting. Error code %s",
GetVisionErrorText(GetLastVisionError()) );
}
dprintf(LOG_DEBUG,"Waiting for camera to initialize");
Wait(2.0);

// start up the task serving images to PC
dprintf(LOG_DEBUG,"Starting image server");
PCVideoServer pc;
dprintf(LOG_DEBUG,"Image server Running");

// this code demonstates stopping and restarting the PC image server
Wait(60.0);

dprintf(LOG_DEBUG, "Stopping image server");
pc.Stop();
Wait(10.0);

dprintf(LOG_DEBUG, "Starting image server again");
pc.Start();
Wait(999.0);

dprintf(LOG_DEBUG, "Stopping image server");
StopImageToPCServer(); //can use c or cpp call to stop

dprintf(LOG_DEBUG, "CameraToDashboardExample ending");
//-----------------------------------------------------------
if(leftstick.GetRawButton(6) != elevator)
{
if(leftstick.GetRawButton(6) == 1)
{
elevator_left.Set(0.0);
elevator_right.Set(0.0);
roller_left.Set(0.0);
roller_right.Set(0.0);
}
else
{
elevator_left.Set(-0.35);
elevator_right.Set(0.35);
roller_left.Set(1.0);
roller_right.Set(-1.0);
}
elevator = leftstick.GetRawButton(6);
}


//hold trigger to release balls, let go to bring string back up
while(rightstick.GetRawButton(1) != ball_button_state)
{
GetWatchdog().Feed();
myRobot.TankDrive(-leftstick.GetY(), -rightstick.GetY());
//starts an infinite while loop to have a mometary switch
//so all code needs to be put in the loop since it never breaks out
while(rightstick.GetRawButton(1) == 1)
{
GetWatchdog().Feed();
//----------------------INSERTED CAMERA CODE----------------------
SetDebugFlag ( DEBUG_SCREEN_ONLY ) ;
/* start camera task */
if (StartCameraTask(13, 0, k320x240, ROT_0) == -1) {
dprintf( LOG_ERROR,"Failed to spawn camera task; exiting. Error code %s",
GetVisionErrorText(GetLastVisionError()) );
}
dprintf(LOG_DEBUG,"Waiting for camera to initialize");
Wait(2.0);

// start up the task serving images to PC
dprintf(LOG_DEBUG,"Starting image server");
PCVideoServer pc;
dprintf(LOG_DEBUG,"Image server Running");

// this code demonstates stopping and restarting the PC image server
Wait(60.0);

dprintf(LOG_DEBUG, "Stopping image server");
pc.Stop();
Wait(10.0);

dprintf(LOG_DEBUG, "Starting image server again");
pc.Start();
Wait(999.0);

dprintf(LOG_DEBUG, "Stopping image server");
StopImageToPCServer(); //can use c or cpp call to stop

dprintf(LOG_DEBUG, "CameraToDashboardExample ending");
//-----------------------------------------------------------------------------
//if button is pressed realease balls if angles below met

servo_right.SetAngle(0);
servo_left.SetAngle(180);
myRobot.TankDrive(-leftstick.GetY(), -rightstick.GetY());
roller_left.Set(1.0);
roller_right.Set(-1.0);
elevator_left.Set(-0.35);
elevator_right.Set(0.35);



if(leftstick.GetRawButton(6) != elevator)
{
if(leftstick.GetRawButton(6) == 1)
{
elevator_left.Set(0.0);
elevator_right.Set(0.0);
roller_left.Set(0.0);
roller_right.Set(0.0);
}
else
{
elevator_left.Set(-0.35);
elevator_right.Set(0.35);
roller_left.Set(1.0);
roller_right.Set(-1.0);
}
elevator = leftstick.GetRawButton(6);
}



while(rightstick.GetRawButton(1) == 0)
{
GetWatchdog().Feed();
//----------------------INSERTED CAMERA CODE----------------------
SetDebugFlag ( DEBUG_SCREEN_ONLY ) ;
/* start camera task */
if (StartCameraTask(13, 0, k320x240, ROT_0) == -1) {
dprintf( LOG_ERROR,"Failed to spawn camera task; exiting. Error code %s",
GetVisionErrorText(GetLastVisionError()) );
}
dprintf(LOG_DEBUG,"Waiting for camera to initialize");
Wait(2.0);

// start up the task serving images to PC
dprintf(LOG_DEBUG,"Starting image server");
PCVideoServer pc;
dprintf(LOG_DEBUG,"Image server Running");

// this code demonstates stopping and restarting the PC image server
Wait(60.0);

dprintf(LOG_DEBUG, "Stopping image server");
pc.Stop();
Wait(10.0);

dprintf(LOG_DEBUG, "Starting image server again");
pc.Start();
Wait(999.0);

dprintf(LOG_DEBUG, "Stopping image server");
StopImageToPCServer(); //can use c or cpp call to stop

dprintf(LOG_DEBUG, "CameraToDashboardExample ending");
//-----------------------------------------------------------------------------
servo_right.SetAngle(180);
servo_left.SetAngle(0);
myRobot.TankDrive(-leftstick.GetY(), -rightstick.GetY());
roller_left.Set(1.0);
roller_right.Set(-1.0);
elevator_left.Set(-0.35);
elevator_right.Set(0.35);


if(leftstick.GetRawButton(6) != elevator)
{
if(leftstick.GetRawButton(6) == 1)
{
elevator_left.Set(0.0);
elevator_right.Set(0.0);
roller_left.Set(0.0);
roller_right.Set(0.0);
}
else
{
elevator_left.Set(-0.35);
elevator_right.Set(0.35);
roller_left.Set(1.0);
roller_right.Set(-1.0);
}
elevator = leftstick.GetRawButton(6);
}

if(rightstick.GetRawButton(1) == 1){
break;
}
}
}

ball_button_state = rightstick.GetRawButton(1);
}
GetWatchdog().Feed();
//All the possible drives with 2 joysticks, and 2 speed controllers

myRobot.TankDrive(-leftstick.GetY(), -rightstick.GetY());
}
}
};


START_ROBOT_CLASS(RobotDemo);

[byteit101]

We were able to save video from the camera to the crio's 2GBish drive, and then copy it to a computer and assemble it into a avi on a laptop. in the code we used imaqSaveImage or imaqWriteImage (look in nivision.h) to save to a folder, and just ftp'ed to ftp://10.xx.yy.2/ and copied it off after matches. we used a simple C# program that i wrote to play it back, and vidshot capture to save it to avi. here are some of ours:
http://thecatattack.org/VOB/PracticeMatch3Boilermaker2009/
http://thecatattack.org/VOB/PracticeMatch4Boilermaker2009/
http://thecatattack.org/VOB/Match4Boilermaker2009/

[biojae]

you can make the videos directly on the cRio
(it probably takes more space to store though)

AVISession imaqCreateAVI(const char* fileName, const char* compressionFilter, int quality, unsigned int framesPerSecond, unsigned int maxDataSize);
int imaqWriteAVIFrame(Image* image, AVISession session, const void* data, unsigned int dataLength);
int imaqCloseAVI(AVISession session);

[Greg McKaskle]

you can make the videos directly on the cRio
(it probably takes more space to store though)

I think you will find that the AVI functions are documented for IMAQ in general, and they are in the header file as well, but they will not work on a nonWindows OS. I'm not sure if you will get a runtime error or a compiler error.

Greg McKaskle

[biojae]

I think you will find that the AVI functions are documented for IMAQ in general, and they are in the header file as well, but they will not work on a nonWindows OS. I'm not sure if you will get a runtime error or a compiler error.

Greg McKaskle

well, i will try it out on saturday then, (my team doesn't really do offseason robot work and thats the only time that i will have with the robot until IRI), i would imagine that it would at least compile if its in the header file.

just throwing around ideas, as i have enough time to sift through the huge NIvision.h
(Really 200 defines, 150 typedefs and 50 functions in 1 file :ahh:)
i think NI should refactor that thing :]

i did have ours take pictures when a button was pushed on the joystick though

(image taken at LA regional during camera calibration time)

[byteit101]

you can make the videos directly on the cRio
(it probably takes more space to store though)

AVISession imaqCreateAVI(const char* fileName, const char* compressionFilter, int quality, unsigned int framesPerSecond, unsigned int maxDataSize);
int imaqWriteAVIFrame(Image* image, AVISession session, const void* data, unsigned int dataLength);
int imaqCloseAVI(AVISession session);

we tried that, and it does not work.
http://decibel.ni.com/content/thread/2473?tstart=30
This may be due to these functions not be expected to be used by FIRST teams (all of the standard FIRST Vision functions use JPEGs), and therefore are not found in the headerfile. This being said I will double check with R&D to see if this is the case.



Cheers,

Mark

NI FIRST Support
Also, I'm not sure how it works, but it maybe choppy, even if it did work, say you get 3 images in 10ms, then it slows down and the next 3 images are taken over 30ms. we compensated this by saving the images as picnumb auto/telop enable/disable timestamp cameraCapt.jpg (ex. 05 ae 128963 cameraCapt.jpg) and used the timestamp's to set the delay for the next photo. before we did this, we had it play at the average speed, fixed at 16ms, and you can see it slow down, and then speed back up.

[Greg McKaskle]

The AVI stuff will work on a desktop development computer running Windows. It will not work on the PPC cRIO running vxWorks.

The second part about choppy is valid, but the times listed aren't. The camera will not return images faster than one every 33ms, and even that is dependent upon the lighting, size, etc.

I'm not that knowledgeable of codecs for video encoding. Ideally, you'd select one that allows for arbitrary timings on the frames. I'm not sure which support that.

Greg McKaskle