Using the AXIS camera (M1011) with OpenCV

[yash101]

So, I am using cv::CvCapture to get video. This code works with my internal webcam (cvCaptureFromCAM(0);. However, when I switch to reading from the net-cam (AXIS M1011), my app keeps crashing and OpenCV says that it can't find "../../modules/highgui/src/cap_ffmpeg_impl.hpp(545)"

What am I doing wring and why is it triggering?

the code is below.


//-----------------------------------------------------------------------------------------------------------
//face-tracking by Dev, 1.0
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/objdetect/objdetect.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <iostream>
#include <math.h>
#include <string>
#include <sstream>
#include <vector>
#include <WinSock.h>

using namespace std; //So I don't have to use std:: --don't worry. it's nothing big.
using namespace cv; //so I don't have to use cv:: --same here :D. Just saves time

void processDisplay( Mat processed, Mat unprocessed ); //Subroutine declaration, processDisplay
//String camIP = "xxx"; //Put your Camera MJPEG stream IP here
int capMode = 1; //capture mode: 1 for internal; 2 for AXIS M1011; 3 for AXIS M1014;
int FaceCount = 0; //variable for the nimber of faces detected
int capFailures = 0; //Failure count in camera frame fetch. CURRENTLY UNUSED
const int maxCapFailures = 10; //Max Failure count in cam frame fetch before exiting, with error code, -1. CURRENTLY UNUSED

CascadeClassifier face; //dscribes what a face looks like
CascadeClassifier eyes; //describes what eyes look like

int main( int argc, const char** argv ) { //main function
CvCapture* capture; //creates struct, "capture".*************************************************
//capture.set( capture, CV_CAP_PROP_FRAME_WIDTH, 640 ); //Sets the capture size width to 640px. Currently deactivated
//capture.set( capture, CV_CAP_PROP_FRAME_HEIGHT, 480 ); //Sets the capture size height to 480px. Currently deactivated
//cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH, 640); //Sets the capture size width to 640px. deactivated
//cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, 480); //Sets the capture size height to 480px. deactivated
CvCapture* capture2; //Creates struct, "capture2"
Mat unprocessed; //unprocessed image
Mat processed; //processed image
if( face.load( "D:\\SSDWIN7PRGFILES\\opencv\\opencv\\sources\\data \\haarcascades\\haarcascade_frontalface_alt.xml" ) ) { //loads one face cascade
if( eyes.load( "D:\\SSDWIN7PRGFILES\\opencv\\opencv\\sources\\data \\haarcascade\\haarcascade_eye_tree_eyeglasses.xml" ) ) { //loads another face cascade
cout << "I was able to load the cascades!" << endl; //message
} else { //if not loaded second cascade
cout << "Failed to load cascades!\n\n\tOptions:\n\t[ESC] to exit\n\tAny other key to try again"; //message|error analysis
if( waitKey(0) == 27 ) { //waits and updates screen. Also detects keyboard commands
return -1; //returns error code, -1;
} else { //if no key was pressed
cout << "Trying Again" << endl; //message|info
} //close brace
} //close brace
} else { //message|error
cout << "Failed to load cascades!\n\n\tOptions:\n\t[ESC] to exit\n\tAny other key to try again"; //message|error action
if( waitKey(0) == 27 ) { //wait for [ESC] key
return -1; //exit program with error code, -1
} else { //if no [ESC] key, try to configure cascade again
cout << "Trying Again" << endl; //message|info
} //closing brace
} //closing brace
if( capMode == 1 ) { //check capture mode, descibed above at variable creation
capture = cvCaptureFromCAM(0); //set camera of "capture" to cam 0
capture2 = cvCaptureFromCAM(0); //set camera of "capture2" to cam 0
//capture2 = cvCaptureFromFile("root:paradise@http://10.11.65.10/jpg/image.jpg?size=3");
cout << "I'm going to use the internal camera." << endl; //message|info
} else if( capMode == 2 || capMode == 3 ) { //if different cam mode
//if( camIP.empty() ) {
// cout << "IP not set" << endl;
//} else {
capture = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480"); //hook to network camera
capture2 = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480"); //hook to network camera
cout << "MJPEG stream opened" << endl; //message|info
//}
} //closing brace

if( capture ) { //check to see if capture was true
bool msgq = 0; //to prevent message from repeating
while( true ) { //infinite loop
unprocessed = cvQueryFrame( capture ); //gather image from cam
processed = cvQueryFrame( capture2 ); //gather image from cam
//unprocessed = imread("http://10.11.65.10/jpg/image.jpg?size=3");
//processed = imread("http://10.11.65.10/jpg/image.jpg?size=3");
if( !unprocessed.empty() ) { //check to see if camera is outputing footage
if( msgq = false ) { //make sure message doesn't repeat
cout << "The Webcam Stream has been opened properly!" << endl; //message|info
} //closing brace
processDisplay( processed, unprocessed ); //run the processing algorithm
msgq = true; //set flag to true
} else { //if cam frame was not present
cout << "Could not access the camera. Please maake sure nothing is already using the camera. Retrying" << endl; //message|error
} //closing brace
int c = waitKey(1); //+check to see if the [ESC] key was pressed
if( (char)c == 'c' ) { //|
break; //|
} //+closing brace
} //closing brace
} //closing brace
waitKey(0);
return 0; //closes program with safe exit code, 0!
} //closing brace

void processDisplay( Mat processed, Mat unprocessed ) { //processing algorithm
Mat gray; //New matrix for the grayscale image
vector<Rect> faces; //new vector to store the faces, where they are and how big they are
vector<Rect> eye; //New vector to store info about the eyes

cvtColor( processed, gray, CV_BGR2GRAY ); //converts color image to grayscale
equalizeHist( gray, gray ); //cleans the image

face.detectMultiScale( gray, faces, 1.1, 2, 0|CV_HAAR_SCALE_IMAGE, Size(30,30) ); //detects faces using a cascade
eyes.detectMultiScale( gray, eye, 1.1, 2, 0|CV_HAAR_SCALE_IMAGE, Size(30,30) ); //dtects eyes using a cascade

Point2i p1(0,240); //point
Point2i p2(640,240); //point
Point2i p3(320, 0); //point
Point2i p4(320, 480); //point
Point2i p5(0, 0); //point
Point2i p6(640, 480); //point
Point2i p7(640, 0); //point
Point2i p8(0, 480); //point
line( processed, p1, p2, Scalar(113, 37, 105), 1, CV_AA); //line
line( processed, p3, p4, Scalar(113, 37, 105), 1, CV_AA); //line
line( processed, p5, p6, Scalar(1.0), 1, CV_AA); //line
line( processed, p7, p8, Scalar(1.0), 1, CV_AA); //line

cout << "I detected " << faces.size() << " face(s)! \n\t"; //message|output

int count = 0; //counter integer

for( count = 0; count < faces.size(); count++ ) {
//Point center( faces[i].x + faces[i].width * 0.5, faces[i].y + faces[i].width * 0.5 );
Point center( faces[count].x + faces[count].width * 0.5, faces[count].y + faces[count].height * 0.5 );
ellipse( processed, center, Size( faces[count].width * 0.5, faces[count].height * 0.5 ), 0, 0, 360, Scalar(22, 127, 255 ), 2, 8, 0);
cout << "Face Number:" << count+1 << ";\tPosition: X: " << faces[count].x << " Y: " << faces[count].y << endl;
Point2i p9(320, 240);
Point2i p10(faces[count].x + faces[count].width*0.5, faces[count].y + faces[count].height*0.5);
line( processed, p9, p10, Scalar(33, 191, 152), 1, CV_AA );
}

count = 0;

for( count = 0; count < eye.size(); count++ ) {
Point centerEye( eye[count].x + eye[count].width * 0.5, eye[count].y + eye[count].height * 0.5 );
ellipse( processed, centerEye, Size( eye[count].width * 0.5, eye[count].height * 0.5 ), 0, 0, 360, Scalar( 57, 64, 192 ), 2, 8, 0 );
}

imshow( "Processed Image", processed );
imshow( "GrayScale Image (Processing)", gray );
imshow( "Original Image", unprocessed );
FaceCount = FaceCount + faces.size();
cout << endl << "I have detected " << FaceCount << "Face(s) so far!" << endl;
}

//--------------------------------------------------------------------------------------------------------------


Thank goodness that CD shrinks the CODE box! Otherwise this post would be a couple pages long :D

Thanks! ;)

[virtuald]

My guess is that you either didn't compile OpenCV with ffmpeg support, or perhaps it can't find the ffmpeg library. Check the OpenCV file to see what the code is doing at that point. You could post the entirety of the error message and context (like where it is being issued) too.

[yash101]

My guess is that you either didn't compile OpenCV with ffmpeg support, or perhaps it can't find the ffmpeg library. Check the OpenCV file to see what the code is doing at that point. You could post the entirety of the error message and context (like where it is being issued) too.

That is the entire error message. It was during the runtime. I have a screen recording of it. If you want, I can drop it into my CDN so you can watch it.

The funny thing is that the code compiles correctly. The code also works perfectly when I use cvCaptureFromCAM(0); instead of reading it from a file, with a URL.

If you change, int capMode to 2, modify the IP URLs on:

capture = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480"); //hook to network camera

capture2 = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480"); //hook to network camera,
the AXIS camera on the bot will do cascade-based vision tracking.

How do I fix this issue about FFMPEG not being correctly detected? I spent over two hours trying to modify code, find a better stream URL, etc., but couldn't find out anything, so now it is time that I ask the experts ;)


Also, what is the URL that I should use to gather the MJPEG stream to the AXIS (I think it is M1011)?

Also, should I use another drop-in command like capture = cvCaptureFromFIle("URL"); ?

Thanks for your time and effort and good luck this year ;)

[virtuald]

Are you sure it didn't say "Could not find codec parameters"? There's a similar looking message on line 556 of the file in their github repo (https://github.com/Itseez/opencv/blob/2.4/modules/highgui/src/cap_ffmpeg_impl.hpp#L556). Line 545 says 'error opening file', which implies that it cannot open the stream you've passed to it.

The uri that worked for me last year was 'http://%s/mjpg/video.mjpg'.

[yash101]

I had a feeling that that was a problem. I'll try that URL. If that doesn't work, where would I look to find the camera stream address? Also, how do I authenticate to the camera?

[virtuald]

I'd recommend disabling the authentication on the camera, then you don't have to deal with it. I'm pretty sure there's a way to enable anonymous stream viewing somewhere.

[Greg McKaskle]

If you are only acquiring images, you can turn on anonymous viewing on the camera settings and the URL will work without authentication. If you want to set other parameters, you can mimic what the web page does. And by that I mean to capture using wireshark and spoof it. It is what WPILib does. I can give the codes for FRC/FRC.

Greg McKaskle

[yash101]

I'll have to look into anonymous viewing. If anyone knows how to do it, letting me know would be great!

Thanks :)

[RufflesRidge]

I'll have to look into anonymous viewing. If anyone knows how to do it, letting me know would be great!

Thanks :)

http://wpilib.screenstepslive.com/s/3120/m/8559/l/89729-configuring-an-axis-camera

The image under Configure Users

[yash101]

I got the anonymous viewing set up, and not the steaming works like a charm. I think I get less lag than my laptop's internal camera! :D

Here's my current code. I am currently working on it, but it displays three windows, grayscale, and two instances of the original image, to be processed.


//-----------------------------------------------------------------------------------------------------------
//face-tracking by Dev, 1.0
//Copyright 2014 Devyash Lodha
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/objdetect/objdetect.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <iostream>
#include <math.h>
#include <string>
#include <sstream>
#include <vector>
#include <WinSock.h>

using namespace std; //So I don't have to use std:: --don't worry. it's nothing big.
using namespace cv; //so I don't have to use cv:: --same here :D. Just saves time

void processDisplay( Mat processed, Mat unprocessed ); //Subroutine declaration, processDisplay
//String camIP = "xxx"; //Put your Camera MJPEG stream IP here
int capMode = 1; //capture mode: 1 for internal; 2 for AXIS M1011; 3 for AXIS M1014;
int FaceCount = 0; //variable for the nimber of faces detected
int capFailures = 0; //Failure count in camera frame fetch. CURRENTLY UNUSED
const int maxCapFailures = 10; //Max Failure count in cam frame fetch before exiting, with error code, -1. CURRENTLY UNUSED

CascadeClassifier face; //dscribes what a face looks like
CascadeClassifier eyes; //describes what eyes look like

int main( int argc, const char** argv ) { //main function
CvCapture* capture; //creates struct, "capture".*************************************************
//capture.set( capture, CV_CAP_PROP_FRAME_WIDTH, 640 ); //Sets the capture size width to 640px. Currently deactivated
//capture.set( capture, CV_CAP_PROP_FRAME_HEIGHT, 480 ); //Sets the capture size height to 480px. Currently deactivated
//cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH, 640); //Sets the capture size width to 640px. deactivated
//cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, 480); //Sets the capture size height to 480px. deactivated
CvCapture* capture2; //Creates struct, "capture2"
Mat unprocessed; //unprocessed image
Mat processed; //processed image
if( face.load( "D:\\SSDWIN7PRGFILES\\opencv\\opencv\\sources\\data \\haarcascades\\haarcascade_frontalface_alt.xml" ) ) { //loads one face cascade
if( eyes.load( "D:\\SSDWIN7PRGFILES\\opencv\\opencv\\sources\\data \\haarcascade\\haarcascade_eye_tree_eyeglasses.xml" ) ) { //loads another face cascade
cout << "I was able to load the cascades!" << endl; //message
} else { //if not loaded second cascade
cout << "Failed to load cascades!\n\n\tOptions:\n\t[ESC] to exit\n\tAny other key to try again"; //message|error analysis
if( waitKey(0) == 27 ) { //waits and updates screen. Also detects keyboard commands
return -1; //returns error code, -1;
} else { //if no key was pressed
cout << "Trying Again" << endl; //message|info
} //close brace
} //close brace
} else { //message|error
cout << "Failed to load cascades!\n\n\tOptions:\n\t[ESC] to exit\n\tAny other key to try again"; //message|error action
if( waitKey(0) == 27 ) { //wait for [ESC] key
return -1; //exit program with error code, -1
} else { //if no [ESC] key, try to configure cascade again
cout << "Trying Again" << endl; //message|info
} //closing brace
} //closing brace
if( capMode == 1 ) { //check capture mode, descibed above at variable creation
//capture = cvCaptureFromCAM(0); //set camera of "capture" to cam 0
capture = cvCaptureFromFile("http://10.11.65.10/mjpg/video.mjpg");
//capture2 = cvCaptureFromCAM(0); //set camera of "capture2" to cam 0
//root:paradise
capture2 = cvCaptureFromFile("http://10.11.65.10/mjpg/video.mjpg");
cout << "I'm going to use the internal camera." << endl; //message|info
} else if( capMode == 2 || capMode == 3 ) { //if different cam mode
//if( camIP.empty() ) {
// cout << "IP not set" << endl;
//} else {
capture = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480"); //hook to network camera
capture2 = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480"); //hook to network camera
cout << "MJPEG stream opened" << endl; //message|info
//}
} //closing brace

if( capture ) { //check to see if capture was true
bool msgq = 0; //to prevent message from repeating
while( true ) { //infinite loop
unprocessed = cvQueryFrame( capture ); //gather image from cam
processed = cvQueryFrame( capture2 ); //gather image from cam
//unprocessed = imread("http://10.11.65.10/jpg/image.jpg?size=3");
//processed = imread("http://10.11.65.10/jpg/image.jpg?size=3");
if( !unprocessed.empty() ) { //check to see if camera is outputing footage
if( msgq = false ) { //make sure message doesn't repeat
cout << "The Webcam Stream has been opened properly!" << endl; //message|info
} //closing brace
processDisplay( processed, unprocessed ); //run the processing algorithm
msgq = true; //set flag to true
} else { //if cam frame was not present
cout << "Could not access the camera. Please maake sure nothing is already using the camera. Retrying" << endl; //message|error
} //closing brace
int c = waitKey(1); //+check to see if the [ESC] key was pressed
if( (char)c == 'c' ) { //|
break; //|
} //+closing brace
} //closing brace
} //closing brace
waitKey(0);
return 0; //closes program with safe exit code, 0!
} //closing brace

void processDisplay( Mat processed, Mat unprocessed ) { //processing algorithm
Mat gray; //New matrix for the grayscale image
Mat threshold; //New matrix for the thresholded image

cvtColor( processed, gray, CV_BGR2GRAY ); //converts color image to grayscale
equalizeHist( gray, gray ); //cleans the image

imshow( "Processed Image", processed );
imshow( "GrayScale Image (Processing)", gray );
imshow( "Original Image", unprocessed );
}

//---------------------------------------------------------------------------------------------------------------

[yash101]

So now that I have a working stream address, and I am able to do basic processing, How do I separate the goals from everything else? The camera is saturated from the reflected green, so I just need to create an algorithm to separate that from the rest of the stuff. Whee should I start? If anyone has some sample code, that would be appreciate, even if you want to PM me to not let anyone else see!

So currently, I am able to process the colors as CV_BRG2HSV and CV_BGR2GRAY. I am also able to imshow the images throughout the entire process!


----Different topic----
So many of us have claimed how wifi interference can cause a ton of lag in robot communication. To see what would happen, I will try this wacky test:

step 1: Ask everyone to turn off their phones and electronics, and isolate the appliances from as much interferance as possible. I'll try to get a rough estimate of the lag (with no bandwidth restrictions enabled (similar to what you'd get if you were doing onboard processing without any network usage (except feedback).

step 2: Ask everyone to turn on their phones, bluetooth, wifi and any other communications possible. I will also be running aircrack-ng on another computer to just send a ton of packets and try to cause as much disturbance as possible. This will run on the same channel as the robot communications. To manage consistency, no bandwidth restrictions would be enabled, again. I'd then check the lag times and see whether vision will be a possibility. I'll most likely publish this data to show what my experimentation came up with.

step 3: analyze and publish the results. Also, choose the pathway you wish to follow, there-on. It could be to go by onboard or offboard processing!


---Tell me if I should add another step to this. I will try to use BackTrack Linux off a pendrive iso image.

Also, this is quite overkill. You'll probably never get as much interference competition, so this will be more-of-a worst-case scenario!

What do you guys think?

[XaoS]

You might wanna take a look at thresholding techniques, specifically OpenCV's inRange() function.