Using the AXIS camera (M1011) with OpenCV

[yash101]

So, I am using cv::CvCapture to get video. This code works with my internal webcam (cvCaptureFromCAM(0);. However, when I switch to reading from the net-cam (AXIS M1011), my app keeps crashing and OpenCV says that it can't find "../../modules/highgui/src/cap_ffmpeg_impl.hpp(545)"

What am I doing wring and why is it triggering?

the code is below.

PHP Code:

//-----------------------------------------------------------------------------------------------------------
//face-tracking by Dev, 1.0
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/objdetect/objdetect.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <iostream>
#include <math.h>
#include <string>
#include <sstream>
#include <vector>
#include <WinSock.h>

using namespace std;                                                            //So I don't have to use std:: --don't worry. it's nothing big.
using namespace cv;                                                                //so I don't have to use cv::  --same here :D. Just saves time

void processDisplay( Mat processed, Mat unprocessed );                            //Subroutine declaration, processDisplay
//String camIP = "xxx";                                                            //Put your Camera MJPEG stream IP here
int capMode = 1;                                                                //capture mode: 1 for internal; 2 for AXIS M1011; 3 for AXIS M1014;
int FaceCount = 0;                                                                //variable for the nimber of faces detected
int capFailures = 0;                                                            //Failure count in camera frame fetch. CURRENTLY UNUSED
const int maxCapFailures = 10;                                                    //Max Failure count in cam frame fetch before exiting, with error code, -1. CURRENTLY UNUSED

CascadeClassifier face;                                                            //dscribes what a face looks like
CascadeClassifier eyes;                                                            //describes what eyes look like

int main( int argc, const char** argv ) {                                        //main function
    CvCapture* capture;                                                            //creates struct, "capture".*************************************************
    //capture.set( capture, CV_CAP_PROP_FRAME_WIDTH, 640 );                        //Sets the capture size width to 640px. Currently deactivated
    //capture.set( capture, CV_CAP_PROP_FRAME_HEIGHT, 480 );                    //Sets the capture size height to 480px. Currently deactivated
    //cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH, 640);                //Sets the capture size width to 640px. deactivated
    //cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, 480);                //Sets the capture size height to 480px. deactivated
    CvCapture* capture2;                                                        //Creates struct, "capture2"
    Mat unprocessed;                                                            //unprocessed image
    Mat processed;                                                                //processed image
    if( face.load( "D:\\SSDWIN7PRGFILES\\opencv\\opencv\\sources\\data\\haarcascades\\haarcascade_frontalface_alt.xml" ) ) {            //loads one face cascade
        if( eyes.load( "D:\\SSDWIN7PRGFILES\\opencv\\opencv\\sources\\data\\haarcascade\\haarcascade_eye_tree_eyeglasses.xml" ) )  {                    //loads another face cascade
            cout << "I was able to load the cascades!" << endl;                    //message
        } else {                                                                //if not loaded second cascade
            cout << "Failed to load cascades!\n\n\tOptions:\n\t[ESC] to exit\n\tAny other key to try again";                            //message|error analysis
            if( waitKey(0) == 27 ) {                                            //waits and updates screen. Also detects keyboard commands
                return -1;                                                        //returns error code, -1;
            } else {                                                            //if no key was pressed
                cout << "Trying Again" << endl;                                    //message|info
            }                                                                    //close brace
        }                                                                        //close brace
    } else {                                                                    //message|error
        cout << "Failed to load cascades!\n\n\tOptions:\n\t[ESC] to exit\n\tAny other key to try again";                                //message|error action
        if( waitKey(0) == 27 ) {                                                //wait for [ESC] key
            return -1;                                                            //exit program with error code, -1
        } else {                                                                //if no [ESC] key, try to configure cascade again
            cout << "Trying Again" << endl;                                        //message|info
        }                                                                        //closing brace
    }                                                                            //closing brace
    if( capMode == 1 ) {                                                        //check capture mode, descibed above at variable creation
        capture = cvCaptureFromCAM(0);                                            //set camera of "capture" to cam 0
        capture2 = cvCaptureFromCAM(0);                                        //set camera of "capture2" to cam 0
        //capture2 = cvCaptureFromFile("root:paradise@http://10.11.65.10/jpg/image.jpg?size=3");
        cout << "I'm going to use the internal camera." << endl;                //message|info
    } else if( capMode == 2 || capMode == 3 ) {                                    //if different cam mode
        //if( camIP.empty() ) {
        //    cout << "IP not set" << endl;
        //} else {
            capture = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480");    //hook to network camera
            capture2 = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480");    //hook to network camera
            cout << "MJPEG stream opened" << endl;                                //message|info
        //}
    }                                                                            //closing brace

    if( capture ) {                                                                //check to see if capture was true
        bool msgq = 0;                                                            //to prevent message from repeating
        while( true ) {                                                            //infinite loop
            unprocessed = cvQueryFrame( capture );                            //gather image from cam
            processed = cvQueryFrame( capture2 );                                //gather image from cam
            //unprocessed = imread("http://10.11.65.10/jpg/image.jpg?size=3");
            //processed = imread("http://10.11.65.10/jpg/image.jpg?size=3");
            if( !unprocessed.empty() ) {                                        //check to see if camera is outputing footage
                if( msgq = false ) {                                            //make sure message doesn't repeat
                    cout << "The Webcam Stream has been opened properly!" << endl;                                                    //message|info
                }                                                                //closing brace
                processDisplay( processed, unprocessed );                        //run the processing algorithm
                msgq = true;                                                    //set flag to true
            } else {                                                            //if cam frame was not present
                cout << "Could not access the camera. Please maake sure nothing is already using the camera. Retrying" << endl;        //message|error
            }                                                                    //closing brace
            int c = waitKey(1);                                                    //+check to see if the [ESC] key was pressed
            if( (char)c == 'c' ) {                                                //|
                break;                                                            //|
            }                                                                    //+closing brace
        }                                                                        //closing brace
    }                                                                            //closing brace
    waitKey(0);
    return 0;                                                                    //closes program with safe exit code, 0!
    }                                                                            //closing brace

void processDisplay( Mat processed, Mat unprocessed ) {                            //processing algorithm
    Mat gray;                                                                    //New matrix for the grayscale image
    vector<Rect> faces;                                                            //new vector to store the faces, where they are and how big they are
    vector<Rect> eye;                                                            //New vector to store info about the eyes

    cvtColor( processed, gray, CV_BGR2GRAY );                                    //converts color image to grayscale
    equalizeHist( gray, gray );                                                    //cleans the image

    face.detectMultiScale( gray, faces, 1.1, 2, 0|CV_HAAR_SCALE_IMAGE, Size(30,30) );                                    //detects faces using a cascade
    eyes.detectMultiScale( gray, eye, 1.1, 2, 0|CV_HAAR_SCALE_IMAGE, Size(30,30) );                                        //dtects eyes using a cascade

    Point2i p1(0,240);                                                            //point
    Point2i p2(640,240);                                                        //point
    Point2i p3(320, 0);                                                            //point
    Point2i p4(320, 480);                                                        //point
    Point2i p5(0, 0);                                                            //point
    Point2i p6(640, 480);                                                        //point
    Point2i p7(640, 0);                                                            //point
    Point2i p8(0, 480);                                                            //point
    line( processed, p1, p2, Scalar(113, 37, 105), 1, CV_AA);                    //line
    line( processed, p3, p4, Scalar(113, 37, 105), 1, CV_AA);                    //line
    line( processed, p5, p6, Scalar(1.0), 1, CV_AA);                            //line
    line( processed, p7, p8, Scalar(1.0), 1, CV_AA);                            //line
    
    cout << "I detected " << faces.size() << " face(s)! \n\t";                    //message|output

    int count = 0;                                                                //counter integer

    for( count = 0; count < faces.size(); count++ ) {                            
        //Point center( faces[i].x + faces[i].width * 0.5, faces[i].y + faces[i].width * 0.5 );
        Point center( faces[count].x + faces[count].width * 0.5, faces[count].y + faces[count].height * 0.5 );
        ellipse( processed, center, Size( faces[count].width * 0.5, faces[count].height * 0.5 ), 0, 0, 360, Scalar(22, 127, 255 ), 2, 8, 0);
        cout << "Face Number:" << count+1 << ";\tPosition: X: " << faces[count].x << " Y: " << faces[count].y << endl;
        Point2i p9(320, 240);
        Point2i p10(faces[count].x + faces[count].width*0.5, faces[count].y + faces[count].height*0.5);
        line( processed, p9, p10, Scalar(33, 191, 152), 1, CV_AA );
    }

    count = 0;

    for( count = 0; count < eye.size(); count++ ) {
        Point centerEye( eye[count].x + eye[count].width * 0.5, eye[count].y + eye[count].height * 0.5 );
        ellipse( processed, centerEye, Size( eye[count].width * 0.5, eye[count].height * 0.5 ), 0, 0, 360, Scalar( 57, 64, 192 ), 2, 8, 0 );
    }

    imshow( "Processed Image", processed );
    imshow( "GrayScale Image (Processing)", gray );
    imshow( "Original Image", unprocessed );
    FaceCount = FaceCount + faces.size();
    cout << endl << "I have detected " << FaceCount << "Face(s) so far!" << endl;
}

//-------------------------------------------------------------------------------------------------------------- 


Thank goodness that CD shrinks the CODE box! Otherwise this post would be a couple pages long

Thanks!

[virtuald]

My guess is that you either didn't compile OpenCV with ffmpeg support, or perhaps it can't find the ffmpeg library. Check the OpenCV file to see what the code is doing at that point. You could post the entirety of the error message and context (like where it is being issued) too.

[yash101]

Originally Posted by virtuald

My guess is that you either didn't compile OpenCV with ffmpeg support, or perhaps it can't find the ffmpeg library. Check the OpenCV file to see what the code is doing at that point. You could post the entirety of the error message and context (like where it is being issued) too.

That is the entire error message. It was during the runtime. I have a screen recording of it. If you want, I can drop it into my CDN so you can watch it.

The funny thing is that the code compiles correctly. The code also works perfectly when I use cvCaptureFromCAM(0); instead of reading it from a file, with a URL.

If you change, int capMode to 2, modify the IP URLs on:

PHP Code:

capture = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480");    //hook to network camera

capture2 = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480");    //hook to network camera 


,
the AXIS camera on the bot will do cascade-based vision tracking.

How do I fix this issue about FFMPEG not being correctly detected? I spent over two hours trying to modify code, find a better stream URL, etc., but couldn't find out anything, so now it is time that I ask the experts


Also, what is the URL that I should use to gather the MJPEG stream to the AXIS (I think it is M1011)?

Also, should I use another drop-in command like capture = cvCaptureFromFIle("URL"); ?

Thanks for your time and effort and good luck this year

[virtuald]

Are you sure it didn't say "Could not find codec parameters"? There's a similar looking message on line 556 of the file in their github repo. Line 545 says 'error opening file', which implies that it cannot open the stream you've passed to it.

The uri that worked for me last year was 'http://%s/mjpg/video.mjpg'.

[yash101]

I had a feeling that that was a problem. I'll try that URL. If that doesn't work, where would I look to find the camera stream address? Also, how do I authenticate to the camera?

[virtuald]

I'd recommend disabling the authentication on the camera, then you don't have to deal with it. I'm pretty sure there's a way to enable anonymous stream viewing somewhere.

[Greg McKaskle]

If you are only acquiring images, you can turn on anonymous viewing on the camera settings and the URL will work without authentication. If you want to set other parameters, you can mimic what the web page does. And by that I mean to capture using wireshark and spoof it. It is what WPILib does. I can give the codes for FRC/FRC.

Greg McKaskle

[yash101]

I'll have to look into anonymous viewing. If anyone knows how to do it, letting me know would be great!

Thanks

[RufflesRidge]

Originally Posted by yash101

I'll have to look into anonymous viewing. If anyone knows how to do it, letting me know would be great!

Thanks

http://wpilib.screenstepslive.com/s/...an-axis-camera

The image under Configure Users

[yash101]

I got the anonymous viewing set up, and not the steaming works like a charm. I think I get less lag than my laptop's internal camera!

Here's my current code. I am currently working on it, but it displays three windows, grayscale, and two instances of the original image, to be processed.

PHP Code:

//-----------------------------------------------------------------------------------------------------------
//face-tracking by Dev, 1.0
//Copyright 2014 Devyash Lodha
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/objdetect/objdetect.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <iostream>
#include <math.h>
#include <string>
#include <sstream>
#include <vector>
#include <WinSock.h>

using namespace std;                                                            //So I don't have to use std:: --don't worry. it's nothing big.
using namespace cv;                                                                //so I don't have to use cv::  --same here :D. Just saves time

void processDisplay( Mat processed, Mat unprocessed );                            //Subroutine declaration, processDisplay
//String camIP = "xxx";                                                            //Put your Camera MJPEG stream IP here
int capMode = 1;                                                                //capture mode: 1 for internal; 2 for AXIS M1011; 3 for AXIS M1014;
int FaceCount = 0;                                                                //variable for the nimber of faces detected
int capFailures = 0;                                                            //Failure count in camera frame fetch. CURRENTLY UNUSED
const int maxCapFailures = 10;                                                    //Max Failure count in cam frame fetch before exiting, with error code, -1. CURRENTLY UNUSED

CascadeClassifier face;                                                            //dscribes what a face looks like
CascadeClassifier eyes;                                                            //describes what eyes look like

int main( int argc, const char** argv ) {                                        //main function
    CvCapture* capture;                                                            //creates struct, "capture".*************************************************
    //capture.set( capture, CV_CAP_PROP_FRAME_WIDTH, 640 );                        //Sets the capture size width to 640px. Currently deactivated
    //capture.set( capture, CV_CAP_PROP_FRAME_HEIGHT, 480 );                    //Sets the capture size height to 480px. Currently deactivated
    //cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH, 640);                //Sets the capture size width to 640px. deactivated
    //cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, 480);                //Sets the capture size height to 480px. deactivated
    CvCapture* capture2;                                                        //Creates struct, "capture2"
    Mat unprocessed;                                                            //unprocessed image
    Mat processed;                                                                //processed image
    if( face.load( "D:\\SSDWIN7PRGFILES\\opencv\\opencv\\sources\\data\\haarcascades\\haarcascade_frontalface_alt.xml" ) ) {            //loads one face cascade
        if( eyes.load( "D:\\SSDWIN7PRGFILES\\opencv\\opencv\\sources\\data\\haarcascade\\haarcascade_eye_tree_eyeglasses.xml" ) )  {                    //loads another face cascade
            cout << "I was able to load the cascades!" << endl;                    //message
        } else {                                                                //if not loaded second cascade
            cout << "Failed to load cascades!\n\n\tOptions:\n\t[ESC] to exit\n\tAny other key to try again";                            //message|error analysis
            if( waitKey(0) == 27 ) {                                            //waits and updates screen. Also detects keyboard commands
                return -1;                                                        //returns error code, -1;
            } else {                                                            //if no key was pressed
                cout << "Trying Again" << endl;                                    //message|info
            }                                                                    //close brace
        }                                                                        //close brace
    } else {                                                                    //message|error
        cout << "Failed to load cascades!\n\n\tOptions:\n\t[ESC] to exit\n\tAny other key to try again";                                //message|error action
        if( waitKey(0) == 27 ) {                                                //wait for [ESC] key
            return -1;                                                            //exit program with error code, -1
        } else {                                                                //if no [ESC] key, try to configure cascade again
            cout << "Trying Again" << endl;                                        //message|info
        }                                                                        //closing brace
    }                                                                            //closing brace
    if( capMode == 1 ) {                                                        //check capture mode, descibed above at variable creation
        //capture = cvCaptureFromCAM(0);                                            //set camera of "capture" to cam 0
        capture = cvCaptureFromFile("http://10.11.65.10/mjpg/video.mjpg");
        //capture2 = cvCaptureFromCAM(0);                                        //set camera of "capture2" to cam 0
        //root:paradise
        capture2 = cvCaptureFromFile("http://10.11.65.10/mjpg/video.mjpg");
        cout << "I'm going to use the internal camera." << endl;                //message|info
    } else if( capMode == 2 || capMode == 3 ) {                                    //if different cam mode
        //if( camIP.empty() ) {
        //    cout << "IP not set" << endl;
        //} else {
            capture = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480");    //hook to network camera
            capture2 = cvCaptureFromFile("http://10.11.65.10/axis-cgi/mjpeg/video.cgi?camera=1&resolution=640x480");    //hook to network camera
            cout << "MJPEG stream opened" << endl;                                //message|info
        //}
    }                                                                            //closing brace

    if( capture ) {                                                                //check to see if capture was true
        bool msgq = 0;                                                            //to prevent message from repeating
        while( true ) {                                                            //infinite loop
            unprocessed = cvQueryFrame( capture );                            //gather image from cam
            processed = cvQueryFrame( capture2 );                                //gather image from cam
            //unprocessed = imread("http://10.11.65.10/jpg/image.jpg?size=3");
            //processed = imread("http://10.11.65.10/jpg/image.jpg?size=3");
            if( !unprocessed.empty() ) {                                        //check to see if camera is outputing footage
                if( msgq = false ) {                                            //make sure message doesn't repeat
                    cout << "The Webcam Stream has been opened properly!" << endl;                                                    //message|info
                }                                                                //closing brace
                processDisplay( processed, unprocessed );                        //run the processing algorithm
                msgq = true;                                                    //set flag to true
            } else {                                                            //if cam frame was not present
                cout << "Could not access the camera. Please maake sure nothing is already using the camera. Retrying" << endl;        //message|error
            }                                                                    //closing brace
            int c = waitKey(1);                                                    //+check to see if the [ESC] key was pressed
            if( (char)c == 'c' ) {                                                //|
                break;                                                            //|
            }                                                                    //+closing brace
        }                                                                        //closing brace
    }                                                                            //closing brace
    waitKey(0);
    return 0;                                                                    //closes program with safe exit code, 0!
    }                                                                            //closing brace

void processDisplay( Mat processed, Mat unprocessed ) {                            //processing algorithm
    Mat gray;                                                                    //New matrix for the grayscale image
    Mat threshold;                                                                //New matrix for the thresholded image

    cvtColor( processed, gray, CV_BGR2GRAY );                                    //converts color image to grayscale
    equalizeHist( gray, gray );                                                    //cleans the image

    imshow( "Processed Image", processed );
    imshow( "GrayScale Image (Processing)", gray );
    imshow( "Original Image", unprocessed );
}

//--------------------------------------------------------------------------------------------------------------- 


[yash101]

So now that I have a working stream address, and I am able to do basic processing, How do I separate the goals from everything else? The camera is saturated from the reflected green, so I just need to create an algorithm to separate that from the rest of the stuff. Whee should I start? If anyone has some sample code, that would be appreciate, even if you want to PM me to not let anyone else see!

So currently, I am able to process the colors as CV_BRG2HSV and CV_BGR2GRAY. I am also able to imshow the images throughout the entire process!


----Different topic----
So many of us have claimed how wifi interference can cause a ton of lag in robot communication. To see what would happen, I will try this wacky test:

step 1: Ask everyone to turn off their phones and electronics, and isolate the appliances from as much interferance as possible. I'll try to get a rough estimate of the lag (with no bandwidth restrictions enabled (similar to what you'd get if you were doing onboard processing without any network usage (except feedback).

step 2: Ask everyone to turn on their phones, bluetooth, wifi and any other communications possible. I will also be running aircrack-ng on another computer to just send a ton of packets and try to cause as much disturbance as possible. This will run on the same channel as the robot communications. To manage consistency, no bandwidth restrictions would be enabled, again. I'd then check the lag times and see whether vision will be a possibility. I'll most likely publish this data to show what my experimentation came up with.

step 3: analyze and publish the results. Also, choose the pathway you wish to follow, there-on. It could be to go by onboard or offboard processing!


---Tell me if I should add another step to this. I will try to use BackTrack Linux off a pendrive iso image.

Also, this is quite overkill. You'll probably never get as much interference competition, so this will be more-of-a worst-case scenario!

What do you guys think?

[XaoS]

You might wanna take a look at thresholding techniques, specifically OpenCV's inRange() function.