Network Tables

[lucas.alvarez96]

Yeah, I know everybody has asked for this before, but I really need a Network Tables library for C++. I'd used pynetworktables and had a fully functioning code with Python, but the latency while using an Axis IP cam (even while connected to my laptop's ethernet port) is too dang high! When I tried C++, it worked like a charm, but there's no network table implementation for sending simple values such as x and y coordinates. Network sockets are sort of intimidating, and I haven't the faintest on how to implement them on our DS (with C++ and the openCV library) and the robot (using java). I would be truly grateful if someone could send me a part of their sockets code, just to get an idea on how to code it.
Thanks in advance!

[virtuald]

How high is the latency? We haven't had any significant problems with it, and all of our code is implemented in python (robot + image processing + custom dashboard).

[lucas.alvarez96]

I'd reckon like 3 seconds of latency...my FPS is great, it's just that there has to be some problems with the buffer

import cv2
import urllib 
import numpy as np

stream=urllib.urlopen('http://10.25.76.11/mjpg/video.mjpg')
bytes=''
while True:
    bytes+=stream.read(16384)
    a = bytes.find('\xff\xd8')
    b = bytes.find('\xff\xd9')
    if a!=-1 and b!=-1:
        jpg = bytes[a:b+2]
        bytes= bytes[b+2:]
        i = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8),cv2.CV_LOAD_IMAGE_COLOR)
        cv2.imshow('i',i)
        if cv2.waitKey(1) ==27:
            exit(0)

I got the code from StackOverflow. I had tried using a simple VideoCapture(ip), and got it to work on C++, but Python just throws up some errors:

Traceback (most recent call last):
  File "C:\Users\Lucas\Documents\opencv\opencv\webcam.py", line 7, in <module>
    cv2.imshow("window", img)
error: C:\slave\WinInstallerMegaPack\src\opencv\modules\core\src\array.cpp:2482: error: (-206) Unrecognized or unsupported array type

And when I print out the value of "ret" (the first variable outputted by VideoCapture::read()), I get False, which indicates that there is no image being captured (duh).

Any ideas?

[virtuald]

I use this:

        vc = cv2.VideoCapture()
        vc.set(cv2.cv.CV_CAP_PROP_FPS, 1)
        
        if not vc.open('http://%s/mjpg/video.mjpg' % self.camera_ip):
            return

        while True:
            retval, img = vc.read(buffer)
            ...

One bug present in OpenCV that hasn't been fixed can be found on their bug tracker here: http://code.opencv.org/issues/2877 . If you compile your own OpenCV you can patch the bug. I've been meaning to patch it in a better way but haven't done so yet, as I don't have an axis camera easily available for testing.

[lucas.alvarez96]

Ok so I tried this:

import cv2
import numpy as np
import time

camera_ip = "10.25.76.11"

vc = cv2.VideoCapture()
vc.set(cv2.cv.CV_CAP_PROP_FPS, 1)
        
if not vc.open('http://%s/mjpg/video.mjpg' % camera_ip):
    time.sleep(0)

while True:
    retval, img = vc.read(buffer)
    cv2.imshow("img", img)
    if cv2.waitKey(20) == 27:
        break

cv2.destroyAllWindows()
exit(0)

And got this:

Traceback (most recent call last):
  File "C:/Users/Lucas/Desktop/cdch/render_stream3.py", line 14, in <module>
    retval, img = vc.read(buffer)
TypeError: <unknown> is not a numpy array

So truth be told, I'm not quite sure what's going on...

And yeah, I'd read somewhere that ffmpeg could sometimes be the source of the problem, but I'm on windows and haven't the faintest idea on how to compile from source...

I'm very sorry Dustin if the problem is to obvious, but I've been struggling with this for weeks and my team REALLY needs it for the championships...

[virtuald]

Sorry, I copy/pasted that incorrectly. You should remove the buffer from the vc.read() call for initial testing. buffer happens to be a python keyword, and I was using it as a variable, so I wasn't allocating a new image buffer each time. So it was actually something like...

            h = vc.get(cv2.cv.CV_CAP_PROP_FRAME_WIDTH)
            w = vc.get(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT)
            
            capture_buffer = np.empty(shape=(h, w, 3), dtype=np.uint8)

            while True:
                retval, img = vc.read(capture_buffer)

[lucas.alvarez96]

Yeaaaaah.....same error....

Traceback (most recent call last):
  File "C:\Users\Lucas\Desktop\cdch\render_stream3.py", line 19, in <module>
    cv2.imshow("img", img)
error: C:\slave\WinInstallerMegaPack\src\opencv\modules\core\src\array.cpp:2482: error: (-206) Unrecognized or unsupported array type

Using this code:

import cv2
import numpy as np
import time

camera_ip = "10.25.76.11"

vc = cv2.VideoCapture()
vc.set(cv2.cv.CV_CAP_PROP_FPS, 1)
        
if not vc.open('http://%s/mjpg/video.mjpg' % camera_ip):
    time.sleep(0)

h = vc.get(cv2.cv.CV_CAP_PROP_FRAME_WIDTH)
w = vc.get(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT)
capture_buffer = np.empty(shape=(h, w, 3), dtype=np.uint8)

while True:
    retval, img = vc.read(capture_buffer)
    cv2.imshow("img", img)
    if cv2.waitKey(20) == 27:
        break

cv2.destroyAllWindows()
exit(0)

[virtuald]

Interesting. Don't pass it a capture buffer then, and see what happens.

[virtuald]

Wait, the error is on the imshow. Odd. What type/shape is the image?

print img
print img.shape
print img.type

[lucas.alvarez96]

With or without buffer, there's a problem with the capture. retval returns false, and the img array returns None, so I can't print img.shape or img.type.

AttributeError: 'NoneType' object has no attribute 'shape'

[virtuald]

Originally Posted by lucas.alvarez96

With or without buffer, there's a problem with the capture. retval returns false, and the img array returns None, so I can't print img.shape or img.type.

AttributeError: 'NoneType' object has no attribute 'shape'

Odd. That's very strange. Must be something with the way your ffmpeg is compiled.

[JamesTerm]

Originally Posted by lucas.alvarez96

Yeah, I know everybody has asked for this before, but I really need a Network Tables library for C++. I'd used pynetworktables and had a fully functioning code with Python, but the latency while using an Axis IP cam (even while connected to my laptop's ethernet port) is too dang high! When I tried C++, it worked like a charm, but there's no network table implementation for sending simple values such as x and y coordinates. Network sockets are sort of intimidating, and I haven't the faintest on how to implement them on our DS (with C++ and the openCV library) and the robot (using java). I would be truly grateful if someone could send me a part of their sockets code, just to get an idea on how to code it.
Thanks in advance!

Did you try SmartCppDashboard? Its in first forge with full source to network tables using winsock2. It also uses ffmpeg to support h264 a special build to minimize latency for it... I can get link later once I get to a PC.

[lucas.alvarez96]

Oh that's right! You have to copy the ffmpeg DLL to C:\Python27 for it to work correctly. It should be included with the opencv binary distribution. It's rather odd to me that they're separate. It'll be called something like 'opencv_ffmpeg2xx.dll'

Ok Dustin, so I copied the .dll and discovered that I had an old .dll in there (246. I'm using 248). So it's still not working, but I did discover in the source folder that there's an ffmpeg folder in the 3rdparty folder which include a make file and dll files. So I'm gonna give that a try.

Did you try SmartCppDashboard? Its in first forge with full source to network tables using winsock2. It also uses ffmpeg to support h264 a special build to minimize latency for it... I can get link later once I get to a PC.

James, that would be awesome! Please post the link as soon as you can. Thanks man!

[JamesTerm]

Originally Posted by lucas.alvarez96

James, that would be awesome! Please post the link as soon as you can. Thanks man!

Here
https://www.youtube.com/watch?v=nLmviNrMers

Shows a demo of everything... and in here is a link to the first forge... I'll include here for convenience:
http://firstforge.wpi.edu/sf/projects/smartcppdashboard

Now... this code is due for an update, but we are just about ready to head out to Lonestar regionals... so after that I may get the code all updated probably in the next two weeks, but in its current state it should get you going pretty well.

Here is a sneak peak of what the newer stuff can do (not yet checked in there): https://www.youtube.com/watch?v=fccxxlvMqY0

[virtuald]

I think he's talking about this: http://firstforge.wpi.edu/sf/sfmain/...rtcppdashboard

The opencv files should be structured something like so:

C:\Python27\opencv_ffmpeg2xx.dll
C:\Python27\lib\site-packages\cv2.pyd

I custom compiled my version of ffmpeg, but at this point I don't recall if I had to do something special to get MJPG support. I'll have to go find the source tree if I still have it...