Hmm, finding contours from a 320x240 image shouldn't be taking that long. What Raspberry PI are you using? There will be a big difference between 1 and 3. There are also more capable ARM based computers out there. We're using the ODROID C2 this year for real time video streaming at it is handling 3 320x240@30fps and 1 640x480@30fps H.264 streams at about 50% CPU usage.
A couple of recommendations, based on what you've shared:
- Get frames from the camera in YUYV, not MJPEG. MJPEG requires addition CPU overhead while only saving USB bandwidth. Unless you have more than 4 cameras, this isn't going to be a concern
- Downsample the image before processing it. Both JPEG and YUYV store color information at half resolution anyways, so if you're doing hue based processing you're not actually losing much here. Half the resolution = 4x faster processing
- Filter your image. You can use OpenCV's morphological operations to filter out noise in a binary image. Little specks can happen due to noise and can cause a lot of performance overhead.
- Tune your OpenCV method parameters. A lot of OpenCV's methods take parameters that impact runtime. Canny is a good example, wide threshold values will find a lot of potential candidates, but this means a lot of processing overhead.
Distributed approaches to problem solving can be really technically rewarding and have a big payoff, but be aware of how much additional complexity this introduces in the system.