Code:
cvSplit(image, b, g, r, null); // b and r are reversed for some reason
The reason b and r are "switched" is that the image is in BGR24, not RGB. The raw feed from the camera is YUV420, which FFmpeg converts to BGR24. Why BGR24? I don't know.
From what I've read, color filtering will work better in HSL/HSV, and am currently trying to figure out how to do that.