[BenBernard]

Without having looked at any of your actual code (sorry, pressed for time) I can tell you that we encountered similar behavior and solved it by opening and initializing both USBCamera instances at the beginning, then passing one or the other in to our CustomCameraServer.startAutomaticCapture method.

We made a slight modification (available gratis on our github) which stops streaming on one camera, then starts it on the other. The delay seemed to be with initializing the cameras, not starting capture.

 
public synchronized void startAutomaticCapture(USBCamera camera) {
        if (camera==null) {
            return;
        }

        if (m_autoCaptureStarted) {
            m_camera.stopCapture();
            m_camera = camera;
            m_camera.startCapture();
            return;
        }
        m_autoCaptureStarted = true;
        m_camera = camera;

        m_camera.startCapture();

        Thread captureThread = new Thread(new Runnable() {
            @Override
            public void run() {
                capture();
            }
        });
        captureThread.setName("Camera Capture Thread");
        captureThread.start();
    }