Virtual Reality 1st Person Driver?

[yash101]

Virtual Reality would be something that I would be able to use quite well. I want some sort of HUD which displays distances to objects, updated at a really high rate (30+FPS). It would also be a good place for diagnostics such as robot speed, direction relative to start and much more.

It's an amazing piece of technology. It's a bit impractical, especially for FRC, though!

[BBray_T1296]

I feel like not the driver, but the guy/girl operating the manipulator/scoring device may have a competitive advantage by having a first-person view.

Take 2011 for example. A camera that gave the manipulating person that kind of view would be a great advantage. If you recall, the robot was scoring facing towards you, potentially 20 ft to your left/right, while your view is obstructed by tubes. Being able to see exactly what is going on from a sane orientation could help a lot.

[iggy_gim]

Yes, definitely the person operating the manipulator should have the VR. Even if the driver operates the driving and manipulator, the second driver could have the glasses just to help with strategy, like for example telling the driver the robots position during a dog pile. I've been working on implementing a VR glasses and I've gotten to the point where I can get the video from the robot to the glasses but at a around 5 FPS, I have to find a better way to package the data.

[sanelss]

a few years back i actually played with this idea. We had 2 axis cameras mounted on the bot for stereoscopic vision on the driver station. We used green/magenta glasses(could also use red/blue) and while it did work. it wasn't responsive enough to really be usable for a match. The low framerate and horrible latency made it be unusable. if the latency could be reduced( at least two orders of magnitude) then it may have potential but until then this type of system simply isn't worth the trouble.

[yash101]

What about a joystick button? You could push the button to download frames from the robot or something similar. This way, you don't have to always be situated with one or the other -- HUD/No HUD.

I'm sure that a HUD would be most useful with some sort of tracking system, where post-processed data could be displayed. Distances to known objects could be printed on the object. Alignment could also be displayed.
For example:

TARGET POSITION:
10 degrees right
12 feet forward

SHOOTING POSITION:
10 degrees right
Move forward 2 feet

[techhelpbb]

Originally Posted by sanelss

a few years back i actually played with this idea. We had 2 axis cameras mounted on the bot for stereoscopic vision on the driver station. We used green/magenta glasses(could also use red/blue) and while it did work. it wasn't responsive enough to really be usable for a match. The low framerate and horrible latency made it be unusable. if the latency could be reduced( at least two orders of magnitude) then it may have potential but until then this type of system simply isn't worth the trouble.

How did you send the data across the network?
As a single preprocessed stream or as 2 cameras over the network?

[sanelss]

Originally Posted by techhelpbb

How did you send the data across the network?
As a single preprocessed stream or as 2 cameras over the network?

two cameras over network. The crio doesn't have the horsepower to pre-process the streams in any meaningful way. The main factors were, and still are, latency and bandwidth. if you're ok with pretty cruddy quality you can work with bandwidth but until latency becomes addressed it's pretty much not worth even dealing with for a frc game.

[AlexanderTheOK]

I happened to do something almost exactly along the lines of what this thread is about over the past year. I wrote a nice 6 page article in SERVO on it if you want to take a look.

To answer some questions that seem to be hanging:

Without more bandwidth on the field this is not a feasible method. Over a direct LAN connection it was pulling a minimum of 18 mbps. This was just barely enough to keep a steady 30 fps at 600x400 per eye.

With the limits the FMS puts on the driver stations it's going to be either impossible to see anything or gut-wrenchingly slow. It's already a tad bit nauseating at the speed it's running.

It also isn't too easy to find an IP camera with a high FOV for cheap. The cameras I found for 40 dollars are webcams so they run through YAWCAM and hopefully later on MJPG-Streamer.

Still super fun to play with, but not the best idea for a fast paced FRC game with network constraints.