I was thinking of using a camera mounted to a Pan/ tilt head controlled by the gyroscopes in a Virtual Reality headset. What rules would i have to worry about? The only thing that comes to mind is rules about robot control.
Please take a look at the Q&A specifically Question 115. You will have to worry about E02 and seeing the drivers eyes due to safety reasons.
1 Like
Besides the safety concerns addressed in that Q&A question, I guarantee that you couldn’t achieve this with a pan/tilt camera. VR requires very low latencies in order to be tolerable (see this post by Valve), so you’d be better off using one or two wide-angle cameras in a fixed position on the bot, and using a custom program to view them in your headset.
Even with that, you have video compression and frame rate to contend with. I can speak from personal experience that even 75 frames per second is a little too slow to be comfortable. The HTC Vive runs at 90. But with the bandwidth limit, I don’t think you’d be able to achieve a comfortable video feed.
And finally, even if you manage to get a high-quality video feed and low-latency tracking, there’s one final thing to worry about: nausea. Moving around in VR invariably causes people to get sick. Some people have a higher tolerance than others, but you or other drivers may not. And I can tell you as a former driver that I wouldn’t want to feel sick for the remainder of the match.
Anyway, I don’t mean to discourage you too much, but keep those things in mind. In the end, my personal recommendation would simply be to mount one or two wide-angle cameras, put the feeds on your driver station dashboard, and practice driving with those.
3 Likes
While not VR, you could attempt to do video feedback over “AR”. I picked up a VuFine for our driver last season so he didn’t have to keep looking down at the laptop screen to see the targeting information. In the end he didn’t like it and we didn’t use it much (personal preference), but it’s still an interesting option.
This year we’re probably just going to see how many big monitors we can stick on our drivers station instead.
2 Likes
Okay. these are exactly the answers i was looking for, thank you. Ill look into the VuFine, that looks promising. Thanks Again.
One thing to keep in mind, in past years drivers were not allowed to be wearing anything attached to the driver station prior to match start (not sure if that applies this year). The VuFine takes a few seconds to start up after you plug it in, so you have to account for that in your setup (either allowing for the delay or mounting it onto something you can put on after the starting bell).
There is a headband mount for the VuFine, so in the event that they wont just let us swap out glasses, that’s an option.
We did try the headband at one point but found it to be a bit finicky (it was too easy to accidentally bump it into the wrong position). One approach our driver tried was to mount them to a pair of “Over the Glasses” safety glasses, and then put them on over top of his regular safety glasses. We also tried mounting them to some glasses with lenses that could be removed and putting those over safety glasses.
None of the Refs we spoke to had any issues with either of these setups (provided we weren’t wearing them at the start of the match) but your mileage may vary.
1 Like
Hey there! So this has been done before. (There are probably more examples out there but this is the one I know about.)
Latency isn’t much of an issue because you can compensate for it by separating render movement and camera movement, though in certain cases it’s not even necessary.
You will probably run into a bandwidth issue though, since you only get 2 Mb/s per eye, which isn’t much even if you switch from MJPEG to H.265.
The most prominent reason to stick with a conventional setup though is that you would have to develop this, which takes time away from prototyping you may want to do. If you have the spare time, go ahead, but I’d recommend waiting until off season to try this out.
Yeah, we can’t do VR because it blocks our eyes/vision. We can do AR and HUD, which honestly are better options anyway because it’s more versatile.
The 4 Mbps bandwidth limitation is quite severe this year when compared to previous years (per the FMS White Paper) . Running 2 standard USB cams through the OM Radio is probably going to be challenging.
Okay, say i wanted to show sensor data, a countdown, and the camera on the VuPoint, like a second display, how would i go about doing that? ive never even used a second monitor. how would I tie this into the driver station? How would i keep the gaps between each object clear?
This looks much more viable than in previous years. In the past, you had to be disconnected from the driverstation during auto, meaning you had to pick this up and put it on at the start of teleop - a bit of a pain. This year though, there’s no such rule. You could be set up with the VuFine during introductions and not have to mess with it when the match starts. Definitely going to look into this a bit more. The HDMI output from the laptop may be an issue, I’m not sure if our driver station laptop has HDMI.
2 Likes
Bought a viewfine within 30 minutes of cracking open the rule book this year. As soon as I saw that we could be standing there with it on and not have to put it on after the 15 second mark I knew we could use one successfully. The two mounts that come with it are a little flaky and I definitely don’t like how easily it falls off with the magnetic retention system. Definitely going to do some taping and things to make it stay where the driver wants it.
1 Like