So I got this HoloLens last February

FIRST General Forum
1

If you’re unaware HoloLens is Microsoft’s mixed reality headset. It’s really amazing. If you get a chance to play with one, take advantage.

I haven’t had as much time to work with it as I’d like, but I did whip up a Windows 10 Universal app version of the FRC field monitor. Last weekend I had a chance to actually use it on the field and record a couple of matches.

View from behind the scoring table.

FTA view

It’s hard to see the depth in the videos, but the display is anchored across the field and is about 7 feet high. In the first video you can see the HoloLens try to account for Walt the head ref occluding the virtual display a few times.

So, there you go, a real use case for mixed reality in FRC.

I would have loved to track robots and put their status bar and graphs above the robots but that kind of tracking is, hard. There are some thoughts on how that can be resolved though, so maybe at some point it will be possible.

Another thought I’ve had is to put the HoloLens on the driver and put a virtual version of the robot over the actual robot. With a good accelerometer/motion tracker on the robot it should be possible to keep the virtual and real robot aligned. That would have let the driver see the robot (or at least it’s virtual version) through the drawbridge or other obstacles.

It would also be interesting to use the virtual robot alone for driver practice.

If you’re curious about what the field monitor is showing, the bar across the top of each robot section represents the Ethernet status (is the Drivers Station physically plugged into the port), the IP communication status to the Drivers Station, then there’s the radio indicator which is green if the DS/FMS can talk to the radio on the robot. Finally there’s the indicator for the RoboRIO and the robot code ready status. The graphs are showing robot voltage, radio signal quality and trip time. The voltage goes yellow if it drops below 9.5 volts and red if it drops below 6.5 volts.

I’d be curious to hear what other folks might do with a HoloLens in FRC.

(Oh, and I’ve already considered that the HoloLens could project a virtual lake onto the field. So yeah, water game.)

2

Whoa, that’s absolutely awesome! Thanks so much for sharing. I’m curious, I assume you bought the developers edition?

Edit: Has anyone tried something similar with Google Glass?

3

That is awesome! Have you tried to use it from a driver’s perspective? You could have cool indicators like ready to fire, pressure, maybe even draw vision calculations.

I agree with frcguy, ive never experienced either hololens or glass, but my guess is the glass has less interference? How is the vision inside the Holocene?

4

I’d like to see a picture of you wearing this while wearing safety glasses. I’m curious how they physically match up with each other.

5

The outer case is polycarbonate, no need for safety glasses, just side shields. One of the big requests is a Z-87 certified version, virtually all of the industrial uses require it.

You can wear glasses under the HoloLens, but the closer the wave guides are to your eyes, the wider the apparent holographic field of view is so I prefer not to.

6

The big problem with using it at a drivers station right now is that the HoloLens is a self contained computer that is designed to connect via Wi-Fi. To make it FRC legal for the drivers under the current rules you’d have to disable the Wi-Fi and use a USB tether to the driver’s station.

But I’m guessing you’d like to see something like this.

As for Google Glass, it’s a different tech. HoloLens puts holograms in the real environment using binocular RBG holographic waveguides. Glass was monocular, so there is no depth perception or understanding. If I put a holographic shark in the moat, it stays there, swimming in the moat, it doesn’t mater where I move in space, the shark stays put. (Unless it’s coded to move of course.)

7

Is there any interest in seeing something like an FRC field monitor or any other FRC related apps for Glass? Gravity and I may have the opportunity to obtain a pair to develop with.

8

Interest, I just did it because it’s there. The market for an FTA/FTAA app is pretty rarified, I don’t think there are even 100 FTAs worldwide. When it’s done I’ll probably drop it into Xamarin so I can put out an Android version as well, but again, that’s just because I can not because of any market interest.

9

The number of FTAs (official or unofficial off season FTAs) with Google Glass is likely pretty small (maybe on the order of 0-5), so I wouldn’t spend a ton of time on it.

That being said, I made my own Android field monitor last year. Since Glass uses the Android SDK and all the same framework paradigms, it would be super trivial to port the logic over to Glass. The biggest constraint is the UX, since the Glass screen is very small.

I’m sure the number interested in a HoloLens or Glass monitor is incredibly high, but the number who would drop the money on said devices is fairly low.

10