Hololens and Augmented Reality

[MarcDub]

Here are three projects showing the first results obtained with the Microsoft Hololens and a LabVIEW toolkit in development. The Microsoft Hololens is a untethered Augmented Reality Helmet. Go to hololens.com for more details.

I am looking for feedbacks: comments, ideas, suggestions, etc. before completing the toolkit.

Introduction to the Hololens toolkit

Hololens demo with a Robot

Hololens in a NDT Application

[Blokk]

Looks very interesting! Right now I just imagined a special add-on to your toolkit, a kind of "label-up and show in AR" application. We have several large glove-boxes with hundreds of hand valves, flow controllers, pumps, pressure/temperature/etc sensors. It is very often happens that even an experienced operator spends minutes with finding a particular object in the glove boxes. Even with the help of the 2D P&ID...

I imagine such system to use in similar environments, but I am not sure about the limitations/difficulty level to create a working solution. For example, using such helmet, the software would show AR labels in the field of view... or if the user searches (speech recognition? 🙂 ) for a particular hand valve, the hololens helmet would guide him/her to the exact location...

It would be also very useful if the toolkit could load/read an existing 3D drawing (CATIA, AutoCAD, etc.). After that only some reference points would be required (corners of a glove-box for example, special markers?), and the application "would know" what is there where the user looks...

[MarcDub]

thank you Blokk for your feedback.

Currently, my idea is to have very limited capabilities in the Hololens itself. The application running on the Hololens is just a client receiving 3D objects from the LabVIEW application. Therefore, if you can load a CAD file into LabVIEW as a 3D object, it can be seen in the Hololens. The 3D object is positioned in the AR world by LabVIEW.

The question of positioning the AR world with the actual world is a good one. It is possible to use some objects (like a pipe) or the floor as scanned by the Hololens to align the worlds.

I like the application you propose. The Hololens has speech recognitions capabilities but I have not looked at it yet.

[DarkLordOfG]

Marc,

Can you visualize "live" plots of data?  I want to overlay a real-time scope-trace on a physical object.  Does this toolkit provide this capability?

John

[MarcDub]

Yes, an image can be uploaded to a window (called panel) that can be updated at a rate of at least a few hertz (depending on the image size). One of the examples provided with the provided with the library demonstrates that capability.

Let me know if you have more questions.