Mixed and virtual reality for engineering applications – Hololens and Oculus Rift
Our company was lucky to get hold on a couple of Microsoft Hololens devices short time ago. Perhaps we were one of the first companies in Norway? A couple of weeks later I was lucky to try out yet another device as well. This time the Oculus Rift running some prototypes made by our industry partner Ceetron in Norway. Even though either Hololens or Oculus are on our short-term product roadmaps for Sesam and Nauticus, I thought it could be interesting to write a bit about my experiences with this relatively new technology.
At the moment many of the Hololens-projects are focusing towards consumer-applications like gaming and fun. However, the device (or the whole Windows holographic platform) may have some really interesting aspects for a company like DNV GL working in the maritime, oil&gas and energy market segments.
If you have not heard about the Hololens-device before, I can tell you that it is a pair of powerful “glasses” shown in the picture below. When you wear them, it feels like you have turned into a sort of RoboCop. They are quite smooth to wear, but you will probably get a bit stiff if you were them too long. The 3D surround system in the Hololens is awesome.
Basic information about the Hololens and holograms
Holograms can be explained in a very physical way, but I will keep it simple and let you know that it is somewhat similar to the projected bodies you for instance can see in various sci-fi movies like Star Wars. There is a “virtual” 3D-object (an animated body, a shape or similar) projected into the real world. You can see it in 3D, it looks very real, but you cannot feel it.
The Hololens is a so-called Augmented Reality (AR) device and differs from other well-known Virtual Reality devices like Oculus Rift, Magic Leap and so on. Sometimes we also refer to “mixed reality”, so it is a mix of virtual objects mapped into the real physical world. The HoloLens can be considered as a face-computer, because absolutely all the hardware and sensors are embedded into the device. It is using a technique called “spatial mapping” to sense the room you are inside and map the walls, floor, ceiling, tables, chairs and so on. For a person like me with structural engineering background, it looks like the device is constantly trying to “mesh” the room you are located in (like a finite element mesh). It is using the spatial map to create the mesh, so that it knows where it is allowed to project virtual 3D objects (holograms). The speed and dynamics of the meshing was impressive and the way you interact with the device through the Cortana speech-engine or finger gestures was simply amazing. Within a few minutes I understood how to draw up holograms from the onboard library and project them to walls and tables in my room. Suddenly my whole living room was decorated with a holographic 84” flat-screen on the wall, some cooking receipts on the kitchen workbench, a tiger on the floor and a monkey eating bananas on the table. Cool!
The Oculus Rift device on the other hand, was also impressive. The level of detail of the 3D-models and the way you can “dive” into the model or environment was really amazing. However, it had some disadvantages. It is cabled and it only provides a virtual reality. Thus, you cannot see what is “real” around you and I ended up banging the head in the wall, crashing with a table and so on.
Potential usage areas for AR and VR in the world of ship and offshore design
But most of our colleagues and customers of DNV GL are engineers or surveyors. They are spending their time designing huge structures, analyzing them or inspecting them. So what kind of usage areas can we have for AR and VR devices in the future?
First of all, I think the devices need to be “industrialized”. In other words they have to become far more robust than the Hololens which will probably break into thousand pieces if you loose it on the floor or get water and oil on it. The devices also need to be a bit cheaper and perhaps even more powerful in order to show really huge structural models or environments.
But once that problem is solved, we are probably good to go.
The visionary scenario
Think about the following scenario:
- A ship surveyor climbs into the cargo hold and turn on his AR-device
- The device quickly does spatial mapping of the interior of the cargo hold
- It looks up in it´s “digital twin” and find the corresponding 3D-model with for instance notifications from the class society or analysis results like Finite Element or code-check results
- The surveyor can map the analysis-results (hologram) to the real structure (reality). This is his “mixed reality world”
- He can communicate with some experts in the approval center or design office
- Together they can sort out if the findings are critical, i.e. if they are located on highly stressed areas or similar
- The surveyor can use the onboard camera or video-recorder on the AR-device and use the recordings to annotate the hologram or virtual model
I know this scenario is a bit visionary, but I strongly believe we will get there some time in the future. We just have to get the ball rolling and try out the technology with some small-scale mock-ups of cargo-holds and some demonstrators of the applications. At the moment we just got some simple prototypes up and running and we are also exploring two programming models in the Hololens; the Unity gaming engine approach and the Universal Windows Platform approach. Luckily we have a bunch of very detailed Unity-models from our Survey Simulator which was made some years ago.
Short time ago, DNV GL also had some really innovative demos at their innovation days in Singapore. A mock-up of a cargo-hold inspection via drones were demonstrated. The same type of scenario could be done with the AR-device as well….hopefully. I look forward to post the results of our prototyping with Hololens at a later stage!