Tech Review

This is What the Future of Mixed Reality Looks Like...

Pick a letter, any letter — Apple calls it AR, Qualcomm prefers XR, Microsoft — and more recently Magic Leap — have gone for MR. But whether Augmented, Extended, or Mixed, these immersive holographic experiences are going to be entering our own reality soon, and we best get ready.

“When you change the way you see the world, you change the world you see, and this is what has happened with the HoloLens,” said Satya Nadella as he kicked off the much-anticipated launch, over 3 years after the device was first launched in 2016.

Let’s be honest, 2016 seems like a very long time ago now. Think back, if you can, to a time when Trump wasn’t the President of the United States. That was also the first time I tried the HoloLens, and I knew then that I had experienced the first generation of a game-changing technology.

When I first tried the HoloLens in your London office, let me have a sneaky look! first time I tried the HoloLens, and I knew then that I had experienced the first generation of a game-changing technology.

Over these past three and a half years, Microsoft has been listening to their customers, said HoloLens inventor and Microsoft Technical Fellow Alex Kipman as he took his turn on stage. Those enterprise customers have enthusiastically been adopting Microsoft’s Mixed Reality technology in a variety of productivity, collaboration, training and visualization use cases while Microsoft, in turn, have delivered the entire wish list to those customers. But in unveiling the HoloLens 2, they also made a product which — although they dare not utter the C-word just yet — is starting to look like something that consumers might very well start hankering for in the not-too-distant-future. So here’s the low-down:

It’s all about the Cloud

It’s no secret that Microsoft’s Mixed Reality strategy centers on the Cloud, and the “intelligent edge”, so the HoloLens team has worked very closely with Azure to bring it all together.

“Building on the unique capabilities of the original HoloLens, HoloLens 2 is the ultimate intelligent edge device. And when coupled with existing and new Azure services, HoloLens 2 becomes even more capable, right out of the box,” says Julia White Corporate Vice President at Microsoft

More Immersive

FOV! Yes, let’s get that out of the way, Microsoft says they’ve more than doubled the Field of View on the HoloLens 2. Now, I’m one of those people who think the folks who got really hung up on the limited FOV were kind of missing the point, as most people got over it as their brain compensated for the initial weirdness after a few hours of use.

That said, having a wider FOV will undoubtedly make experiences on the device smoother and more dynamic, so I won’t argue against it. Crucially, Microsoft has done this while maintaining their industry-leading pixel density of 47 pixels per degree of sight. To top it off, they also promise that their new display system enables them to achieve these improvements in performance at low power.

But that elusive quality of “immersiveness” is not about the Field of View or Pixel Count so much as the way that the digital elements actually interact with you and the environment. And in the demo given by Julia Schartz, Senior Researcher at Microsoft, we got a glimpse of what these “instinctual interactions” feel like. This boils down to better eye-tracking — where you can trigger an action merely by directing your gaze at a specific spot, and better gesture recognition (in the demo, the device also scanned Julia’s hands, adjusting to their particular shape and size) rather than relying on the very specific “pinch” action which the original HoloLens relied on.

Users can also log in with Windows Hello enterprise-grade authentication through iris recognition, which plugs into the company’s overall message of prioritizing privacy and security as well as intuitive usability.

Better Comfort

When you’re getting as much ROI from a product as most enterprise customers get from the HoloLens, they forgive a lot of discomfort. That doesn’t mean improvements won’t be welcome though, and Microsoft has clearly put in a lot of work in making this new HMD something that people can wear for longer periods of time without getting an ugly bruise on their nose or having the headset slip around on their sweaty heads.

• every time Microsoft sells a HoloLens, it does so on the assumption that the business making that purchase is going to get some value in return.

HoloLens 2 has a much more balanced center of gravity (so less nose-heavy) through using carbon fiber, and a mechanism that lets you put it on without readjusting a crank at the back, “like putting on your favorite hat,” according to Kipman. Like the first HoloLens (and a distinct advantage compared to Magic Leap) the device also enables glasses wearers to simply slot the device in front of their spectacles. And — one of my favourite touches — when you’re done, you can just flip up the visor and switch back to the real world.

“We’ve improved the thermal management with new vapor chamber technology and accounted for the wide physiological variability in the size and shape of human heads by designing HoloLens 2 to comfortably adjust and fit almost anyone,” says White.

Greater Value

At the end of the day, every time Microsoft sells a HoloLens, it does so on the assumption that the business making that purchase is going to get some value in return. This usually comes in spades, but it can take a while, given on-boarding and customization demands. But with the accelerated rollout of Microsoft Mixed Reality applications like Dynamics 365 Remote Assist, Dynamics 365 Layout and the newly announced Dynamics 365 Guides applications in addition to the device’s in-box value, they promise to deliver value across a range of industries and use cases. Which brings us to the million-dollar question, how much will all this magic cost?

Well, anybody expecting a radical price drop will be disappointed. The HoloLens Bundle still costs about $3,500 to buy outright. On the other hand, the Mixed Reality as a Service rental packages start from as little as $125 a month, which makes them affordable even to most SMEs. This is specially true given how much effort Microsoft has put into making partner content available so that — in Kipman’s words, those enterprises can start getting the ROI from deploying HoloLens pretty much from day one.

Azure Kinect Developer Kit

This all started (many moons ago) with the Kinect, of course, which was also Kipman’s baby, and the original answer to his question of how we could make humans and computer interact in a more natural and intuitive way. Now that legacy comes full-circle with the Azure Kinect DK — a developer kit that combines the time-of-flight depth sensor developed for HoloLens 2, high-def RGB camera and spatial audio with a 7-microphone circular array that will enable development of advanced computer vision and speech solutions with Azure. This means the device not only records, but also sees, hears and understands its surroundings with unprecedented accuracy.

This translates in enabling smarter solutions for a range of use cases, such as with healthcare provider Ocuvera, which is currently using this technology to prevent patients from falling in hospitals. With Azure Kinect, the environmental precursors to a fall can be determined and a nurse notified to get to patients before they fall. Initially available in the U.S. and China, the Azure Kinect DK is available for preorder from today at $399.

Dynamics 365 Guides

Dynamics 365 Guides is a new mixed reality app available in preview starting today which enhances learning with step-by-step instructions that guide employees to the tools and parts they need and how to use them in real work situations. In addition to the experience of using Guides on HoloLens, a Guides PC app makes it easy to create interactive content, attach photos and videos, import 3D models and customize training to turn institutional knowledge into a repeatable learning tool.

Customization Program

HoloLens has already been used in many challenging working environments, from construction sites and operating rooms to the International Space Station. To do that, it had to pass the basic impact tests from several protective eyewear standards used in North America and Europe. It has been tested and found to conform to the basic impact protection requirements of ANSI Z87.1, CSA Z94.3 and EN 166. With HoloLens 2, however, they’re making that process smoother by introducing the Microsoft HoloLens Customization Program to enable customers and partners to customize HoloLens 2 to fit their environmental needs.

The first company to take advantage of that program is Trimble, one of its long-standing Mixed Reality Program partners, which unveiled the Trimble XR10, a new wearable hard hat device incorporating the HoloLens 2.

Commitment to Openess

Microsoft wrapped up by outlining its commitment to fostering an open Mixed Reality ecosystem.

“We believe that for an ecosystem to truly thrive there should be no barriers to innovation or customer choice,” said Kipman. HoloLens, he continued, will embrace the principles of open stores, open browsers and open developer platforms. Down with the “walled garden” approach favoured by Apple, in other words.

An example of what this looks like is the announcement that Mozilla will be bringing a prototype of its Firefox Reality browser to HoloLens 2. Tim Sweeney, founder and CEO of Epic Games, then took to the stage to say that Unreal Engine 4 support is coming to HoloLens. “Our success lies in the success of our partners and customers and the entire ecosystem,” concluded Kipman.

And that, in the end is probably key to how the HoloLens will eventually transition from a successful enterprise and productivity tool to a consumer must-have. In an ecosystem where every tech company and organization will be empowered to build and personalize their own digital technology, Microsoft wants to enable them to thrive in, and augment that ecosystem in directions that they might not necessarily want to invest heavily in themselves. Today’s announcement gave us a glimpse of an amazing-looking future, but that future is still very much at an early conceptual stage. If we’re ever to realize its full potential, it will require collaboration, so Microsoft’s approach makes sense.

For those who do want to get on board, you can preorder HoloLens 2starting today. It will be initially available from May in the United States, Japan, China, Germany, Canada, United Kingdom, Ireland, France, Australia and New Zealand.

For companies looking to get into Immersive technologies our VRConsultancy service offers comprehensive support in strategic deployment of Virtual, Augmented and Mixed Reality