Merged VR: Augmented Reality Cubed

There is considerable development activity at the high end of hardware and content creation for virtual reality and augmented reality, as well as such AR aliases as mixed reality, extended reality and others.

Most industry investment is aimed at leading-edge gaming and industrial application development. However, the low-end of the market is also worth looking at, to see how people and institutions without big budgets -- consumers and education -- might adopt these technologies sooner, rather than waiting years for all the advancements to trickle down to lower price points.

An example of AR done on a more human-friendly scale is Merge VR. Its founders seem to have taken the hint from The Graduate ("One word: plastics"), added software and a bit of creativity, and then applied it to AR.

First, Merge VR built a soft plastic headset (which they call "goggles") that fits any smartphone, even the largest. The optics worked well for me. The physical controls to adjust the optics are easy to use, the headset is comfortable to wear, the plastic is easy to clean, and it is inexpensive (US$60 retail in mid August 2017). Two indirect touch buttons provide limited in-app touch control while the smartphone is mounted in the headset. Cord management is easy -- both for keeping the smartphone powered and for plugging in headphones.

It comes in a variety of colors; I have seen at least 10 colors in Merge VR's offices. The device looks friendly, not high-tech. What's more, the headset has met the safety requirements of a broad swath of regulators for use by kids as young as 10 years old.

So far, so good. Merge VR deserves credit for its clever design and use of materials, but what about tech?

A Deceptively Simple Cube

A front-panel insert, when installed, keeps the inside of the headset dark for VR applications, especially when it's used with smaller smartphones. For AR applications, wearers can remove the insert and expose the smartphone's camera through proper orientation of the smartphone in the headset.

There is huge industry-wide investment in developing transparent displays for AR. Transparent displays promise to deliver full field of vision (FoV) graphic overlays -- or at least as much field of vision as wraparound sunglasses afford. The challenge is that creating totally clear displays is very hard; so is creating full FoV displays. Current transparent displays are somewhat dark; wearing them is very much like wearing sunglasses indoors.

Another way to achieve transparency is to use the smartphone camera to transmit the image to the phone's display, but cameras are still much lower resolution than looking at reality through a physical lens -- especially smartphone cameras.

While camera resolution is increasing, a single smartphone camera cannot provide high-quality stereoscopic vision. In this early stage of AR, I think "through the lens" viewing wins over transparent displays, simply because smartphone-based AR viewers can be built and sold at mass market consumer prices.

What can a smartphone based, single-camera AR solution do?

Smartphones now have enough processing power to run many classes of video image recognition in real time. The graphics processing power used to deliver mobile games also can create real-time 3D overlays that track objects in a video stream.

The process of aligning graphics onto video artifacts is called "registration." Modern smartphones have the compute power to do three-dimensional registration on three axes of rotation.

Running video image recognition and registration in real time without consuming all a smartphone's batter power requires simplifying the image recognition task.

Merge VR designed a plastic artifact specifically for easy real-time video image recognition to support simple, low-cost education and entertainment AR apps. The artifact is a cube, called, simply enough, the "Merge Cube."

The Merge Cube is 72mm (2.83 inches) on a side and easy for a child or adult to hold. It is manufactured with unique, easily differentiated patterns embossed into each side. The flat tops of the patterns are reflective silver; the space between patterns is matte grey.