From Magic Lenses to Magic Mirrors and Back Again

In recent years, commercial outdoor advertising has made increasing use of screen based digital signage. These can be used for video based advertising campaigns as well as “carousel” style displays where the same screen can be used to display different adverts in turn. But in a spirit of playfulness, they may also be used as magic lens style displays, similar in kind to the handheld magic lens applications described in the post “Magic Lenses” and See-Through Displays. In 2014, the Pepsi Max “Unbelievable” ad campaign by Abbott Mead Vickers BBDO tricked passengers waiting in London bus shelters into think a customised bus shelter had a transparent side wall, when it fact it was a large magic lens – the Pepsi Max “Unbelievable Bus Shelter”.

Magic lenses provide both a view of the world in front of the display as well as mediated, augmented or transformed version of it. But what if we replace the idea of a lens with that of a mirror, that augments the scene captured by a front-mounted, user facing camera?

Another part of the Pepsi Max “Unreality” campaign replaced a real mirror with a “magic mirror” that transformed the “reflection” seen by the subject by replacing their face with a virtually face-painted version of it:

Reference: Campaign, Pepsi Max “unbelievable” by Abbott Mead Vickers BBDO.

Just as mobile phone provide a convenient device for viewing the scene directly in front of the user via a screen, with all that entails in terms of re-presenting the scene digitally, front mounted cameras on smart phones allow the user to display a live video feed of their own face on the screen, essentially using the user-facing camera+live video display combination as a mirror. But can such things also be used as magic mirrors?

Indeed they can. Several cosmetics manufacturers already publish make-up styling applications that show the effect of applying different styles of make-up selected by the user. The applications rely on identifying particular facial features, such as lips, or eyes, and then allow the use to apply the make-up virtually. (You will see how this face-capture works in another post.)

Another application, ModiFace, offers a similar range of features.

For an academic take on how an augmented reality make-up application can be used for make-up application tutorial purposes, see de Almeida, D. R. O., Guedes, P. A., da Silva, M. M. O., e Silva, A. L. B. V., do Monte Lima, J. P. S., & Teichrieb, V. (2015, May). Interactive Makeup Tutorial Using Face Tracking and Augmented Reality on Mobile Devices. In Virtual and Augmented Reality (SVR), 2015 XVII Symposium on (pp. 220-226). IEEE.

In much the same way that the Pepsi Max bus shelter used a large size display as a magic lens, so to can human size displays be used to implement magic mirrors.

Once again, the fashion industry has made use of full length magic mirrors to help consumers “try on” clothes using augmented reality. The mirror identifies the customer and then overlays their “reflection” with the items to be tried on. The following video shows the FXGear FXMirror being used as part as a shop floor fitting room.

EXERCISE: Read the blurb about the FXGear FXMirror. What data is collected about users who model clothes using the device? How might such data be used?

EXERCISE: How else have marketers and advertisers used augmented and mediated reality? Try searching through various marketing trade/industry publications to find reports of recent campaigns using such techniques. If you find any, please provide a quick review of them, along with a link, in the comments.

Augmented Reality Apps for the Design Conscious

When the 2013 Ikea catalogue was first released at the start of August 2012, as part of a campaign developed in association with the McCann advertising agency, it was complemented by an augmented reality application that allowed customers to place catalogue items as if in situ in their own homes. Each year since then, the augmented reality app has been updated with the latest catalogue items, demonstrating Ikea’s ongoing commitment to this form of marketing.

For an early report, see for example: Wired – So Smart: New Ikea App Places Virtual Furniture in Your Home, August 2013.

Perhaps not surprisingly, the use of augmented reality in the context of interior design extends far beyond just an extension of the Ikea catalogue.

One of the drawbacks of the current generation of augmented reality interior design applications is the low quality of the rendering of the digital 3D object. As we shall see elsewhere, the higher powered computer processors available in today’s desktop and laptop computers, compared to mobile devices, means that it is becoming possible to render photorealistic objects in a reasonable amount of time with a personal computer. However, meeting the realtime rendering requirement of augmented reality apps, as well as the ability to ensure that that the rendered object is appropriately shaded given the lighting conditions of the environment and the desired location of the artificial object, presents further technological challenges.

EXERCISE: read the Accenture report from 2014 Life on the digital edge: How augmented reality can enhance customer experience and drive growth and then answer the following questions:

• what does the report describe as “one of the main goals of any retailer’s digital investment”? How do they claim augmented reality might achieve that goal? To what extent do you think that claim is realistic? What obstacles can you think of that might stand in the way of achieving such a goal using augmented or mediated reality?
• according to the report, how might augmented reality be used in retail? The report was published in 2014 – can you find any recent examples of augmented reality being used in ways described in the report? Is it being used for retail in ways not identified in the report?
• what does the report identify as the possible business value benefits of using augmented reality? In that section, a table poses the question “What augmented reality use case would increase your likelihood of purchasing the product?”. Can you find one or more current or historical examples of the applications described? Do such applications seem to be being used more – or less – frequently in recent times?

A lot of hype surrounds artificial reality although in many respects is value other than as a novelty are yet to be determined. To what extent do you think augmented reality applications are a useful everyday contribution to the marketer’s toolkit, and to what extent are they simply a marketing novelty fit only for short lived campaigns? What are the challenges to using such applications as part of an everyday experience?