The launch of several virtual reality headsets into the consumer market in the first half of 2016 saw a flurry of new hype around this rather old idea, at least in terms of computer technology: a technical report from the Institute of Computer Graphics and Algorithms at the Vienna University of Technology saw fit to report on Virtual Reality History, Applications, Technology and Future over twenty years ago, in 1996. However, of the disadvantages of immersive virtual reality, other than oft reported visually induced “VR sickness”, is that the apparatus required to enter it covers your eyes, and completely occludes your direct view of the physical world with a computer generated one. On the other hand, in an augmented reality system, you still directly perceive physical world elements, even if they are overlaid, or annotated, with additional digital information.
One of the seminal papers in augmented reality research, Milgram, Paul & Fumio Kishino, “A taxonomy of mixed reality visual displays”, IEICE TRANSACTIONS on Information and Systems 77, no. 12 (1994): 1321-1329) describes a Mixed Reality environment as “one in which real world and virtual world objects are presented together within a single display, that is, anywhere between the extrema of the virtuality continuum”.
The paper also describes an operational definition of Augmented Reality (AR) as “any case in which an otherwise real environment is ‘augmented’ by means of virtual (computer graphic) objects…. not for lack of a better name, but simply out of conviction that the term Augmented Reality is quite appropriate for describing the essence of computer graphic enhancement of video images of real scenes” and we shall find it convenient to adopt a similar definition, although other definitions exist.
For example, as Azuma et al. (Azuma, Ronald, Yohan Baillot, Reinhold Behringer, Steven Feiner, Simon Julier, and Blair MacIntyre. “Recent advances in augmented reality.” IEEE computer graphics and applications 21, no. 6 (2001): 34-47) define it:
An AR system supplements the real world with virtual (computer-generated) objects that appear to coexist in the same space as the real world.[A]n AR system [is defined] to have the following properties:
- combines real and virtual objects in a real environment;
- runs interactively, and in real time; and
- registers (aligns) real and virtual objects with each other.
Note that this definition is not limited to any particular display technology or sensory modality.
Augmented reality is itself a form of mediated reality, or computer mediated reality. Mediated realities may themselves be thought of in terms of the extent to which information is added to an environment (augmented reality) or subtracted from an environment (which we might term a diminished reality). In addition, the notion of hyper-reality describes a system where no externally derived information is added to the system.
To implement a mixed reality system requires the presence of some sort of physical system, or apparatus, that can typically capture a visual scene, often from the viewer’s perspective, and render it back to the viewer, replete with augmentations. In a visually based system, we also need a computational system that is capable not only of registering and tracking, in real time, objects or locations within the scene, but also transforming them in some way in order to generate the augmented view of the physical reality.
In this series of posts, created as part of a scoping activity for a short unit in a new Open University introductory computing course, we’ll be stopping short of discussing fully immersive virtual environments, but we will be looking at augmented reality, exploring how digital technologies are blurring the ground in terms of the physical reality of what we see whenever we look at – or through – a screen, as well as capturing physical depictions of form and movement so that they can be rendered within mixed reality spaces. We will also consider non-visual mixed realities, such as mixed realities that we can listen to, rather than see visually.
When taken to extremes, such technologies may present us with a nightmarish, rather than compelling, vision of the future, as imagined by Keiichi Matsuda in his video short, “Hyper-Reality” [review]. Fortunately, perhaps, the physical technology required to implement such a system is still several years away!
See also: Infinity AR Augmented Reality Concept Video.
That isn’t to say, however, that frivolities such as Pokemon Go, released to global audiences at the start of July 2016, won’t have their five minutes of global appeal!
Across the posts, we will be focusing primarily the notion idea of virtual overlays or real or virtual transformations of real objects, looking at how we can overlay virtual scenes and information onto views of the real world, as well as how to get representations of physical objects into the virtual world so that they can be virtually transformed. This will include a consideration of how to capture real objects so that can be represented as faithful virtual objects which provide the basis for the virtual transformation, where the real and the virtual are combined in a composite view of the world, as well as a consideration of the apparatus required to implement such techniques.
As the posts are produced (and they may well be subject to change after posting!), I’ll add them to the list here:
Digital Worlds - Distorted Reality 
I find the Milgram et al definition of augmented reality problematic because it focuses simply on the use of one sense for augmentation. (For non positivists, there is also the question of what the reality is that we are augmenting – but that’s probably not a question to dig into at level 1.)
When thinking about mediated reality, it is worth bearing in mind that our version of reality is already mediated – for example by sunglasses, glasses, contact lenses or hearing aids. These are forms of mediation that we no longer perceive as mediation.
In the case of astronomy, our view of the ‘reality’ of much of outer space is completely mediated – we use telescopes (and sometimes glasses/contact lenses as well) to look at stars as they appeared tens or thousands of years ago.
@r3becca “When thinking about mediated reality, it is worth bearing in mind that our version of reality is already mediated – for example by sunglasses, glasses, contact lenses or hearing aids. These are forms of mediation that we no longer perceive as mediation.”
Agreed. And our indirect experience of reality through video and photos is also mediated. One of the themes that started to come out as I wandered through the posts was how the techniques used by film makers to mediate reality in time-consuming post-production activity are now becoming achievable as realtime manipulations…