Augmented Reality projects in our lab

25 June 2019

Augmented Reality googles

AR technologies are providing endless opportunities for research projects and critical discussion throughout the School of Computing and Information Systems. Most recently, in a collaboration between IDL and the Microsoft Research Centre for Social Natural User Interfaces (SocialNUI), the Augmented Studio project provided a breakthrough in technologically-facilitated learning.

The development, showcased at 2019’s Melbourne Knowledge Week festival, introduces a system of on-body projection mapping, to enhance learning outcomes in the physiotherapy classroom. Using data projectors and multiple Xbox Kinect devices, a projection of the human body – which can be categorically viewed in muscle groups, as a skeleton or focus on the heart – is placed onto a chosen person in the classroom.

The projection then reacts to the movement of that individual. If they are to bend and extend their elbow, so too will the image of the musculoskeletal system overlaid on their clothing. Projects like Augmented Studio have paved the way for future IDL projects, now supported by impactful technological advancements in the lab.

The work is hosted at IDL’s tech lab. The entrance to the lab is unassuming, tucked away in the west wing of the Level 5 lab. However, what appears immediately as a white-walled, empty space, is ultimately the perfect blank canvas for the future of AR research – ready to bring a new, digital world into our physical reality.

Six, white Optoma projection screens have been rigged to create an enclosed environment, providing close to 360-degrees of projection capacity – with the final floor screens to be added soon, completing the set-up. The projectors provide greater efficiency for researchers with their extensive battery life and enable enhanced results with their capacity to produce higher resolution images, at superior colour ranges.

The AR world is mobilised by pioneering technologies, which have become available to the talented team of software engineers at IDL. Alongside the state-of-the-art projectors, the studio is fitted with two OptiTrack systems, and 12 OptiTrack infrared cameras, set to track movement within the 6 x 6 metre AR space. The lab’s Microsoft Hololens is used to access AR visualisations, while the full experience is supported by the OptiTrack systems and accompanying object and human body tracking software.

To complete the picture – a full, 5.1.2 surround sound system has been installed, including 1 subwoofer and 2 Dolby Atmos speakers – creating an all-encompassing sound experience.

What’s next for the AR research?

Aside from the continuation and refinement of the Augmented Studio project, another research pursuit is focusing on facilitating Virtual Co-presence between researchers. With the support of the lab’s new OptiTrack systems, researchers are developing a means of scanning a particular researcher to create a photorealistic avatar of them, which can then be projected in another AR studio, to simulate their presence in the environment.

A full OptiTrack set-up has also been placed at Phoria in Fitzroy, the location of a globally recognised research team focused on immersive technologies, to facilitate cross-facility collaboration between researchers at both labs. With the rate of technological innovations continuing to increase, collaboration between researchers in Human-Computer Interaction will be integral in progressing the field to match the rate of new advancements in the industry.

IDL’s re-imagined Level 5 lab space has placed the group in a prime position, to continue to be a high-contributing part of the broader community of researchers developing AR technologies around the world.