Augmented Studio presented at Melbourne Knowledge Week

30 May 2019

Students watching movement of a person with anatomical structures projected on to white clothes.

So, what if you could view how your body looked and moved internally – externally?

The University of Melbourne’s Human-Computer Interaction (IDL) brought a ground-breaking question to this year’s Melbourne Knowledge Week festival. North Melbourne’sMeat Market art space was open to the public, from 22–26 May, for a week-long showcase of some of the City of Melbourne’s most advanced, forward-thinking academic groups and research institutions.

The diverse exhibits explored emergent technologies and their impacts from multi-disciplinary perspectives. Researchers and organizations promoted innovative ways to approach the issues that will impact our future and encouraged the importance of fostering intellectual curiosity and rigour in solving these problems.

The IDL stall featured a real-time demonstration of our Augmented Studio project. Blending cutting-edge Augmented Reality (AR) functions, superior motion capture techniques and game design processes, the Augmented Studio display proved a popular attraction among festival-goers. 

Academic Staff, Software Engineers and Graduate Researchers from IDL alternated daily management of the Melbourne Knowledge Week stall, representing our wider research group throughout the event.

Members of the public stepped up to the exhibit to don a white lab coat and accurately view the results of a long-term collaboration between Microsoft and the University of Melbourne. The attendees were then asked to stand in front of the advanced projection system, supported by Unity-developed programming and Microsoft Kinect motion tracking.

With a mirror placed in front of them, the participant could view a projected musculo-skeletal system over their clothing, that reacted in real-time to replicate muscular contractions and extensions in a participant’s arms or legs.

Participants of all ages marvelled at the sight of an interactive musculo-skeletal model that scaled efficiently to match the projection of a human skeleton to the individual tracked by the motion capture system.

Eager requests for adjustments were met by enthusiastic IDL researchers available to take participants through multiple layers of the skeleton and muscular system projections, and a ‘heart-only’ layer, for use in future modifications.

Compelling discussions were generated between researchers and interested members of the public. The capacity for students to use this system as a means of amplifying education in health sciences, with initial tests ran in physiotherapy classrooms in Australia, provided many talking points around AR and e-Learning tools, for health sciences and a range of practical applications.

The team will continue to develop the Augmented Studio project to meet the advancements of a fast-paced industry, and explore the full potential of this technology. Plans to incorporate a fit-bit tracker to monitor heart-rate, as well as expand on current design features, are already underway at the IDL Lab.

You can read the latest report on Augmented Studio, authored by Dr Hoang and Mr Reinoso, this time, collaborating with Mr Hasan Ferdous, and Prof Frank Vetere from the Human-Computer Interaction.