Project overview

Teleconsultation refers to the electronic communication between a clinician and patient for the purpose of diagnostic or therapeutic advice. Teleconsultations are particularly useful in situations where face-to-face consultation may be difficult or impossible. To date, tele-consultation typically occurs by video conferencing systems (such as Skype) via web cameras. While these systems are helpful for enabling communication, they do not adequately support the diagnosis and treatment of physical ailments, especially those that require touch and movement, as is common in physiotherapy.

Physiotherapists assess, diagnose and treat people with movement problems. Their assessments and diagnoses rely on close observations and hands-on work with patients to detect subtle differences in body movement, such as a lack of balance in squats, abnormal distribution of weight on an affected foot, or limitations in range of movement for joints. Treatments are typically based on exercises and education to help patients improve their movements and resume their normal lifestyle.

The aim of this project is to understand and improve the video consultation experience between a patient and their physiotherapist. We extend the experience through the use of wearable technology to support bodily interactions. We seek to better understand how physiotherapists assess and treat patients during video consultation and how interactive technologies can enhance the capabilities of physiotherapists and make video consultations more effective.

The innovation is based on an analysis of existing tele-consultation practises by physiotherapists assessing patients for lower body movement. The video consultation is augmented with a wearable ‘smartsox’, called SoPhy, to assist in the diagnosis and treatment of lower limb rehabilitation. SoPhy (pictured below) consists of two parts: (a) a pair of socks with embedded sensors for patients to wear, and (b) a web interface that presents the information to physiotherapists in real-time. SoPhy captures and presents information related to weight distribution, foot orientation and range of movement. During a tele-consultation, when the patient wearing the SoPhy socks performs lower limb movements, the physiotherapist can see the bodily information in real-time on the SoPhy interface.

In collaboration with the Royal Children’s Hospital, we are evaluating SoPhy to better understand how it can enhance a physiotherapist’s ability to assess patients' lower limb movements during tele-consultations. Physiotherapists are finding SoPhy to be a valuable tool for assessing lower limb impairments and treating patients with weight bearing issues.

Collaboration

This project is a collaboration between the Royal Children’s Hospital via Mark Bradford in the Chronic Pain Management CPMS and the Microsoft Research Centre for Social Natural User Interfaces (SocialNUI) at the University of Melbourne.

Project team

• 
  Deepti Aggarwal, PhD Candidate, Microsoft Research Centre for SocialNUI, The University of Melbourne
• 
  Frank Vetere, Professor, The University of Melbourne
• 
  Bernd Ploderer, Honorary Fellow, University of Melbourne and Lecturer, Queensland University of Technology
• 
  Thuong Hoang, Lecturer, School of Information Technology, Deakin University
• 
  Steven Baker, Research Fellow, Microsoft Research Centre for SocialNUI, The University of Melbourne
• Mark Bradford, Physical Therapist, Royal Children’s Hospital

Contact details

• 
  Deepti Aggarwal

  Email: daggarwal@student.unimelb.edu.au

• 
  Prof Frank Vetere

  Email: f.vetere@unimelb.edu.au

Publications

Aggarwal D., Hoang T., Zhang W., Ploderer B., Vetere F., Bradford M. (2017) SoPhy: Smart Socks for Video Consultations of Physiotherapy. In: Bernhaupt R., Dalvi G., Joshi A., K. Balkrishan D., O’Neill J., Winckler M. (eds) Human-Computer Interaction – INTERACT 2017. INTERACT 2017. Lecture Notes in Computer Science, vol 10516. Springer, Cham

Aggarwal,D, Zhang,W, Hoang,T, Ploderer,B, Vetere, F & Mark Bradford, M. (2017) SoPhy: A Wearable Technology for Lower Limb Assessment in Video Consultations of Physiotherapy. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, USA. [DOI]

Aggarwal, D., Ploderer, B., V., Vetere, F., Bradford, M., & Hoang, T. (2016). Doctor, can you see my squats? Understanding bodily communication in video consultations for physiotherapy. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS 2016). New York: ACM Press, pp.1197–1208 [PDF] [DOI]

Aggarwal, D. (2016) Supporting Bodily Communication in Video Based Clinical Consultations. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI 2016). New York: ACM Press, pp. 188–192 [PDF] [DOI]

Aggarwal, D., Ploderer, B., & Vetere, F. (2015) Addressing research gaps in teleconsultation settings. In CHI 2015 Workshop on Everyday Telepresence: Emerging Practices and Future Research Directions. Seoul, Korea. [PDF]

Media

The SoPhy smart socks have received coverage in over 180 media outlets (including radio, TV news, newspapers and magazines) in Australia and overseas, with a combined coverage of over 86 million people. The story was covered overseas in the United Kingdom, United States, Malaysia, Israel, Italy, New Zealand, Iran, Canada and China. Some highlights of the coverage are listed below.