Project overview

Digital technology presents significant opportunities to support zoos in their goals of animal welfare and enrichment. Researchers from the Microsoft Research Centre for Social Natural User Interfaces (SocialNUI) at the University of Melbourne are excited to be collaborating with Zoos Victoria to explore how natural user interface technologies (NUIs) can improve animal wellbeing.

Orang-utans are a highly intelligent species critically endangered due to deforestation of their natural habitat for palm oil plantations.  Through our collaborative project we are studying how digital enrichment can improve the welfare of Melbourne Zoo’s 6 orang-utans and building a better understanding of the way orang-utans learn and interact with technology. By studying the design and use of digital enrichment we are seeking to better understand the zoo as a context for human-computer interaction (HCI) research, and develop design methodologies for non-human users.

The research team are applying ‘user-centered design’ methods to approach the development of enrichment for orang-utans. This involves observing interactions between orang-utans and keepers as well as conducting interviews and workshops with animal welfare experts, keepers and other personnel, to identify orang-utans’ motivations and interests and define the opportunities for NUIs in this space.

Early on in this process, the SocialNUI researchers studied how Melbourne Zoo was utilising technology with the orang-utans. As well as projecting movies into the enclosure, the keepers have explored the use of tablet computers to provide enrichment in the form of puzzles, games, musical instruments and painting. While these activities show the potential for digital enrichment, they are limited as the orang-utans, who would readily break fragile tablet computers, can only interact with their fingertips through metal bars while a zoo keeper holds the tablet outside the enclosure. The current project utilises NUI technologies to give the orang-utans a wider range of ways to interact with technology through more active, full body movement.

The proof of concept system was tested in February 2016 at Melbourne Zoo. This solution uses a Microsoft Kinect and a projector placed outside the enclosure, which avoids the need to give the orang-utans physical devices that could be broken. The system projects moving images on to the floor of the orang-utan enclosure. Using custom software, the Microsoft Kinect sensor tracks the orang-utans’ movements and detects their touches and interactions with this area of the floor, so that the projection becomes a large touch-enabled surface.

The orang-utans were given complete choice over whether to interact with the system or not, and all trials were overseen by the Zoo’s animal welfare expert.

The team developed a variety of games and simple applications for this system. The applications aim to explore how orang-utans prefer to interact with the system, and demonstrate opportunities for enrichment. The games are designed to encourage orang-utans to play, explore and learn about what they can do with the system, building on their natural problem-solving abilities.

The games were developed in close collaboration with the zoo keepers, drawing on their knowledge of the orang-utans different personalities, likes and dislikes. For example, researchers discovered that Kiani (Suma), Melbourne Zoo’s 37 year old female orang-utan, loves looking at photos of herself. The team developed an application that showed a gallery of photographs that the orang-utans could browse, select and interact with. The orang-utans have experience of painting on canvas and seem to find this enriching, so a painting application was also developed.

Orang-utans’ willingness and ability to use this technology without training or encouragement indicates that it can provide a foundation for diverse cognitive challenges which respond to their individual needs and interests, helping to keep their minds and bodies active.

During the trial, the research team interviewed zoo visitors who watched the orang-utans interacting with the system. They found that visitors responded with empathy for the animals, and reflected on what the animals might be thinking and feeling as they interacted with the system.

As part of the trial the orang-utans were offered the opportunity to interact with humans through the technology. As orang-utans seem to find humans interesting, the team created a shared digital space where orang-utans could choose to interact safely with keepers and even visitors. In the first trial of a game for humans and orang-utans, the team observed with delight that the orang-utans chose to play even with people they had not met before, creating a powerful sense of connection for the human player.

The next phase of this research is to explicitly examine the benefits of digital technology to the orang-utans’ welfare. Zoo personnel and the orang-utan keepers are integrally involved in the design and execution of the study. At the same time, SocialNUI researchers will be conducting a visitor experience study to further examine visitor perceptions of the use of technology by zoo animals.

Zoo Technology Research Group

We facilitate discussion between zoo professionals, technology researchers and specialists in animal welfare and behaviour, through the Zoo Technology Research Group online network. The network was established following the Technology at the Zoo’ workshop, convened by the project team at the CHI2016 conference, in collaboration with pre-eminent zoo-based researchers from around the world.

If you are interested in joining the Zoo Technology Research Group, please contact us.

Collaboration

This project is a collaboration between the University of Melbourne’s Microsoft Research Centre for Social Natural User Interfaces (SocialNUI) and Zoos Victoria and proudly supported by Microsoft.

Project team

• 
  Sarah Webber, Doreen Thomas Postdoctoral Fellow, School of Computing and Information Systems, The University of Melbourne
• 
  Zaher Joukhadar, Lead Software Engineer, The University of Melbourne
• 
  Marcus Carter, Department of Media and Communications, University of Sydney
• 
  Sally Sherwen, Animal Welfare Specialist, Wildlife Conservation & Science, Zoos Victoria
• 
  Wally Smith, Associate Professor, School of Computing and Information Systems, The University of Melbourne
• 
  Ruining Dong, Masters Student, The University of Melbourne

Contact details

• 
  Sarah Webber

  Email: s.webber@unimelb.edu.au

Publications

Dong, R., Carter, M., Smith, W., Joukhader, Z., Sherwen, S. & Smith, A. (2017) Supporting Animal Welfare with Automatic Tracking of Giraffes with Thermal Cameras In Proceedings of the 29th Australian Conference on Computer-Human Interaction (OzCHI 2017), 28 Nov – 1 Dec, Brisbane, Australia.

Webber, S., Carter, M., Smith, W. & Vetere, F. (2017) Interactive technology and human-animal encounters at the zoo International Journal of Human-Computer Studies 98:150–168. [DOI]

Webber, S., Carter, M., Sherwen, S., Smith, W., Joukhader, Z. & Vetere, F. (2017) Kinecting with Orangutans: Zoo visitors’ emphathetic responses to animals’s use of interactive technology. In In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, USA. [DOI]

Webber, S., Carter, M., Watters, J., Krebs, B., Sherwen, S., Mancini, C., French, F. & O’Hara, K. (2016). HCI Goes to the Zoo: Workshop proposal. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. [DOI]

French, F., Kingston-Jones, M., Schaller, D., Webber, S., Väätäjä, H. & Campbell, M. (2016). Don’t cut to the chase: Hunting experiences for zoo animals and visitors. In ACI '16 Proceedings of the Third International Conference on Animal-Computer Interaction, Article No. 19. [DOI]

Pons, P., Carter, M., & Jaen, J. (2016). Sound to your Objects: A Novel Design Approach to Evaluate Orangutans’ Interest in Sound-based Stimuli. In Proceedings of The Third International Congress on Animal Computer Interaction. ACM. [PDF]

Webber, S., Carter, M., Smith, W., & Vetere, F. (2016). Interactive Technology and Human-Animal Encounters at the Zoo. International Journal of Human-Computer Studies [DOI] [PDF]

Carter, M., Webber, S., & Sherwen, S. (2015). Naturalism and ACI: Augmenting Zoo Enclosures with Digital Technology. In Proceedings of The Second International Congress on Animal Computer Interaction. ACM. [PDF]

Rault, J.-L., Webber, S., & Carter, M. (2015). Cross-Disciplinary Perspectives on Animal Welfare Science and Animal-Computer Interaction. In Proceedings of the Second International Congress on Animal Computer Interaction. ACM. [PDF]

Media