‘Decorating with Light’ is an engaging and revealing interactive installation that forms part of Dr Marcus Winter’s research in the field of human interaction with machine learning (ML).
‘I’m looking into people’s understanding of ML applications and how they react to them,” says Dr Winter. ‘ML can do fascinating things, but it can also make spectacular mistakes. I want to explore how people react to this. What is their understanding of the ML applications they’re using? What kinds of errors are deemed acceptable and which ones are not?
‘In order to study these reactions, we created an accessible and immersive ML application at The Regency Town House, a museum and community centre in Brighton. The building was designed by 19th century architect C A Busby, who is well known for his elegant regency buildings, although his colourful and intricate interior wall designs are much less recognised.
‘We designed an ML application that allowed visitors to interact with Busby’s interior decorations projected onto the walls of the Regency Town House, where they would have been in real life. The ML of the application recognised exhibition visitors’ body poses and movement, enabling them to use their hands and feet as ‘virtual paintbrushes’ to splash Busby’s designs, via the application and projector, onto the walls of the house.
‘As well as seeing the designs appear on the walls, participants also see their on-screen body representation as predicted by the ML application. Sometimes, there are inaccuracies in the predicted body pose, or the system doesn’t recognise a gesture or misinterprets it. This is caused by the inherent uncertainty in ML predictions, and we examined how participants react to this.
‘We found that participants noticed these problems, but they didn’t mind that it was happening. Their reaction was usually to try again and see if they got a different result. Some tried using extreme body poses to challenge the system and explore its limitations. This is encouraging to see, as it helps people to better understand what ML can and cannot do.
‘People’s reaction might be different in other contexts. An art installation is a ‘non-critical’ situation. What would be their reaction to similar ML errors relating to something like a bank transaction or a health diagnosis? Where would the level of toleration be, if at all?
‘Decorating with Light was an instructive case study with scope to inform future development and research into human aspects of ML, and its potential for visitor engagement in museums.’
Leave a Reply