The magic of touch: How deafblind people taught us to ‘see’ the world differently during COVID

Issy McGrath has type 2 Usher syndrome. Completely blind and severely deaf, she has a passion for music and plays the flute. Using a combination of touch, smell and keen imagination – her “inner eye” – Issy says she frequently senses things that are beyond the grasp of sight: the “almost solid” nature of the winter air in the morning, or the enchanting atmosphere of a frozen landscape.

For Issy and many others like her, the COVID pandemic had a devastating effect on day-to-day life. “Two-metre social distancing felt like the world had turned its back on me,” she recalls. “It was too far for me to reach out and touch everything around me. Yet it’s mainly through touch that I get a sense of what a person is like.”

A retired teacher living in Glasgow, Scotland, Issy speaks poignantly about her COVID struggles in an audio diary that was part of my research into the experiences of deafblind people during the pandemic:

As I approach my garden gate, feeling around for the latch to open it, a thought occurs to me. There is a pandemic sweeping the world and maybe I will catch the virus from this wooden fence. Maybe it’s on the latch I have just touched. I shake my hands to free myself from these thoughts. I make my way back to my house and wash my hands thoroughly, trying to free my mind of these fearful imaginings.

As a filmmaker, I am constantly questioning how and what we see – and what we don’t see. This has led me to work closely with deafblind communities around the UK, to understand how their view of the world differs from everyone else’s – in an ocularcentric society that privileges vision over all other senses.

Perceiving through touch takes time. By methodically stroking different surfaces, deafblind people build up a mental image not only of a person or object, but their place in the surrounding room or landscape. Deafblind people’s hands and skin are, I think, unusually sensitive to different levels of rigidity, to the feeling of different textures, and to slight differences in movement or temperature.

John Whitfield, a training officer at Deafblind Scotland, has been severely deaf since birth and now has only 5% of his vision left. He describes how much concentration is required to understand the world around him and keep up with conversations. “Sometimes that is very, very tiring,” he admits.

Because you are so conscious of the restriction on your hearing and vision, your brain has to compensate – and your body is having to compensate too by getting information in whatever way it can. My sense of smell is heightened, for example. You are just desperate to get as much information from the environment as you possibly can, so you will use any method.

For Roger Wilson-Hindr, who lives with his vision-impaired wife in a small village in the Midlands of England, touching means more than just receiving sensory input or holding on to information. He says every tactile interaction is a chance to form a new relationship, adding that “touch and physical contact take on greater significance if your eyes and ears are badly damaged like mine”.

Corneal scars and glaucoma suffered during childhood limit what Roger can perceive – he is able to see colour but with little definition. Trees, one of his favourite things, appear as a golden or green mass.

But when gardening, he can still “feel” the seasons through the bendability, texture and direction of the stems and branches. He says there is a “magic” to touch – “you can feel the energy of things” – and that it’s not always just about making up for a lack of vision. Deafblind people’s tactile world contains much joy.

Imagine, then, the impact for Roger and all other blind and deafblind people when COVID transformed the meaning of touch and proximity to others – from a life-enricher to a potential life threat. As Issy puts it:

Social distancing meant the world both passed me by and left me constantly conflicted. Do I allow people into my space so that I can interact and make sense of the world, risking catching the virus? Or do I ask people to respect the two-metre social distance rule, and allow a creeping sense of isolation to overwhelm my emotional wellbeing?

There are two common misconceptions about deafblind people: that they require continuous assistance and are not easy to communicate with. During our research, we heard how these perceptions contribute to their exclusion from wider society and can have a damaging effect on their confidence. This was all made worse by the pandemic, as Issy explains:

Holding someone’s hand provides me with so much information – to feel the fabric of someone’s clothing means I can get a real sense of their being. Suddenly [with the onset of COVID], to be so far away from the scent of their perfume or the texture of their hair … it was all gone. Even with the relaxing of social-distancing, the joy I had in reaching out to touch and link arms with other people has become subdued and cautious, as I warily navigate my world through my sense of touch.

When we think about touch, we usually think of hands and fingertips. But Roger highlights that, for deafblind people, “touch uses all aspects of our body – from the top of our head to feel the sunlight, to our feet for feeling where we are on the street”. Indeed, all of our interviewees emphasised the importance of touching with their feet – helping them to scan and perceive the environment while walking, to recognise the characteristics of different spaces and create a mental map.

As the first lockdown was easing, Issy recalls being reduced to tears in the middle of a street in her suddenly unfamiliar Glasgow neighbourhood. With cafés and restaurants expanding outside and altering the usual pedestrian layout, she found herself continually bumping into unexpected obstacles and people. As well as the frustration of having to create a new internal map of the area, she worried that people might become annoyed because of her lack of social distancing.

At the same time, she also felt a new threat from people invading her personal space:

I remember standing outside a supermarket, waiting for my husband, when someone tapped me abruptly on my shoulder and asked where the nearest car park was. Realising he had touched me was a shock and made me feel so uncomfortable. I asked if he was socially distancing and he replied that he had been trying to attract my attention for ages. Until that moment I was totally unaware he was there.

Before the pandemic took a grip of the world, much of my research was focused on pixels. In particular, how these tiny areas of illumination join forces to create an uninterrupted experience of film without ever revealing themselves – each undergoing a different rate of change depending on the codes they receive.

This led to some fascinating conversations with a quantum physicist, Daniele Faccio from my university’s physics department, about how new technology might reveal hitherto imperceptible light phenomena. His team were using single-photon cameras that can detect light waves as particles and thus “freeze” light in motion, taking photographs of a light pulse or video of light as it moves through a room.

As a videomaker, I found this technology fascinating – and I wondered if we could pool our knowledge to help blind people “watch” moving images by translating them into a tactile experience. In other words, develop a platform that could work as a form of “video Braille”.

In 2019, we began experimenting with ultrasound technology to focus soundwaves and create pressure spots that could be felt on someone’s hands. In this way, we hoped we could turn pixels from moving images into a range of tactile experiences linked to a film’s content (e.g. facial expressions, emotions, movement). The tactile sensations could include different temperatures, pressures and movements on the palm of each hand.

Then the pandemic intervened, our project was put on hold, and time slowed to a frustrating crawl. A saving grace, though, was my growing understanding of the way deafblind people take such care to understand their surroundings, never rushing the process of learning about a new situation. This helped me to slowly accept and learn from this extraordinary period, rather than trying to escape it.

Once lockdown ended, I tried to convey this by filming Issy in her kitchen as she made a cup of tea and arranged a vase of purple flowers. What to sighted viewers might look like “fumbling and stumbling” (as Issy calls it) is actually her way of learning and knowing. We see her gently touching the flowers, smelling their scent, imagining their forms as she measures their length, cuts and carefully arranges them into a vase. She is taking as much time as her touch needs:

Although the way I move around might look to you like a struggle, it’s not. I am putting my hand out to reach and touch things, pick things up, make sense of what’s in front of me, because that is the way I interact with my world. I am drawing up a map in my mind of what’s out there. So instead of thinking I am struggling, let me fumble and stumble – that is all information for me. The reward I get is that I will be, and am, a much more autonomous and resilient deafblind person.

The insights offered by Issy and our other deafblind collaborators during the early days of COVID made us determined to develop a tool that could help give them some independence in navigating the newly opened-up spaces after lockdown. This shifted our attention from developing a video Braille tool to one that could accurately locate the people and objects around them.

The synergy we’d already found between arts and quantum physics resulted in our idea for a new “spatial awareness” tool. Over a series of workshops starting in June 2021, Issy and John helped our research team to understand how deafblind people imagine, memorise and map a space both with and without touch – and thus what they needed from our device.

The prototype consisted of two elements: a portable radar and wearable feedback devices (a headband and an armband). “I am going to be honest and say I felt like the borg from Star Wars,” recalls Issy, our first tester. “But wow, it was fascinating.”

The radar would scan the space up to six metres in front and to each side of the tester, tracking people as they came into range and moved about. This information was turned into vibrations of different intensity using tiny coin vibration motors in the headband and armband, which activated depending on the distance and direction of the detected person.

In our first test in a large theatre room at the University of Glasgow, Issy – having turned off her hearing aids to avoid getting any other environmental clues – was asked to indicate the direction of a person entering the near-space in front of her based on the vibrations she felt in the headband.

Most of the time, without hesitation, she correctly indicated where they were standing. It was an emotional moment for her, and all of us, when we told her about the accuracy of her answers. For the first time since she went completely blind, she was sensing where people were without relying on touch:

Goodness, it would be so nice to walk up the road with this technology. Along with Yang my guide dog, I’d have a device that can tell me much more about the space around me and what’s happening – you know, how many people are in front of me, to the side, where are they? Am I walking right into a big crowd?

In our second test, Issy used both the headband (to indicate the person’s direction) and armband (for their proximity) – but struggled to correctly detect how far away a person was. After a few trials, we realised the coin vibrations motors were too close together for her to differentiate the signals, and that the forearm location was also confusing. It would be better to combine the two sets of information (distance and direction) into one headband, and use the intensity of vibrations to indicate how far away the person was.

After further trials, we refined the tool enough to be implemented into a cap. From the outset, our participants had stressed the importance of creating wearable technology that could blend in with everyday clothing if it was to be of true benefit to users such as Issy:

The fact that it could give me an extra sense of my surroundings is fascinating. I actually just wanted to say to the guys: ‘Do you fancy going up Great Western Road with it now?’

In May 2022, I was giving Issy a tour of our TouchScreen event at the Centre for Contemporary Arts in Glasgow. She was immediately drawn to a video installation called Trees, by Wolfgang Weileder. The video shows trees in different locations being cut down.

While standing in front of the large screen, she said she could sense the trees in the video via her cane. The sound frequencies from the audio were travelling from the speakers through the ground – she was thrilled because she felt included in the experience of the artwork.

As we stood there, I shifted my attention from seeing to feeling with my feet – and I could sense the vibrations too. This new layer of experience had been imperceptible to me a moment ago, yet now I felt physically related to the trees as they were being cut down. I also became aware of the ground connecting me with Issy. The sound was touching us both.

Favouring vision over other senses means we risk missing out on a host of rich experiences and connections – not least with people like Issy, Roger, John and other differently-abled people.

So the ambition of our ongoing research – combining deeper understanding of the needs of deafblind people with cutting-edge quantum technology – is not only to enable deafblind people to play a bigger role in society. We also want to use their unique understanding of the world to enrich everyone else’s.

There could be more research into technology that enables them to communicate more independently. For example, by looking at how mmWaves (the type of radio waves used in airport security scanners) could be used to recognise hand gestures and touch-based communication beyond sign-language.

Certainly, there is more for us all to learn about the value of touch in the aftermath of the pandemic. If our eyesight is about knowing through a safe distance, touch is about forming intimate relations and becoming entangled with the surrounding world. As Issy says:

You know, as somebody who has lost their eyesight, I was just too busy trying to get on with things. You don’t stop for two minutes and think: ‘Well actually, I hadn’t thought … how much I rely on touch and how much it means to me. How much it helps me to visualise the world.’

For John, touch is a “holistic way of feeling” through the body. For Issy it is about “imagination” and knowing through “fumbling and stumbling”. For Roger, touch is like “magic” that reveals the joy in the world.

It is sad that it has taken a pandemic to bring greater understanding of the significance of touch – and in particular, touch deprivation – in our daily lives. But perhaps the disconnectedness we all experienced has also evoked greater empathy for the struggles deafblind people have been experiencing throughout history, such as isolation, lack of effective communication and exclusion from society.

It’s time we embraced their unique insights and learn about the way they “see” and feel the world. Or as Issy puts it:

I always say to people, ‘You come into my space for two minutes and I’ll show you the way, in my world and my deafblind culture. The way I interact and connect with my space. Walk with me and I’ll show you the way – not through your eyes … but by connecting with me and my hands through touch.