“Technology is evolving for visual disability”

How do you meet the low vision needs of patients at Nolke Opticians?

Kjell Nolke (KN): I have a selection of products, including OrCam’s MyEye 2, so that when I see a patient with low vision I can give them a reasonably good scope of choice. If you have a patient who is losing their sight, invariably they or their family will research what they can do to help and they will find something like OrCam. I think it’s important to be able to access world-leading technology locally.

How has offering OrCam’s MyEye helped your patients?

KN: It’s about patient selection at all levels of vision loss and patient ability. A product will suit one patient, but not another. I’ve demonstrated the OrCam device for two years. A patient is trialling it at the moment and she has the right level of vision where OrCam can help her. She can see where the print is but can’t read it, so she points to the text and the device reads it to her.

When during the patient journey do you discuss low vision solutions?

KN: When I do a low vision assessment, I find out what their lifestyle needs are, if they are working and how active they are. The biggest part of any low vision assessment is finding out what they would like to be able to do. I then help them be as realistic as possible about outcomes and talk to them about how I can help facilitate this. I demonstrate the devices to them and find out what suits them.

The biggest part of any low vision assessment is finding out what they would like to be able to do

How do you demonstrate the MyEye device to patients?

KN: I put it on myself, point to an object and the device starts reading. I have staff members whose faces I have programmed into the device so when they come into the room, the device identifies them. I demonstrate it that way and then let them use it. I have to find out what their dexterity is like in terms of angling their chin down to look at the print. There is a learning curve with using it as the camera is on the side of their head and it needs to be pointing at the right part of the print. I gauge the patient’s interest in the device and their potential to use it. Then, I let them take it away for a week or two so they can use it in their own home. After the trial, we sit down and have a chat about if it worked for them.

How has your approach to low vision changed as a result of introducing OrCam technology in practice?

KN: I think low vision has always been about what the next product is. It’s about facilitating the right product to the right patient and to use technology as it moves on. OrCam definitely allows me to approach more patients with more choice. Low vision is a difficult thing to deal with and patients want to know what’s available to them. Some patients love that technology is available to make their lives easier. Technology is evolving for visual disability and it’s nice that patients have that facility locally.

What did OrCam set out to achieve with MyEye?

Leon Paull (LP): OrCam was established by two entrepreneurs who, prior to founding OrCam in 2010, supplied computer vision technology to the automotive industry for accident prevention. With OrCam they set out to apply the technology to human purposes. The obvious starting point was people who have lost or are losing their sight. The idea was to give them a level of independence that they would not otherwise be able to have or to do specific tasks that would be normally be difficult.

What formed the basis for the development of the device?

LP: We spoke to hundreds of visually impaired people. It was about getting close to the people who need to use this device, understanding what is important to them and what they’d like to get from this kind of solution. The first thing we heard was that they’d love for artificial vision to be able to restore or replace their vision – which it cannot do. The next most important thing for us was to understand the specific capabilities.

The obvious starting point was people who have lost or are losing their sight

During testing, what changes were discovered that needed to be made?

LP: That’s a constant process because the development of the product is ongoing. For example, the first generation of the device that we released was a camera connected to the glasses and a cable that ran down to a smartphone sized base unit. One of the key pieces of feedback that we received was that users did not want a cable running around them. That feedback got injected into the process and we miniaturised the whole thing into a smart wearable camera, without the cable, sitting on the head unit. Throughout the early development process, it was all about improving the performance of the different features of the device, such as being more accurate in recognising text or people.

How has artificial intelligence been utilised in the device?

LP: Artificial intelligence is at the core of OrCam. One example is the ability to recognise faces. That is a feature of deep learning-based technology, which gives us the ability to teach the computer to recognise and distinguish what a face is compared to another object or item, as well as between faces.

Another area it is used for is recognising hand gestures, which is the main way of operating the device. This is the only wearable artificial intelligence technology that is activated with simple human gestures. You just need to point to something and it will recognise and distinguish your finger from others that it may see and understand that as the trigger to read or identify whatever it is you are pointing to.

Artificial intelligence is at the core of OrCam

How will the device be developed in the future?

LP: We have around 200 people in research and development who are working on this technology. The things we see happening in the near future are functions such as voice commands. Making the use and interaction with the device that much more intuitive by simply being able to speak to it.

Another area of development that we are working intensely on is last metre orientation. For example, working out where the doorway is in any given room or where there are stairs. It gives the wearer more confidence and the ability to be independent in their immediate surroundings.