News

Lego uses AI to translate instructions into braille and audio

The toy giant has launched a global trial with four building sets and hopes to release more next year.
The toy giant has launched a global trial with four building sets and hopes to release more next year. The toy giant has launched a global trial with four building sets and hopes to release more next year.

Lego is building its way to making instructions friendlier for the visually impaired by using artificial intelligence (AI) to translate instructions into braille and audio.

The Danish toy company has launched a global trial with four building sets and hopes to release more next year depending on feedback.

Using the design script used by Lego creators, known as LXFML data, the Austrian Research Institute for Artificial Intelligence developed software that is able to turn the instructions into braille and English audio.

The idea came about after blind entrepreneur and Lego enthusiast Matthew Shifrin previously relied on a friend to translate instructions into braille, which enabled him to build independently for the first time.

After his friend died, Mr Shifrin was introduced to Lego’s Creative Play Lab and decided to push for development of the technology.

“This is extremely important for blind children because there aren’t a lot of places where we can say ‘Look Mum and Dad! I built this on my own… I did this’,” he said.

Matthew Shifrin plays Lego
Matthew Shifrin plays Lego Matthew Shifrin, who was born blind, inspired the idea to help children with visual impairments (Lego/PA)

“For blind children, we don’t have access to what sighted kids are used to.

“Lego bricks enable us to learn about our environment, to see the world.

“It is so important because blind kids get left out of a lot of social stuff, especially in elementary school.

“But Lego building is one of the things we can do.”

Lego said there is still “a lot of progress” to be made with the AI software and automating the process, but it hopes to introduce more languages in the future.

“As I build a set, I develop a better sense of what a building looks like and how it is laid out and constructed,” Mr Shifrin said.

“For blind people, Lego sets act as miniature 3D substitutes for real-life buildings in lieu of two-dimensional photographs.

Child playing Lego
Child playing Lego Lego is providing audio and braille building instructions to four sets initially (Lego/PA) (T2Studio)

“Lego bricks allow me to see things that are impossible to explore by touch, such as the arches of a Middle Eastern palace or the towers of the London Tower Bridge.”

Lego Group creative director Fenella Blaize Charity said: “His project will help visually impaired children around the world experience the same joy of building and pride of creation that all our fans feel.”

Paul Welham, chief executive of CereProc, the speech synthesis company that provided Lego with the Software Development Kit, said: “This fantastic initiative is a major step forward for people living with visual impairment.

“By adopting Text-to-Speech technology in this way, Lego will not only improve the lives of those using its products, but hopefully also inspire other companies to follow suit with characterful synthetic speech.”