Globally, the number of people of all ages is estimated at 285 million worldwide, of whom 39 million are blind. In an effort to help make this happen, Microsoft launched Project Tokyo in 2016, a partnership between researchers in the US, UK, China, Japan, and India to explore those technologies, helping them to communicate with those around them. World. Four years later, it's the fruit of a device - a modified version of Microsoft's HoloLens Augmented Reality headset - filled with algorithms that provide information about the people around them.

microsoft hololens
According to a blog post published by Microsoft, the research team began with athletes and spectators on a variety of journeys to the UK for the 2016 Paralympic Games in Rio de Janeiro, Brazil, as well as how to interact with other people. He navigated at airports, participated in sports venues and visited sightseeing places, among other activities. Machine learning experts from the Project Tokyo team have developed the above algorithm, which runs on graphical processing units placed on a PC connected to HoloLens, where the front lens is removed.

An LED strip pasted on the HoloLens band of cameras tracks the person to the person closest to the user and turns green when the person is identified so that the communication partners or viewers know that they have received the device. Seen or exited from the area of. Opinion. Computer Vision Model finds the currency of people in the environment, shows where and how far they are. The headset analyzes footage from the camera to identify people and whether they have revealed their names on the system.

All this information is communicated to the wearer via audio signals. For example, if the modified HoloLens detects a person at a distance of one meter to the left of the user, it will click on the one that appears to be moving to the left from that distance. If it recognizes the person's face, it will not play the "bump" sound and if the system recognizes someone who knows it, it will declare their name. A distinctive, second layer sound similar to a stretching elastic band shows the user's face.

When the HoloLens camera is connected to the person's nose, the user hears a high-click and if the person knows the system, their name. Users can optionally ask for an overview and get a spatial description of all the names the system is allowed to recognize, and when they look directly at someone, they are alerted to the spatial chime.

Microsoft says it is using a scaled-down version of the technology to help children with blind and low vision children develop social interaction skills.

Project Tokyo, which is still running, follows the heights of efforts such as Microsoft Seeing AI, a mobile app that helps low-vision and visually impaired users navigate the world around them. Recently, the tech giant has launched its first app, a navigation app called Soundscape, which uses binary audio to help people with visual impairment, help users create mental maps and personal route options in unfamiliar places.

Through AI for Accessibility, announced in May 2018, Microsoft has promised another 25 million in five years of developing AI tools for universities, charities and the disabled. This program is for the most promising peers of candidates in all three categories - work, life, and human connections - with seed funding and funding every financial quarter.
.