XR Helps People with Visual Impairments in the Workplace

Access@KIT and Fraunhofer IOSB are developing an assistive system that eliminates obstacles in everyday work life

People with visual impairments often face challenges in their daily work not because of a lack of qualifications, but because of barriers in their environment. A mobile assistance system being developed by researchers at the Karlsruhe Institute of Technology (KIT) and the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (IOSB) is designed to give them greater independence in their daily work and make it easier for qualified professionals to access jobs. Extended reality (XR) glasses combined with artificial intelligence are designed to adapt the work environment to individual visual needs.

For many visually impaired professionals, everyday tasks pose a challenge: reading a device label, deciphering a handwritten note, or finding their way in unfamiliar rooms takes time or requires assistance. Traditional assistive devices often reach their limits outside the computer workstation. This is where the AssistiveXR4Work project comes in: “We are developing a system that automatically recognizes visual information, enlarges it, displays it with higher contrast, or reads it aloud—thereby adapting to individual visual needs,” says Rainer Stiefelhagen, professor of computer science specializing in human-computer interaction and director of the Center for Digital Accessibility and Assistive Technology ACCESS@KIT at KIT. “In a meeting, for example, it can automatically recognize and enlarge content on a whiteboard or transcribe handwritten notes into plain text for immediate processing.” In technical work environments, the system can help with orientation and highlight relevant components.

The system uses modern XR glasses with high-resolution camera imagery. These continuously capture the surroundings and digitally adapt the display to individual visual needs: contrasts are enhanced, content is enlarged, text is recognized, and, if desired, read aloud. In addition, integrated AI assists with navigation or recognizes objects in the surroundings. If necessary, a person can also be connected via a live stream to provide support.

Inclusion Meets the Need for Skilled Workers

The project not only improves participation in the workforce but also addresses the shortage of skilled workers. This is because many qualified employees lose their jobs or are forced to change fields when their vision deteriorates. AssistiveXR4Work aims to help them continue practicing their profession. According to the Institute for Employment Research, a research institution of the Federal Employment Agency, severe disabilities are widespread in Germany and mostly occur during a person’s working life. The probability of these individuals still being employed five years later is approximately 16 percentage points lower.

A functional prototype is to be developed over the next four years and subsequently tested in real-world settings. The Federal Ministry of Labour and Social Affairs is funding the project with approximately 1.5 million euros.