- Principal Investigator:
- James Coughlan
Abstract
CamIO ("Camera Input-Output") is a project designed to make physical objects, including documents, maps and 3D objects accessible to people who are blind or visually impaired. It makes these objects accessible by means of an audio-based Augmented Reality (AR) interface: the user points to a specific locationon an object of interest and hears audio information about this location read aloud (or sees an enhanced image of the selected location if the user has low vision). Compared with other approaches to making objects accessible to people who are blind or visually impaired, CamIO offers several advantages: (a) there is no need to modify or augment existing objects (e.g., with braille labels or special touch-sensitive buttons), requiring only a smartphone and optional wearable camera; (b) CamIO is accessible even to those who are not fluent braille users; (c) it permits natural explorationof the object aith allfingers to enable audio-haptic readers; and (c) it permits natural exploration of the object with all fingers to enable audio-haptic interactions with the project.
Building on our past work on the prototype CamIO iPhone app, we plan to develop powerful enhancements and new functionality, including finger tracking as an alternative to the use of a pointing stylus, and support for wearable options such as the camera-enabled Envision Glasses that will offer a far superior user interfaces to what is possible with a stand-alone smartphone app. New authoring capabilities will make it easy for blind and visually impaired usersand sighted assistants to create audio labels for objects. We will also pursue two esciting applications of CamIO in depth: using CamIO to make tactile maps fully interactive and easier to understand, and the devlopment of interactive installations such as a 3D scale model of a large playground that helps visually impaired playground visitors familiarize themselves with the playground's layout and features. Finally, people who are blind or visually impaired -- the target users of the CamIO -- will be involved in all aspects of the proposed research, to maximize the impact of the research effort.
Health Relevance
For people who are blind or visually impaired, a serious barrier to employment, economic self-sufficiency and indeopendence is insufficientaccess to a wide range of everyday objects needed for daily activities that require visual inspection on the part of the user. Such objects include printed documents, maps, infographics, appliances and 3D models used in science, technology, engineering, and mathematics (STEM), and are abundant in schools, the home and theworkplace. The proposed research would result in a powerful camera-based smartphone app to provide access to such objects for the approximately 10 million Americans with significant vision impairments or blindness.