James Coughlan

Senior Scientist

Director: Rehabilitation Engineering Research Center

Ph.D.

The goal of our laboratory is to develop and test assistive technology for blind and visually impaired persons that is enabled by computer vision and other sensor technologies.

Download CV
Coughlan Lab
Contact Information:

Email: coughlan@ski.org

Office Phone: (415) 345-2146


2318 Fillmore Street

San Francisco, CA 94115

Links:
Link to Google Scholar

Publications

Schematic showing a graph with shaded areas around modes and edges indicating possible navigation areas. The current intermediate target is a node enclosed in a circle.

S. Mascetti, Dr. Ahmetovic, G. Galimberti and J. Coughlan. “NavGraph: Enhancing Blind Travelers’ Navigation Experience and Safety.” Proc. ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), Volume 9, Issue 3, Article No.: 117, Pages 1 – 27. 2025.

Biggs, B. ., Coughlan, J. ., & Walker, B. (2025). Systematically Evaluating Digital Map Tools Based on the WCAG. Journal on Technology and Persons With Disabilities, 13.

Manzoni, M. ., Mascetti, S. ., Ahmetovic, D. ., Crabb, R. ., & Coughlan, J. . (2025). MapIO: a Gestural and Conversational Interface for Tactile Maps. IEEE Access.

Biggs, B. ., Agbaroji, H. ., Toth, C. ., Stockman, T. ., Coughlan, J. ., & Walker, B. . (2024). Co-Designing Auditory Navigation Solutions for Traveling as a Blind Individual During The COVID-19 Pandemic. Journal of Blindness Innovation and Research, 14(1). (Original work published 2024)

Crabb, R. ., Cheraghi, S. A., & Coughlan, J. . (2023). A Lightweight Approach to Localization for Blind and Visually Impaired Travelers. Sensors, 23(5). https://doi.org/10.3390/s23052701 (Original work published 2023)

Pitcher-Cooper, C. ., Seth, M. ., Kao, B. ., Coughlan, J. ., & Yoon, I. . (2023). You Described, We Archived: A Rich Audio Description Dataset. Journal on Technology and Persons With Disabilities, 11. (Original work published 2023)

Hong, J. ., & Coughlan, J. . (2023). VR Training to Facilitate Blind Photography for Navigation. Journal On Technology And Persons With Disabilities. (Original work published 2023)

Biggs, B. ., Pitcher-Cooper, C. ., & Coughlan, J. . (2022). Getting in Touch with Tactile Map Automated Production: Evaluating Impact and Areas for Improvement.Journal on Technology and Persons With Disabilities, 10. (Original work published 2022)

Tekin, E. ., Vásquez, D. ., & Coughlan, J. . (2013). S-K Smartphone Barcode Reader for the Blind. Journal on Technology and Persons With Disabilities, 1.

Tekin, E. ., Coughlan, J. ., & Simon, H. J. (2013). Improving speech enhancement algorithms by incorporating visual information. The Journal of the Acoustical Society of America, 134, 4237–4237.

Coughlan, J. ., & Shen, H. . (2013). Crosswatch: a system for providing guidance to visually impaired travelers at traffic intersections. Journal of Assistive Technologies, 7.

Manduchi, R. ., & Coughlan, J. . (2012). (Computer) vision without sight. Communications of the ACM, 55, 96–104.

Hand pointing to the wing of an airplane shown in a tactile graphic. The hand skeleton detected by computer vision is shown in green.

Ahmetovic, D., Coughlan, J., Dal Santo, G., Ezrouri, K., Manzoni, M., & Mascetti, S. (2025). CamIO in the Browser: A Cross-Platform Audio Label Tool for Tactile Graphics. MobileHCI 2025

Hong, J. ., & Coughlan, J. . (2024). Enhancing Walk-Light Detector Usage for the Visually Impaired: A Comparison of VR Exploration and Verbal Instructions. Web4All 2024. Presented at the Web4All 2024. Singapore. (Original work published 2024)

Coughlan, J. ., Biggs, B. ., & Shen, H. . (2022). Non-Visual Access to an Interactive 3D Map. Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE ’22). Presented at the Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE ’22). (Original work published 2022)

Biggs, B. ., Toth, C. ., Stockman, T. ., Coughlan, J. ., & Walker, B. N. (2022). Evaluation of a Non-Visual Auditory Choropleth and Travel Map Viewer. International Conference on Auditory Display (ICAD) 2022. Presented at the International Conference on Auditory Display (ICAD) 2022. Virtual. Retrieved from PMCID: PMC10010675 (Original work published 2022)

Coughlan, J. ., Biggs, B. ., & Shen, H. . (2021). Point and Listen: Bringing a 3D Map to Life with Audio-Based AR. 6th Annual Frameless XR Symposium. Presented at the 6th Annual Frameless XR Symposium. (Original work published 2021)

Biggs, B. ., Coughlan, J. ., & Coppin, P. . (2021). Design and evaluation of an interactive 3D map. RESNA 2021 Conference. Presented at the RESNA 2021 Conference. (Original work published 2021)

Fusco, G. ., Cheraghi, S. A., Neat, L. ., & Coughlan, J. . (2020). An Indoor Navigation App using Computer Vision and Sign Recognition. 17th International Conference on Computers Helping People With Special Needs. Presented at the 17th International Conference on Computers Helping People with Special Needs. (Original work published 2020)

Coughlan, J. ., Biggs, B. ., Rivière, M.-A. ., & Shen, H. . (2020). An Audio-Based 3D Spatial Guidance AR System for Blind Users. 17th International Conference on Computers Helping People With Special Needs. Presented at the 17th International Conference on Computers Helping People with Special Needs. (Original work published 2020)

Biggs, B. ., Coughlan, J. ., & Coppin, P. . (2019). Design and Evaluation of an Audio Game-Inspired Auditory Map Interface. The 25th International Conference on Auditory Display (ICAD 2019). Presented at the The 25th International Conference on Auditory Display (ICAD 2019). Northumbria University, Newcastle-upon-Tyne, UK. (Original work published 2019)

Fusco, G. ., & Coughlan, J. . (2018). Indoor Localization using Computer Vision and Visual-Inertial Odometry. International Conference on Computers Helping People With Special Needs (ICCHP ’18). Presented at the International Conference on Computers Helping People with Special Needs (ICCHP ’18). Linz, Austria. (Original work published 2018)

Fusco, G. ., Tekin, E. ., Ladner, R. E., & Coughlan, J. . (2014). Using Computer Vision to Access Appliance Displays. ASSETS. Presented at the ASSETS. (Original work published 2014)

Manduchi, R. ., & Coughlan, J. . (2014). The last meter: blind visual guidance to a target. Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, 3113–3122. ACM.

Zaman, N., Potluri, V., Biggs, B., & Coughlan, M. J. (2025). WhatsAI: Transforming Meta Ray-Bans into an Extensible Generative AI Platform for Accessibility. arXiv preprint arXiv:2505.09823.

Fusco, G. ., Tekin, E. ., & Coughlan, J. . (2016). Sign Finder Application – Technical Report.

Coughlan, J. ., & Manduchi, R. . (2013). Camera-based access to visual information. In Assistive technology for blindness and low vision (pp. 219–246).

Tekin, E. ., & Coughlan, J. . (2012). BLaDE: Barcode localization and decoding engine.

Coughlan, J. . (2012). Markov random fields and techniques for performing inference with them.

Shen, H. ., & Coughlan, J. . (2012). Towards a real-time system for finding and reading signs for visually impaired users. In Computers Helping People with Special Needs (pp. 41–47). Springer Berlin Heidelberg. (Original work published 2012)

Coughlan, J. . (2011). Mechanisms for propagating surface information in 3-D reconstruction. In C. W. Tyler (Ed.), Computer Vision: From Surfaces to 3D Objects. Chapman and Hall/CRC Press.

Tekin, E. ., & Coughlan, J. . (2010). A mobile phone application enabling visually impaired users to find and read product barcodes. Springer Berlin Heidelberg.

Coughlan, J. . (2010). Maximum Entropy Distributions and Their Relationship to Maximum Likelihood.

Coughlan, J. . (2009). Basic 3D Geometry for One and Two Cameras.

Coughlan, J. . (2009). A Tutorial Introduction to Belief Propagation.

Ivanchenko, V. ., Coughlan, J. ., & Shen, H. . (2008). Crosswatch: a camera phone system for orienting visually impaired pedestrians at traffic intersections. Springer Berlin Heidelberg.

Projects

  • ScribbleEyes
    Active
    ScribbleEyes

    Scribble Eyes is an Apple iPad/Pencil-based app that automates the data collection, analysis and interpretation process for paper-based tests traditionally used to screen patients with central visual field loss in the clinic.

    Read More
  • Interactive Tactile Map at Google ADC Milan
    Interactive Tactile Map at Google ADC Milan

    This is an installation of the CamIO project that adds interactivity to a tactile map of the Google Accessibility Discovery Center in Milan, Italy.

    Read More
  • Empowering Data Vision: A Self-Paced Introduction to the Convergence of AI and Data Science
    Active
    Empowering Data Vision: A Self-Paced Introduction to the Convergence of AI and Data Science
    Read More
  • Using VR to Help Train Visually Impaired Users to Aim a Camera
    Active
    Using VR to Help Train Visually Impaired Users to Aim a Camera

    People with visual impairments increasingly rely on camera-enabled smartphone apps for tasks like photography, navigation, and text recognition. Despite the growing use of these applications, precise camera aiming remains a significant challenge. This project explores the impact of virtual reality (VR) exploration in the context of learning to use a camera-based app. So far we have studied this approach in the context of training a visually impaired person to use walk-light detector app at traffic intersections.

    Read More
  • Smith-Kettlewell Summer Institute
    Smith-Kettlewell Summer Institute

    The Smith-Kettlewell Summer Institute program is a multi-year, learning opportunity designed to provide training for blind and visually impaired students, recent graduates, and early career researchers interested in data skills that will enhance Science, Technology, Engineering and Math (STEM) jobs within industry, government, non-profit organizations or academia.

    In today’s research climate, individuals are expected to have many skills at their fingertips. Often researchers are required to build software and hardware environments to run studies, to gather and analyze data and to prepare…

    Read More
  • Empowering Data Vision: Data Science Course for Blind Individuals using Python
    Active
    Empowering Data Vision: Data Science Course for Blind Individuals using Python

    Call for Participation:

    Second Smith-Kettlewell Summer Research Institute, “Empowering Data Vision: Data Science Course for Blind Individuals using Python”

    The Smith-Kettlewell Eye Research Institute, the University of Michigan and the University of Illinois at Urbana-Champaign are delighted to present our second on-line course dedicated to data science for blind individuals.

    Overview:

    This course, “Empowering Data Vision: A Data Science Course for Blind Individuals using Python,” will teach blind individuals the basics of using Python to interact with datasets. The course will be offered on…

    Read More
  • CamIO Hands
    Active
    CamIO Hands

    This project builds on the CamIO project to provide point-and-tap interactions allowing a user to acquire detailed information about tactile graphics and 3D models.

    The interface uses an iPhone’s depth and color cameras to track the user’s hands while they interact with a model. When the user points to a feature of interest on the model with their index finger, the system reads aloud basic information about that feature. For additional information, the user lifts their index finger and taps the feature again. This process can be repeated multiple times to access additional levels of…

    Read More
  • Talking Signs
    Completed
    Talking Signs

    Created by William Loughborough in 1979, Talking Lights was a system of infrared transmitters and receivers allowing blind and visually impaired travelers to quickly and easily “read signs” at a distance.

    Read More
  • Empowering Data Vision: A Data Science Course for Blind Individuals Using R
    Active
    Empowering Data Vision: A Data Science Course for Blind Individuals Using R

    The goal of the “Empowering Data Vision: A Data Science Course for Blind Individuals” program is to help people with visual impairments acquire advanced skills related to STEM subjects and careers.

    Read More
  • Magic Map
    Active
    Magic Map

    The Magic Map is an interactive 3D map installed at the Magical Bridge Playground in Palo Alto, California. It consists of a 1/100 scale 3D bronze representation of the playground, which includes over seventy play structures organized into multiple play zones and paths. When the user’s index fingertip touches a specific feature on the map, the name and description of the feature are read aloud in audio. This interactivity allows visitors with visual impairments to navigate the map without requiring them to read braille.

    Read More

Labs

  • Coughlan Lab
    Coughlan Lab
    Principal Investigator:
    The goal of our laboratory is to develop and test assistive technology for blind and visually impaired persons that is enabled by computer vision and other sensor technologies.
    Read More

Centers

Collaborators

Internal Collaborators

External Collaborators

Trainees

Current Trainees

Past Trainees