Improving recognition of cluttered objects using motion parallax in simulated prosthetic vision

The efficacy of visual prostheses in object recognition is limited. While various limitations exist, here we focus on reducing the impact of background clutter on object recognition. We have proposed the use of motion parallax via head-mounted camera lateral scanning and computationally stabilizing the object of interest (OI) to support neural background decluttering. We mimicked the proposed effect using simulations in a head-mounted display (HMD), and tested object recognition in normally sighted subjects. Images (24° field of view) were captured from multiple viewpoints and presented at a low resolution (20´20). All viewpoints were centered on the OI. Experimental conditions (2´3) included: clutter (with or without) ´ head scanning (single viewpoint, nine coherent viewpoints corresponding to subjects’ head positions, and nine randomly associated viewpoints). Subjects utilized lateral head movements to view OIs on the HMD and report what they thought the OI was. The median recognition rate without clutter was 40% for all head scanning conditions. Performance with clutter dropped to 10% in the static condition, but it was improved to 20% with the coherent and random head scanning (corrected p = 0.005 and p = 0.049, respectively). Background decluttering using motion parallax cues but not the coherent multiple views of the OI improved object recognition in low-resolution images. The improvement did not fully eliminate the impact of background. Motion parallax is an effective but incomplete decluttering solution for object recognition with visual prostheses. https://www.unr.edu/neuroscience/people/students/kassandra-lee