Living with low vision, I often use smart cameras and visual assistance tools to provide additional context or clarification about something I am looking at, especially if it is something ambiguous or hard to see. I started using the Microsoft Seeing AI app with low vision when it was released in 2017 to help with different visual tasks in my dorm so that I didn’t have to call someone else for help, and have continued to use it on my iPad and phone for accessing images, reading short text, and so much more. Here is an overview of the Microsoft Seeing AI smart camera application, and how I use Seeing AI with low vision.
What is Seeing AI?
Microsoft Seeing AI is a free visual assistance app powered by artificial intelligence and machine learning that provides information about objects and text for people with vision loss (inclusive of blind/low vision). With Seeing AI, users can use their device camera to take a photo or point it at something in their environment to get a description, or upload something from their camera roll to get a visual description within seconds. The Seeing AI interface is self-voicing and uses large, bold text by default, so users do not need to enable VoiceOver or other features to get access to the different accessibility settings.
Seeing AI is currently available for iOS and Android and requires iOS 12 or later/Android 6.0 or later. Some channels (features) require internet access to work properly, but there is no requirement to create an account. All users must be 13 years old or older.
RELATED LINKS
HOW TO USE SEEING AI
When users open the Seeing AI app, the app defaults to showing a live feed of the back camera, and has a series of buttons at the bottom of the screen for identifying different types of content, such as short text, handwriting, currency, and others— these are referred to as channels.
Depending on the channel, Seeing AI will either automatically narrate or describe information that is within the view of the camera as the user moves around, or it will prompt the user to take a picture of an item, generate a description, and then read it out loud while displaying spoken text in large print. For pictures, users can select the More Info option to generate a more detailed description or save content to their device’s camera roll.
Another option for using Seeing AI is to browse for photos from the gallery, and have Seeing AI generate a description or explore different elements of a photo such as text. I have an entire post on this feature linked below.
RELATED LINKS
SEEING AI CHANNELS
Microsoft Seeing AI can analyze the following types of content, which are displayed in this order by default. Users can rearrange the order of channels within the app settings menu
SHORT TEXT
Short text reads environmental text such as labels, price tags, or other small amounts of text. There is no character limit for how “short” the text has to be, though users may prefer the Document view for longer text excerpts. Short text reads text within the view of the camera lens and does not require the user to take a picture/scan a copy of text.
DOCUMENT
Document scans in a copy of a document such as a flyer, page in a book, business card, or similar typed text and extracts the text so it can be read out loud or copied/pasted into another application. This does not scan in a copy of the document for another application, it just extracts the text and reads it out loud/displays in large print.
To scan copies of documents and save them in their original format, I recommend using the free Microsoft Lens app, which is also available for iOS and Android.
PRODUCT
Product scans barcodes and can identify a variety of different products such as food and medication. Seeing AI has enhanced support for Haleon branded products, which includes popular products such as Voltaren, Advil, Centrum, Tums, and many more. Users can scan barcodes of products and receive enhanced dosing instructions and other information read out loud or displayed in large print.
PERSON
With the Person feature, users can figure out how many people are around them and their approximate location, as well as provide basic visual descriptions (age, hair color, what they are wearing) and their facial expression. Users can take photos and selfies with the Person feature, as well as train Seeing AI to recognize specific faces.
CURRENCY
Currency can recognize over 15 different international currencies and works best with paper money. In my own testing, I had mixed results with US coins as these tend to reflect environmental lighting and can appear distorted.
SCENE
With Scene, users can take a photo or upload a photo from their camera roll and receive a description of what is in the image— there is also an option to save images to the camera roll and copy descriptions to the clipboard. Scene can be used to describe a variety of photos, including pictures of food, room layouts, shelves, landscapes, animals, and more.
WORLD
Available for iOS only, World is a preview feature that provides indoor navigation support guided by Spatial Audio, which requires headphones. I have not tried this feature personally since it isn’t practical to hold my iPad while walking, but have included a video link with more information on the feature below.
COLOR
Color acts as a color reader and can provide information about the color of clothing, walls, and other items in the environment.
HANDWRITING
Handwriting can recognize text that is written on paper, on a whiteboard, or similar surfaces, and requires the user to take a picture with the Seeing AI camera to have text read out loud or displayed in large print. Users can save this image to the gallery if desired.
LIGHT
Light uses musical audio tones to let a user know how much light there is in a room, with a higher pitch indicating that a room/area is bright.
RELATED LINKS
- How I Use Microsoft Lens With Low Vision
- Assistive Technology For Medication Labels
- Taking Selfies With Vision Loss
- How To Identify Money With Low Vision
- Seeing AI World View on YouTube
- Smartphone Apps For Orientation and Mobility
- Reading Handwriting With Assistive Technology
- Lighting And Low Vision
Examples of how I use Seeing AI with low vision
One of the reasons I like using Seeing AI is because it gives me information about the object I am looking at and provides the option to request a more detailed description when appropriate, which can be helpful with late-night visual tasks or situations where I don’t need to use a search engine to get more information about something. Some examples of how I use Seeing AI with low vision in different contexts include:
SEEING AI AT HOME
- Taking a selfie with friends, as Seeing AI ensures everyone made it into the picture
- Reading memes and text screenshots sent by friends, which can be uploaded with the Recognizing Images tool
- Checking expiration dates with Short Text
- Reading items that came in the mail, either with Short Text or Document channels
- Reading text in another language— Seeing AI supports 16 languages including Spanish, French, German, Japanese, and others.
- If I’m unsure of what is in a picture, I will use Scene to provide additional context
- When my friend sent me a short video without letting me know if it had flashing lights, I closed my eyes and used the Seeing AI Light channel to see if the light became rapidly darker/brighter at any point. While this wasn’t perfect, it did let me know a video had multiple lighting changes and would likely not be safe to watch.
Seeing AI in the (college) classroom
- Reading signs or other environmental text
- Having handwritten notes from professors converted to large print or read out loud
- Using the Color channel to provide information about a data visualization— this was helpful when I was working with another student who is colorblind
- Recognizing images with Scene and getting additional detail or extracting text from tables
- Extracting text from a document or image that I can copy/paste somewhere else
SEEING AI AT WORK/INTERNSHIP
- Using Scene to check if I left anything on my desk— this was especially helpful when I lost a white stylus on a white desk, as it gave me an idea of where it was located.
- When someone sends an image without alt text or an image description, I use Scene to figure out what is in an image so that I don’t send a response that makes no sense or miss out on something
- Reading door numbers, sticky notes, and short text content
- When filling out forms, Seeing AI can read text out loud or provide information in large print sizes
- Reading small numbers on a badge or card that contain sensitive information— I wouldn’t want to ask someone I don’t know to help with this task
Related links
- Recognizing Images With Seeing AI
- Lemon On A Pear: Accessible Memes For Low Vision
- Taking Selfies With Low Vision
- How To Write Alt Text For Amateur Art
- Ways To Use Visual Assistance Apps For Virtual Classes
- Finding College Classrooms: College O&M
- High Contrast and Low Vision
- Ten “Odd” Things I Do With Double Vision
- Learning To Navigate My Internship Building: PM Intern With Low Vision
More tips for using Microsoft Seeing AI with low vision
- Have feedback for the Seeing AI team? I have met them in-person and they shared that the best way to send information is to use the Feedback button in the Options menu. Learn more in How To Use The Feedback Tool in Microsoft Office and Accessibility Support Phone Numbers and Low Vision
- Want to learn more about visual assistance apps? Read All About Visual Assistance Apps For Visually Impaired
- I strongly recommend using headphones that are not noise cancelling or that let the user still hear their surroundings— learn more in Mainstream Technology and Low Vision: Headphones
- The voice speed for Seeing AI can be adjusted in settings, and does not affect the voice speed for VoiceOver or TalkBack.
Published January 30, 2018. Updated January 2025