I’ve been using Microsoft’s Seeing AI app for over five years now, and it is one of my favorite assistive technology apps for iOS. I’m always finding new ways to use Seeing AI for various tasks in the classroom, home, and in the workplace. Here is how I use Seeing AI with low vision, as well as a selection of my favorite features.
Overview of Seeing AI
Microsoft Seeing AI is a visual assistance app powered by artificial intelligence and machine learning that provides information about objects and text for people with vision loss (inclusive of blind/low vision). Seeing AI is currently available for iOS and Android and requires iOS 12 or later/Android 6.0 or later.
The Seeing AI interface is self-voicing and uses large, bold text by default, so users do not need to enable VoiceOver or other features to get access to the different accessibility settings.
Related links
How to use Seeing AI
When users open the Seeing AI app, the app defaults to showing a live feed of the back camera, and has a series of buttons at the bottom of the screen for identifying different types of content, such as short text, handwriting, currency, and others- these are referred to as channels. Text and other details are automatically read out loud as the user moves their device through the environment.
Another option for using Seeing AI is to browse for photos from the gallery, and have Seeing AI generate a description or explore different elements of a photo such as text. I have an entire post on this feature linked below.
Related links
Seeing AI channels
Microsoft Seeing AI can analyze the following types of content, which are displayed in this order by default. Users can rearrange the order of channels within the app settings menu
Short text
Short text reads environmental text such as labels, price tags, or other small amounts of text. There is no character limit for how “short” the text has to be, though users may prefer the Document view for longer text excerpts.
Document
Document scans in a copy of a document such as a flyer, page in a book, business card, or similar typed text and extracts the text so it can be read out loud or copied/pasted into another application. This does not scan in a copy of the document for another application, it just extracts the text- if users want to save a copy of the document, I recommend using the free Microsoft Office Lens application.
Product
Product scans barcodes and can identify a variety of different products such as food and medication. Seeing AI has enhanced support for Haleon branded products, which includes popular products such as Voltaren, Advil, Centrum, Tums, and many more. Users can scan barcodes of products and receive enhanced dosing instructions and other information read out loud or displayed in large print.
Person
Person allows users to point a camera and/or take a photo of a person or group of people and receive information about their age, visual description, and facial expression. If they take a photo within the Seeing AI app, users have the option to save this photo to the gallery, making Seeing AI a great option for selfies
Currency
Currency can recognize over 15 different international currencies, but I had mixed results with having it identify US coins- it works best for paper money.
Scene
Scene allows users to take a picture and get information about their surroundings or what is in a picture- for example, I took a picture of the table next to my desk and got a description of the items on top of it. Photos taken with the scene function can also be saved to the gallery.
World
This is a newer feature with Seeing AI that provides indoor navigation support guided by spatial audio, which requires headphones. I have not tried this feature personally since it isn’t practical to hold my iPad while walking, but have included a video link with more information on the feature below.
Color
Color acts as a color reader and can provide information about the color of clothing, walls, and other items in the environment.
Handwriting
Handwriting can recognize text that is written on paper, on a whiteboard, or similar surfaces, and requires the user to take a picture with the Seeing AI camera to have text read out loud or displayed in large print. Users can save this image to the gallery if desired.
Light
Light uses audio tones to let a user know how much light there is in a room, with a higher pitch indicating that a room/area is bright.
Related links
- Why Every Student Needs Microsoft Office Lens
- Assistive Technology For Medication Labels
- Taking Selfies With Vision Loss
- How To Identify Money With Low Vision
- Seeing AI World View on YouTube
- Smartphone Apps For Orientation and Mobility
- Lighting And Low Vision
How I use Seeing AI
I use Seeing AI in a variety of different contexts, including:
Seeing AI at home
- Identifying items in the pantry, medicine cabinet, or on my desk with the barcode scanner
- Reading screenshots that friends send me from text conversations or social media posts
- Checking expiration dates with Short Text
- Identifying money in my wallet
- Scanning in items that came in the mail
- Taking a selfie with friends
Seeing AI in the classroom
- Enlarging signs or other environmental text
- Using Scene to figure out if there are any obstacles in the room
- Reading handwritten notes from professors
- Having paper assignments read out loud
- Getting information about colors in a data visualization- this is helpful for a friend who is colorblind
Seeing AI at work/internship
- Using Scene to check if I left anything on my desk- I used this when I lost a white stylus on a white desk
- Reading sticky notes
- Reading signs that are in various locations
- Having new employee orientation paperwork read out loud
- While at Microsoft, I used Seeing AI to identify various people on my team and also to read signs on rooms/offices
Related links
- How I Use Google Assistant While Traveling
- Reading Handwriting With Assistive Technology
- How I Use My Phone As Assistive Technology In Class
- How To Write Alt Text For Color Palettes
- Paper Size and Low Vision
- How I Learned To Navigate My Internship Building With Low Vision
Submitting feedback for Seeing AI
I’ve personally met the developers of the Seeing AI app multiple times, and even got to “speak” with them on stage at a CSUN event a few years ago- “speak” is in quotation marks because I had laryngitis at the time, but I was really excited to meet Anirudh and Saqib. That said, the Feedback option in the Options menu is the best way to get in contact with the developers directly and share any feedback, good or bad.
Related links
- How To Use The Feedback Tool in Microsoft Office
- Using The Microsoft Disability Answer Desk With Be My Eyes
More tips for using Microsoft Seeing AI with low vision
- Want to learn more about visual assistance apps? Read All About Visual Assistance Apps For Visually Impaired
- I strongly recommend using headphones that are not noise cancelling or that let the user still hear their surroundings. I like using a pair of wireless bone conducting headphones, though AirPods are another popular choice
- The voice speed for Seeing AI can be adjusted in settings, and does not affect the voice speed for VoiceOver
- Seeing AI offers support for over a dozen different languages- an up-to-date list can be found on their website
