The other day, my friend sent me a screenshot of a text conversation and asked if they should type out all of the text in the image so that I would be able to read what was going on. While I’m grateful that my friend was willing to do that, I told them that they wouldn’t have to, as I could read the information by recognizing images with Seeing AI. Here are my tips for recognizing images with Seeing AI for users with visual impairments, and how I use different features within the app.
What is Seeing AI?
Seeing AI is a free app developed by Microsoft that helps people with visual impairments get information in real-time about the world around them using artificial intelligence- which is the “AI” in Seeing AI. Seeing AI requires an internet connection but does not require a Microsoft Account to use. Seeing AI is currently available for iPhone and iPad, and supports several different languages including English, Spanish, Chinese, French, German, Vietnamese, and more.
- Seeing AI download
- Microsoft Seeing AI And Low Vision Review
- Microsoft Immersive Reader Review
- Google Lens Review For Low Vision
How to recognize images with Seeing AI
Users can use the image recognition feature with Seeing AI for existing images by doing the following:
- If needed, download the Seeing AI app to your device
- Make sure the image is in the camera roll or saved to the device
- Open the Photos app and select the image of your choice
- Click the Share button (which looks like a rectangle with an arrow pointing upwards)
- From the Actions menu, select Recognize With Seeing AI
- The finished description will display on the bottom of the image, with information from relevant categories automatically displayed- no need to choose what information you need
Users can identify images without saving them to their camera roll by opening the Seeing AI app, choosing a function within the app, and taking a picture. The app will read information out loud- no screen reader necessary.
For images with text, Seeing AI will write out all visible text in the image verbatim, though some text formatting and spacing may be ignored. The text will be displayed with Dynamic Text settings so that users can easily read information with large print or with a screen reader. I’ve found that Seeing AI typically ignores blurred out or irrelevant background text, such as a stop sign.
Examples of images that I use text recognition with include:
- Screenshots of Tumblr posts and tweets
- Images that include lots of text
- Screenshots of text conversations
- Pictures with captions
- Diagrams with labels
- How To Write Alt Text and Image Descriptions for the Visually Impaired
- How To Add Alt Text On Social Media
- Texting Etiquette for Low Vision
- How Do People With Visual Impairments Use Emoji?
- Quick Ways To Improve Accessibility For Virtual Learning Materials
What’s going on in that photo? Seeing AI can provide scene descriptions that describe relevant parts of an image. This is a great way to generate alt text and image descriptions, though unfortunately users can’t copy and paste text from the image recognition feature at this time. That said, the scene description feature still has a ton of other uses and is especially great with photos.
Examples of images I use scene description with include:
- Photos of animals
- Basic information about selfies that my friends send me
- Short descriptions of photos, typically one sentence or less
- How To Write Video Descriptions For Animal Videos On Social Media
- How To Write Alt Text and Image Descriptions For Instagram
- How To Write Alt Text For Memes
Descriptions of people
For photos with people looking at the camera, Seeing AI can provide descriptions of what each person in the photo looks like, including the following information:
- Approximate age
- Hair color
- Any identifying features, i.e glasses
- Their facial expression, such as happy or neutral
In my experience, Seeing AI does not provide descriptions of clothing, and faces that are imported into the app’s facial recognition feature aren’t identified when using the image recognition feature in the photo gallery.
- How Do People With Visual Impairments Take Selfies?
- How Do People With Visual Impairments Guide Each Other?
- How To Be An Ally For Disabled Friends
Users can explore photos by touch by selecting the option “Explore Image” at the bottom of the screen. This will display the image in full resolution, and the user can move their finger across the screen to get information about different elements in a picture. This can include where people/objects are located, any text in the image, facial recognition information, and similar.
Examples of images that I use Explore with include:
- Layouts of rooms to find obstacles
- Pictures where there are a lot of visual elements
- Scrolling through text conversations to figure out who said what
- Exploring menus that are saved as images
- Seven Places I Don’t Take My Blindness Cane
- How I Learned To Navigate My Internship Building With Low Vision
- How To Create High Resolution Images For Users With Low Vision
- Choosing High Contrast Color Schemes For Low Vision
- How I Use My Phone As Assistive Technology In Class
Recognizing images with Seeing AI has helped me tremendously with being able to figure out what is in an image without asking for help from others. While it is not a replacement for alt text or image descriptions, the Image Recognition feature in Seeing AI is an awesome tool for people who are blind or that have low vision as it can help with getting information about all sorts of different images, from memes to screenshots of important text. I highly recommend trying it out!