Visual search: Everything you need to know.

Visual Search — The Ultimate Guide: Statistics, News, Trends, and Tips

A comprehensive visual search resource for the latest trends, stats, news, and strategy tips.

Clark Boyd

19 min readAug 6, 2018

Last updated: December 19, 2022.

Visual Search Guide: Table of Contents

Visual Search Definition
Visual Search Statistics
Visual Search for Retailers
Visual Search Marketing: Brands Using Visual Search Today
In the News
Visual Search Acquisitions and Fundraising
Pinterest Visual Search News
Google Visual Search News
Amazon Visual Search News
Instagram/Facebook Visual Search News
Quotes
Charts and Graphs
Trends
Challenges
Visual Search APIs
Visual Search Engines
Further Resources
Get in touch!

What is Visual Search?

Visual search uses real-world images (screenshots, Internet images, or photographs) as the stimuli for online searches.

Modern visual search technology uses AI (artificial intelligence) to understand the content and context of these images and return a list of related results.

It has a host of applications in the eCommerce industry, particularly for fashion and home decor retailers. Visual search allows retailers to suggest thematically or stylistically related items to shoppers in a way they would struggle to do using a text query alone.

Pinterest, Google, and Amazon are the main visual search engines today. Microsoft has also developed impressive computer vision capabilities for its Bing search engine.

A host of retailers including ASOS, Wayfair, Neiman Marcus, Argos, and IKEA have all built proprietary visual search tools.

The latest development in the industry is “multimodal search”, in which text, images, and videos can be used simultaneously to form a query.

Google’s new visual search technology. Consumers can add text-based context to their image query.

You can read a little more in the recent Raconteur article, ‘What is visual search?’ here.

For more stories like this, subscribe to Clark’s free newsletter!

Why is Visual Search Important?

Visual search has the potential to change the way we interact with the world around us.

Our culture is already dominated by the visual, so it seems natural that we would use an image to start a search.

After all, when we shop offline, we rarely start with text. Visual search brings that sense of visual discovery to the online world.

Moreover, we often want to find a new look, outfit, or theme, rather than a single object. Visual search technology helps to tie these items together based on aesthetic links in a way text has never been able to capture.

Visual search strategy for fashion. Fashion brands can now engage with users through mobile phone cameras.

Google, Amazon, Pinterest, Bing, and a host of retailers have all developed their own visual search engines and their collective weight will surely see the technology’s use take hold very soon.

As part of a broader trend, visual search can be packaged under ‘sensory search’ alongside the well-documented growth of voice search and the introduction of more tactile interactions with websites.

The technology is still in its infant stages, but recent trends reveal just how quickly it can advance. In particular, Pinterest’s visual search technology (known as Pinterest Lens) and Google’s visual search (also, imaginatively, called Lens), are leading the way.

So, I have compiled a list of particularly interesting, newsworthy, and/or impressive visual search facts and stats here.

I’ll keep updating it frequently and hopefully it will serve as a useful resource for those who wish to understand how visual search can affect the everyday lives of consumers. Or just for those who need a few visual search statistics for a presentation.

Visual Search Statistics

90% of information transmitted to the human brain is visual. MIT
The human brain can identify images seen for as little as 13 milliseconds. MIT
The image recognition market will grow to $25.65 billion by 2019. MarketsandMarkets
62% of millennials want visual search over any other new technology. Visenze
21% of advertisers believe visual search is the most important trend for their business in 2019. Marin Software
45% of retailers in the UK now use visual search. Tech HQ
36% of consumers have conducted a visual search. The Intent Lab
35% of marketers plan to optimise for visual search through 2020. Search Engine Journal
55% of consumers say Visual Search is instrumental in developing their style and taste. Pinterest
By 2021, early adopter brands that redesign their websites to support visual and voice search will increase digital commerce revenue by 30%. Gartner
Visual information is preferred over text by at least 50% of respondents in all categories except for electronics, household goods and wine and spirits. The Intent Lab
59% think visual information is more important than textual information across categories (vs. 41% who think textual information is more important). The Intent Lab
When shopping online for clothing or furniture, more than 85% of respondents respectively put more importance on visual information than text information. The Intent Lab
20% of app users make use of visual search when the feature is available. GlobalData
The Global Visual Search Market is estimated to surpass $14,727m by 2023 growing at CAGR +9% during the forecast period 2018–2023. Report Consultant

Google Lens Search Statistics

Images are returned for 19% of search queries on Google. Moz
Images are returned for 34% of search results. Econsultancy
Google Lens can detect over 1 billion objects. TechSpot
Google Shopping ads see budget increase of 33%, versus a rise of just 3% for text ads. Merkle report, Q3 2018
Google Lens has been used over 1 billion times. Google (2019)
Google Lens is used over 3 billion times per month. Google (2021)
Google Lens is used over 8 billion times per month. Google (2022)

Pinterest Visual Search Statistics

Brands can now target over 5,000 categories through visual search advertising on Pinterest Lens. Business Insider
Image-based Pinterest Ads have an 8.5% conversion rate. Heap Analytics
Pinterest projected to clear $1 billion in ad revenues per annum by 2020. eMarketer
21% of Pinterest users use traditional search less when they have an option to use visual search. Raymond James study
There are over 600 million visual searches on Pinterest every month. Pinterest
There are 3x as many visual searches on Pinterest as last year. (June 1, 2020) — Pinterest
80% of Pinners (those who use the Lens technology every day) start with visual search when shopping vs. 58% of non-Pinners. Pinterest
When shopping online for clothing or furniture, over 85% of respondents put more importance on visual information than text information.
49% of Pinners say they develop a better relationship with brands they love through visual search.
61% of consumers say visual search elevates their experience while in-store browsing.

**Visual search statistics infographic**

Visual Search Technology: Retailers

ASOS Style Match allows shoppers to purchase using images. Engadget
eBay debuts visual search technology. Business Insider
Neimann Marcus reports strong quarter after developing visual search technology. ZDNet
Ted Baker shoppable videos. Ted Baker
Blippar uses visual search to help customers find their way. TechCrunch
EasyJet launches visual search tool. Digiday
Levi’s and Disney experiment with visual search on Snapchat. Mobile Marketer
Walmart is testing an in-house visual search technology. TechCrunch
Salesforce launches Einstein visual search for retailers. MarTech Today
Syte launches new visual search tool for retailers. PR Newswire
Houseology launches new visual search technology. Houseology
CarStory launches first automotive visual search app: Business Wire
Snapchat’s Lens Creative Partners program helps brands find AR filter creators. Marketing Land
Farfetch launches a visual search app, allowing users to search for images they found on Instagram or Pinterest. Luxury Daily
Marks and Spencer launches new mobile visual search technology. Essential Retail
AI Startup Syte Helps Retailers Enable Visual Search Capabilities. PSFK
Lifestyle adds new retail technologies, including voice and visual search. India Retail
ViSenze brings visual search to Samsung smartphones. RLI
Slyce debuts visual search for grocery stores. Fresh Plaza
Argos launches AI-powered visual search to connect offline stores with online catalogue. Internet Retailing
Lush demos visual search app and fresh ‘digital packaging’ at SXSW. Mobile Marketer
LexSet debuts new visual search software. Furniture Today
The brave new world of computer vision: What retailers need to know. RIS News
5 ways e-commerce brands can leverage the growing demand for visual search. Smart Insights
Can Visual Search Help Retailers Find New Sales? PYMNTS
Retailers wear Artificial Intelligence: Visual Search and the future of the shopping experience. Sales Layer
Argos launches a visual search service for app users. Tech Advisor
IKEA revamps app to allow shoppable visual search at home. Reuters
Balmain launches range of virtual models for new campaign. Independent
Walmart bets on shoppable video. Digiday
Estée Lauder, Sephora unveil shoppable AR makeup try-ons on Pinterest. Mobile Marketer
YNAP aims to add virtual try-on through visual search — Essential Retail
Uniqlo Is Using Customers’ Photos to Create Tailored Product Recs — PSFK
Pixyle.ai offers visual search, automatic product tagging, and similar recommendations — Pixyle.ai
Cadeera is creating AI visual search for home decor — TechCrunch
Cadeera Visual Search Technology Raises Seed Money for AI-Based Home Decor Service — Startup Savant
Taxonomy — the new must-have for fashion tech — Vogue Business
AI’s role in retail — National Retail Federation
[PODCAST] Can Visual Search Help Retailers Find New Sales?

Visual Search Marketing: Brands Using Visual Search Today

Forever 21 increases AOV by 20% through visual search. Retail Touch Points
BooHoo enjoyed an 85 percent higher conversion rate for those using Camera Search, as compared to those who did not. Grit Daily
Wayfair uses visual search to improve the customer experience. Forbes
Walmart Turns Its iPhone App’s Barcode Scanner into an Augmented Reality Price Comparison Tool. Next Reality
PropertyGuru launches its own ‘Lens’ visual search tool: Tech in Asia
Tommy Hilfiger’s image recognition app drives new web traffic and engagement: The Current Daily
IKEA launches a Pinterest-embedded catalogue. Biz Journals
YouTube launches AR for cosmetics. Charged Retail
French retailer Conforama launches visual search powered by Syte. Business Wire
How Samsung is giving brands the edge with fashion visual search. The Drum
Exclusive: Levi’s to Launch Visual Search, Deploy Computer Vision WWD

Visual Search in the News

This AI can find Waldo. The Verge

How visual and voice search are revitalizing SEO. Marketing Land
Forget AR: Visual search will be the hottest e-commerce tech among millennials. Marketing Tech
Visual search is on the verge of a breakthrough. Digiday
Visual search will make a big impact in ecommerce. MediaPost
Is the world ready for the era of visual search? The National
3 ‘Vs’ of the future — voice, visual, and video — take center stage at CES. The Drum
PropertyGuru launches PropertyGuru Lens in Asia. Digital News Asia
Visual search leader Slyce acquires shopping platform Allyke. PR Newswire
Wescover is making the world a showroom. Lonny
As visual search takes off, brands adapt to shifting demands. Street Fight
Bing updates its visual search capabilities. Search Engine Journal
Publishers can use visual search to reinvigorate their offering. Ad Age
Visual search and local search: A match made in Mountain View. Street Fight
Search is getting smarter. Think with Google
The trillion-dollar visual search opportunity. Grit Daily
[2019 STUDY] Who Has the Best Image Recognition Engine? Perficient
[2019 Report] Gartner Optimistic About Visual Search As An Emerging Technology — Media Post
Snapchat launches new ‘Scan’ visual search tool. Digital Media Solutions
Snapchat launches new Augmented Reality and visual search features — The Verge
Visual search is getting crazy good. Should designers worry? Business of Home
Syte announces launch of Discovery Stories — PR Newswire

Microsoft launches Custom Visual Search for retailers. MS Power User
Visual Search Hits The Runway At Fashion Week. PYMNTS
Slyce is piloting a visual search kiosk machine at the Home Depot in South Philly. Technical.ly
Snapchat partners with PlantSnap and Dog Scanner to improve visual search. TechCrunch
Check out this new visual search engine, same.energy — Core 77
Apple goes after Google Lens with new visual search technology — 9 to 5 Google
Google adds visual search features for shoppers — Retail Dive
Google just took visual search to a whole new level — Mashable
Clarifai raises $60M in Series C funding — FinSMEs
The Future of Personalization: Marrying Human Senses With Machine Learning — Total Retail
Ecommerce Guide: Search Clothes by Image & Visual Search — YesPlz
Pinterest introduces shoppable TV feature — Chain Store Age
Snapchat’s camera can identify food and recommend recipes — Engadget
Visual search expert Clark Boyd Q&A with YesPlz.ai — YesPlz.ai
Swap.com selects FIND.Fashion for AI-powered product searches — Retail Insight Network
Search your world, any way and anywhere — Google
[PODCAST] eBay joins the visual search party. eMarketer

Visual Search Acquisitions and Fundraising

Lonely Planet to buy visual search company Trill. Travel Weekly
Visual Search Leader Slyce Acquires IP and Social-Media Monitoring Technology of Ditto Labs. PR Newswire
Donde Search picks up $6 million to bring visual search to more retailers. TechCrunch
Image recognition startup ViSenze raises $20M Series C. TechCrunch
Syte Acquires Top Visual Search Company Slyce to Expand North American Footprint — Syte

Pinterest Visual Search News

Pinterest Lens and Home Depot help people decorate a new home. Pinterest
What makes Pinterest so shoppable? ClickZ
Introducing the next wave of visual search and shopping. Pinterest
Celebrating one year of Pinterest Lens. Pinterest
Pinterest launches features for the visually impaired. Computer Weekly
Pinterest introduces Product Pins to make visual search your own shopping assistant. Engadget
Pinterest launches its Complete the Look visual search tool. Venture Beat
[VIDEO] Total Retail interview with Pinterest’s Amy Vener. Total Retail
[PODCAST] Pinterest CEO Ben Silbermann on the future of visual search. SoundCloud

Wired podcast: Pinterest CEO on the future of visual search.

Pinterest fully automates Shop the Look. Venture Beat
Pinterest launches Catalogues feature to make visual search more shoppable. WWD
The battle for the top of the funnel: Pinterest’s long road to becoming an ecommerce platform. Digiday
Pinterest adds Shop tab to visual serch results — AdWeek
Pinterest enters the online grocery frame — National Retail Federation
Pinterest launches AR eyeshadow try-on — WWD
Pinterest files for IPO. Some stats from the application:

Google Lens Visual Search News

Google introduces shoppable image ads. Search Engine Land
When in nature, Google Lens does what the human brain can’t. Wired
Copy-paste real world text with Google Lens. Android Central
Cool things you can do with Google Lens. CNET
The new PixelBook Pen performs a visual search when you circle an image. 9 to 5 Google
Google Lens comes to the Pixel 3 smartphone. TechCrunch
Google Lens comes to Google image search. Search Engine Land
Google Lens, one year in: Google
Google launches shoppable images in major move towards visual ecommerce: Google
Google Assistant adds new, visual search results: Tech Times
Google Lens to add new translation features and filters: Engadget
Helpful new visual features in Google Lens: Google
Google announces a host of new, visual features in search results: Marketing Land
Google launches Glass Enterprise Edition 2: The Verge
Here’s what Google’s bet on visual search means for advertisers: Ad Age
Google Lens and visual search will change your life. AZ Central
Google Images adds side panel and product details in web revamp. 9 to 5 Google
Google Shopping tests a Google Lens shortcut and prepares for AR item previews and dark mode. XDA Developers
Learn more about what you see on Google Images — Google
Google takes aim at Pinterest with new ‘style ideas’. Social Media Today

Google Lens announcements from I/O 2019:

Google Lens announcements presentation

Google Lens due to launch new ‘Education’ feature to help kids with homework — XDA Developers
Google Cloud launches new visual search API — Google
Google Lens now translates offline — PC Mag
Google multisearch — search by image and text at the same time — Search Engine Land

Amazon Visual Search News

Amazon partners with Getty to provide content for visual searches. TechCrunch
Snap and Amazon working on a visual search product. Fortune
Amazon launches SCOUT for visual search browsing. Digital Commerce 365
Snapchat and Amazon launch shoppable image partnership. TechCrunch
Amazon debuts Showroom visual search shopping. TechCrunch
Amazon launches StyleSnap, a new visual search tool for high-end retailers. engadget
Amazon wants to use AI to recommend you clothing — again. MIT Technology Review
Amazon discontinues the Echo Look, migrates visual search style recommendations to other Alexa apps. VentureBeat

Facebook and Instagram Visual Search News

Facebook launches visual search in Marketplace. TechCrunch
Instagram adds shoppable images. Ad Week
Instagram will let you add alt text to images. Search Engine Journal
Facebook buys visual search technology startup, GrokStyle. engadget
Instagram may soon launch a Pinterest-esque ‘collections’ feature. Business Insider
Instagram takes visual search ecommerce to the next level with one-click checkout. The Motley Fool
Instagram is letting people buy a handful of brands from a select group of influencers. Bloomberg
Social shopping is taking off, and Facebook is the biggest beneficiary. Motley Fool
Powered by AI: Advancing product understanding and building new shopping experiences. Facebook
Facebook Announces the Next Stage of its eCommerce Push, Including Shops on WhatsApp and Visual Search. Social Media Today
Instagram is testing “interest search” to make results more intuitive — Engadget

Visual Search Quotes

“The future of search will be about pictures rather than keywords.”
— Pinterest CEO, Ben Silbermann

“In the English language there’s something like 180,000 words, and we only use 3,000 to 5,000 of them. If you’re trying to do voice recognition, there’s a really small set of things you actually need to be able to recognize. Think about how many objects there are in the world, distinct objects, billions, and they all come in different shapes and sizes. So the problem of search in vision is just vastly larger than what we’ve seen with text or even with voice.”
— Clay Bavor, Google

“By now people have the muscle memory for taking pictures of all sorts of things — not just sunsets and selfies but the parking lot where you parked, business cards, books to read. That’s a massive behavior shift.”
— Aparna Chennapragada, vice president of product for AR, VR, and vision-based products at Google.

“A field linguist has gone to visit a culture whose language is entirely different from our own. The linguist is trying to learn some words from a helpful native speaker, when a rabbit scurries by. The native speaker declares “gavagai”, and the linguist is left to infer the meaning of this new word. The linguist is faced with an abundance of possible inferences, including that “gavagai” refers to rabbits, animals, white things, that specific rabbit, or “undetached parts of rabbits”. There is an infinity of possible inferences to be made. How are people able to choose the correct one?”
— Deepmind

“An advantage of visual search is that it relies entirely on item appearance. There is no need for other data such as bar codes, QR codes, product name, or other product metadata.”
— Brent Rabowsky, Amazon Web Services

Visual search: ecommerce personalisation

“You’re going to see products in stock be increasingly personalized and relevant. When you encounter a product on Pinterest, it won’t just be any product, but something that matches your tastes, style and brands you have an affinity for. The challenge with ecommerce personalization is that it only works when you have massive amounts of engagement data, not just transaction data. Our huge advantage is that we have engagement data on products. Pins links to products and as an extension, we can take all the power of personalization and directly map it to products.”
— Tim Weingarten, Pinterest (ClickZ interview)

“Being able to search the world around you is the next logical step”
— Brian Rakowski, VP Product Management, Google.

Why visual search will take off, according to Amazon

https://www.bondcap.com/report/itr19/#view/86

This is a new way to shop, allowing customers to browse millions of items and quickly refine the selection based solely on visual attributes. It is perfect for shoppers who face two common dilemmas: “I don’t know what I want, but I’ll know it when I see it” and “I know what I want, but I don’t know what it’s called.” Customers simply like an image (by clicking or tapping the thumbs up) or dislike an image (by clicking or tapping the thumbs down) to get tailored recommendations in real time and find the products that best suit their needs. Amazon uses imagery from across its robust selection to extract thousands of visual attributes for showing customers a variety of items so they can select their preferences as they go. This innovative shopping experience is powered by Machine Learning. The result is a beautiful and inspirational image feed, which gives customers the ability to explore a wide range of products in a playful and personalized manner with just a few clicks. It is currently being tested on Amazon.com and in the Amazon App for home furnishings and women’s shoes.
— Amazon statement to CNBC about new visual search platform, Amazon Scout.
“Customers are increasingly using social media platforms, such as Instagram and Pinterest, as a source of inspiration so visual search has the potential to transform how we shop for the home.”
- Mark Steel, Digital Director, Argos

Visual Search Charts and Graphs

Visual Search Use, By Age Group

https://www.retail-insight-network.com/comment/rise-of-visual-search-2019/

Who Has the Best Image Recognition Engine?

https://www.perficientdigital.com/insights/our-research/image-recognition-accuracy-study

Pinterest Visual Search: How it Works

How interested are US internet users in using visual search retail technology?

Emarketer: https://www.emarketer.com/content/how-do-smart-speakers-stack-up-against-visual-based-retail-technology

https://www.bondcap.com/report/itr19/#view/80

https://www.bondcap.com/report/itr19/#view/79

User preference for similar product results through visual search:

How people use visual search today:

Business Insider: https://www.businessinsider.com/target-to-bring-pinterest-lens-to-app-2017-9

Google Lens in action:

https://www.bondcap.com/report/itr19/#view/81

62% of millennials would like to be able to search by image:

https://www.emarketer.com/content/visual-search-2018

Importance of visual or text content across categories (consumer survey):

Number of visual searches each month through Pinterest Lens:

Pinterest Lens: Most popular visual searches of 2017:

Source: https://newsroom.pinterest.com/en/post/celebrating-one-year-of-pinterest-lens

Retailer priorities in 2018:

https://www.businessinsider.com/ebay-debuts-visual-search-for-mobile-app-2017-10?IR=T

Visual search timeline:

Source: https://www.searchenginejournal.com/visual-search-images/213324/

Google Images is the second-largest ‘search engine’:

https://sparktoro.com/blog/new-jumpshot-2018-data-where-searches-happen-on-the-web-google-amazon-facebook-beyond/

Pinterest conversion rates:

https://heapanalytics.com/blog/data-stories/4-key-facts-you-should-know-before-allocating-ad-spend

Pinterest engineering:

Amazon Visual Search:

https://aws.amazon.com/blogs/machine-learning/visual-search-on-aws-part-1-engine-implementation-with-amazon-sagemaker/

Pinterest advertising revenues:

https://www.emarketer.com/content/pinterest-makes-roughly-7-per-user

https://www.raconteur.net/technology/what-is-visual-search

https://www.emarketer.com/content/insider-intelligence-mobile-trends-watch-2022

Visual Search Trends

Integration with chatbots: Visual search starts a conversation; a fact some retailers are exploiting to integrate a chatbot into the shopping experience. Inc
Big retailers are on board: Target has been using Pinterest Lens for the past year. WSJ
Pinterest is combining visual search with text search, which should increase its reach.
Visual platforms like Instagram and Snapchat have a natural tie-in with visual search, if they can make the shopping experience as frictionless as possible.
Brands (notably, Mastercard) are dropping text from their images altogether in favor of a design-driven expression of their identity.
Google Search offers an increasingly visual experience, both through the presence of more images and also in the redesigned user interface. Basically, Google Images looks more and more like Pinterest.
Retailers are developing their own visual search capabilities, rather than relying on search engines and social networks to act as the intermediary.
Visual search technology (particularly Google Lens) is integrated into a lot more functions within smartphones now. This is a natural development; as it stands, there is something slightly jarring about having to open a specific app, click on a function, then take a picture to access visual search. The point of this technology is that it fits into how we perceive and interact with the world. We should expect to see the use of visual search become more natural and intuitive over time, with more features included within Lens.

Challenges Visual Search Must Overcome

Reporting and attribution: It is not yet possible to segment search queries to see which used an image. This is a significant challenge that has deterred some brands, but we can also expect it to be resolved with SQRs and Search Console soon.
User behaviors: People simply aren’t accustomed to searching with an image. It may take some time for this behavioral patterns to adapt, especially as the process of taking a picture before submitting it as a visual search is less than seamless.
Paid search product: It will be difficult to monetize visual search in the same way as text search, as the targeting mechanism could not focus so directly on keywords. As the industry moves towards audience-based bidding, this hesitation may be addressed. TechCrunch