A customer photographs a chair they spotted in a cafe and wants to buy it. A visitor takes a picture of a pair of sneakers seen on the street. This behavior has existed for years on Google Lens and in the apps of the large marketplaces, and it keeps coming up in discussions about the future of e-commerce search.
The question most e-commerce managers are actually asking is not "does this exist" but "do I need to deal with it now, and at what cost". This article takes stock of what visual search really covers, what works today, what it takes to set up, and which directions are taking shape.
Three different things behind one word
"Visual search" refers to at least three distinct features, with very different maturity levels and integration costs.
Search by image on your own site
The visitor uploads a photo, from their gallery or their camera, and the engine returns the visually closest products in the catalog. This is the best-known use case. It relies on comparing numerical representations of images, the same mechanism described in our article on vector search, applied to pixels rather than text.
Object detection within an image
Often called "shop the look". From a lifestyle photo containing several items, a sofa, a rug, a lamp, the system isolates each object and suggests products for each one. This is more complex than plain image search, because the image has to be segmented before anything can be compared.
Inbound traffic from an external visual engine
Here you build nothing. Someone uses Google Lens or a platform's visual tool, and one of your products shows up in the results. This channel depends on the quality of your product feed and your images, not on a component installed on your site. It is the only one of the three that concerns you even if you do nothing.
Where the technology stands
The model side has come a long way. Multimodal embeddings, the representations that place images and text in the same space, are available through standard APIs, notably on Google Cloud with Vertex AI multimodal embeddings. On paper, indexing a catalog's images and querying that index by similarity is no longer a research project.
In practice, result quality depends heavily on the type of product.
What works well
Categories where appearance carries most of the buying decision: fashion, home decor, furniture, lighting, jewelry, stationery. On a printed dress or an armchair, visual similarity maps fairly well to what the customer has in mind. These are also the categories with the poorest vocabulary: describing a pattern or a shape in words is hard, taking a photo is not.
What works poorly
Technical products, where two references that look nearly identical have decisive specs: compatibility, power, precise dimensions. A photo of a drill says nothing about the battery voltage. Cases where intent is ambiguous: a photo of a kitchen could mean "I want that food processor", "I want that worktop" or "I want that pendant light". And any catalog where images are inconsistent, with backgrounds, framing and resolutions that vary from one product page to the next.
What it costs to set up
An image-search project on your site adds four cost items that text search does not have, or not to the same degree.
Generating image embeddings. Every catalog image goes through a model once, then again whenever a visual changes. On a catalog of tens of thousands of references with several images per product, the volume is not trivial, and it repeats at every resync.
Storing and querying the image vector index. This is an index on top of the text one, with its own infrastructure cost proportional to the number of images.
Normalizing the image catalog. Often the heaviest item, because it is manual: reworking thousands of visuals to make backgrounds and framing consistent. This is a catalog project, not a technical one.
Front-end integration. A camera access button, file upload handling, the mobile case where the behavior is most natural. Nothing insurmountable, but it is dedicated development on the storefront.
Against those costs, a useful benchmark: on almost every store, the vast majority of searches are and will remain textual. Visual search is a complement for a segment of visitors, not a replacement for the search field.
What's coming
Three directions are emerging, at different stages of maturity.
Multimodal search. Combining an image and text in the same query: start from a photo of a sofa and add "but in green velvet" or "in a two-seater version". Models can technically do this; the challenge is the interface that makes the gesture natural for a shopper. This is the most concrete near-term direction.
Conversational search with an image. An exchange where the visitor sends a photo then refines through dialogue, in the logic of shopping assistants. The building blocks exist, consumer rollouts remain limited, and the real conversion upside is not yet established.
Visual suggestion from browsing. Inferring intent from the products already viewed to propose visually consistent items, without the visitor photographing anything. Promising, but still closer to classic recommendation than to search.
Is it worth dealing with now?
The answer depends first on the sector.
Fashion, decor, furniture, jewelry: yes, the topic is worth testing, at least as a pilot on part of the catalog, starting by checking the state of your images. This is where the value/cost ratio can hold up.
Every other sector: watch it, without investing. Just make sure your product feed is clean and your images are usable by an external engine, which is useful for Google Shopping anyway.
In both cases, one rule holds: visual search amplifies a layer that works, it does not fix a broken one. If your text search still returns avoidable zero-result pages, if your synonyms are not in place, if your autocomplete does not suggest products, those tasks come first. They touch 100% of searches, at a much lower cost.
First, a text search that converts
Vectail focuses on keyword and natural-language search: typo tolerance, synonyms, autocomplete, merchandising. One line of code to install, powered by Google Vertex AI.
Start free - 14 days, no credit cardOur position
Vectail does not offer image search today. Our priority is text and natural-language search, the layer that handles most of the volume and where the room for improvement is widest on most stores. We are watching multimodal search closely, because it is the evolution that will, in time, plug in most naturally to an engine already built on embeddings.
Visual search is not a fad: the behavior is real and here to stay. But for most online retailers, it is not the next project. The next project, almost always, is making flawless the search that 100% of visitors already use.