Embeddings and vector search, explained for non-engineers

When a company says its assistant "searches your documents," it is usually using embeddings. They are the reason a search for "refund policy" can find a paragraph titled "returns and credits."

What an embedding is

OpenAI's documentation defines an embedding as a list of floating-point numbers that represents a piece of text, where the distance between two embeddings indicates how related the texts are ^[1]. A model reads each passage once and produces its list. To search, the system converts the question into a list the same way and returns the stored passages whose lists are nearest. Because the comparison is on meaning-bearing numbers rather than shared words, it finds relevant text that uses different vocabulary.

OpenAI recommends cosine similarity for the comparison and notes that the choice of distance function usually matters little ^[1].

What it is used for

The same guide lists six uses: search, clustering (grouping similar text), recommendations, anomaly detection (flagging items unlike the rest), diversity measurement and classification ^[1]. In practice, the first is the foundation of retrieval-augmented generation: find the right passages, then hand them to a language model to answer from.

Models differ in measurable ways

Source: OpenAI embeddings guide ^[1]. MTEB is a public embedding benchmark; higher is better.

The gap between models is a few points, while the cost and storage differ more: larger vectors take more space and time to search. OpenAI's guide estimates roughly 62,500 pages per dollar for the small model versus about 9,615 for the large one, assuming around 800 tokens per page ^[1].

Where the vectors live

Vector search does not require a separate product. The open-source pgvector extension adds vector storage and similarity search to Postgres, with exact search by default (perfect recall) and optional approximate indexes (HNSW or IVFFlat) that are faster at some cost to recall ^[2]. For many internal document collections, keeping vectors next to the rest of the data is simpler to secure and operate.

Questions to ask a vendor

1. Which embedding model is used, and can it be changed without re-processing everything?

2. Where are the vectors stored, and who can query them?

3. How is search quality measured on our documents, not on a public benchmark?

For businesses: embeddings make a document collection searchable by meaning at low cost, but the quality of the result depends on your documents and on testing it against questions your staff actually ask.