Embeddings and Vector Search Explained (With an Example)

A visitor searches your documentation for “let a colleague manage invoices”, but the relevant page is titled “Billing administrator roles”. Exact keyword matching may miss the connection. Embeddings offer another signal: represent both the question and document passages as numerical vectors, then compare their positions in the same model's space.

That mechanism is useful, but it is not understanding you can trust without testing. Similarity does not establish permission, factual correctness, or whether a passage answers the actual question. A practical implementation combines vector search with content preparation, metadata, and a small set of deliberately difficult examples.

Understand what an embedding represents

An embedding is an ordered list of numbers produced by a model for an input such as text. The model is trained so that relationships useful to its training objective are reflected in the geometry of those lists. Applications compare vectors rather than interpreting individual coordinates as human-readable labels.

Two passages about account permissions may have similar vectors even when they share few words. Conversely, two passages sharing the word “account” may concern unrelated tasks. The quality of those relationships depends on the model, language, domain, and how well the input resembles the material it handles effectively.

Keep model compatibility explicit. Query and document vectors must come from a compatible embedding setup. Matching dimensionality alone is not enough: two models can produce equally long vectors whose coordinates mean different things. A model migration normally requires re-embedding the corpus and rebuilding the relevant index.

Some embedding systems distinguish query and document input modes. Follow the model's instructions for these modes and any required prefixes. The Sentence Transformers documentation is a useful reference for semantic search concepts and the distinction between symmetric and asymmetric search tasks.

Build a small documentation example

Start with three passages: assigning billing roles, updating payment details, and inviting workspace members. Keep each passage focused but preserve the heading and important qualifications. For example, a billing role may only be assignable by an owner; removing that sentence changes the usefulness of the result.

Create a document record containing a stable ID, source URL, title, text, version, and access metadata. Generate an embedding from the relevant title and passage text. Store the vector alongside the record, not instead of it. The user needs an understandable result, and your application needs a source to inspect.

When the visitor submits the invoice question, embed it using the compatible query setup. Compare that vector with the stored document vectors, return the best candidates, and display their titles and excerpts. No answer-generating model is required for this first version.

A provider-neutral sketch shows the boundary:

const queryVector = await embedQuery(question);
const candidates = await searchVectors({
vector: queryVector,
limit: 8,
filters: {
  workspaceId: session.workspaceId,
  version: currentDocsVersion
}
});
const visibleResults = candidates.map(({ id, title, url, excerpt }) => ({
id, title, url, excerpt
}));

The search adapter must enforce those filters, not merely accept and ignore them. Add an integration test with a highly similar document from a different workspace to verify that it never appears in results.

Choose a similarity measure deliberately

Cosine similarity compares vector direction, while dot product also depends on magnitude unless vectors are normalised. Euclidean distance measures straight-line distance. Use the metric recommended by the embedding model and ensure the database index uses the corresponding operator.

For a tiny illustrative example, imagine normalised vectors with only two coordinates. A query vector of [1, 0] has a cosine similarity of 0.8 with [0.8, 0.6] and zero with [0, 1]. Real embeddings usually have many more coordinates, but the comparison still produces a ranking rather than a proof that a document answers a question.

Do not present the similarity score as a probability of correctness. A score of 0.8 does not mean an 80 percent chance that the result is right. Score distributions vary with the model, query, corpus, and metric. Any acceptance threshold needs calibration against labelled examples from your application.

The pgvector documentation describes supported distance operators and indexes. Exact search provides a useful baseline. Approximate indexes trade some recall for speed, so compare them with that baseline before deciding whether their latency benefit is worth missed neighbours.

Improve chunks, metadata, and hybrid retrieval

Chunk boundaries can matter more than a new database. A whole documentation site is too broad as one vector, while a sentence stripped of its subject may be too vague. Use headings and coherent sections, then inspect actual failures to adjust passage size and overlap.

Use the word counter to compare the lengths of sample passages while reviewing chunk consistency. Word counts are a rough editorial signal, not an embedding token limit. Check the model's tokenizer or documented input limit before sending production content, especially for code and multilingual pages.

Store metadata filters for language, version, tenant, publication status, and other real constraints. Similarity should rank eligible material, not decide who is allowed to read it. Include these constraints in caches as well as database queries, and remove stale vectors when documents are deleted.

Add lexical search for exact identifiers, error messages, and command flags. A hybrid search system combines complementary rankings; it should not blindly average incomparable raw scores. Rank fusion or a calibrated reranker can be more sensible, but each extra stage adds latency and another component to evaluate.

The PostgreSQL full-text search documentation covers a lexical baseline. Keep that baseline during experiments so you can see whether semantic retrieval improves actual tasks rather than just producing interesting-looking neighbours.

Evaluate search before generating answers

Build a labelled query set containing natural paraphrases, exact terms, ambiguous requests, and questions with no relevant document. For each query, identify acceptable source passages. Measure whether those passages appear among the top results and review where irrelevant results outrank them.

Track recall at your chosen result count, ranking quality, response time, and permission violations. Test rare terms and different languages separately rather than letting common English queries hide coverage gaps. Include newly published and recently deleted pages to verify that indexing behaviour matches the content lifecycle.

Inspect failure categories before changing models. The problem may be a missing document, an oversized chunk, an incorrect filter, or an approximate index returning too few eligible candidates. Preserve model and index versions so changes can be compared reproducibly.

Only add generation after search is useful. The RAG guide explains how to supply retrieved passages as evidence and validate citations. The structured output guide covers validating machine-readable answers. Neither technique rescues irrelevant retrieval; a polished answer can simply make a weak search result harder to notice.

Key takeaways

• Embed queries and documents with a compatible model configuration.
• Treat similarity as a ranking signal, not a correctness probability.
• Preserve meaningful chunks, source metadata, and access constraints.
• Compare semantic search with lexical and exact-search baselines.
• Evaluate retrieved evidence before adding answer generation.