What is a Vector Embedding?

A vector embedding is a numerical representation of a piece of content — text, image, audio, code — as a list of floating-point numbers in a high-dimensional space, arranged so that semantically similar inputs land at nearby coordinates. Embeddings are the primary mechanism for similarity search, retrieval-augmented generation, recommendation systems, and clustering in modern AI applications.

How embeddings work

An embedding model is a neural network trained to map inputs to fixed-dimensional vectors (commonly 768, 1024, 1536, or 3072 dimensions). The model is trained so that pairs of inputs with similar meaning produce vectors with high cosine similarity, while unrelated inputs produce vectors that are nearly orthogonal.

The vector itself isn't human-readable — its coordinates have no individual meaning. The geometry is the meaning: distances and angles between vectors encode semantic relationships.

Common embedding models:

• OpenAI — text-embedding-3-small (1536d), text-embedding-3-large (3072d)
• Cohere — embed-english-v3.0 (1024d), multilingual variants
• Voyage — domain-specific embeddings (legal, code, finance)
• Open-source — BGE, E5, Nomic, all-MiniLM
• Sentence Transformers — academic baseline family

Where embeddings are used

• Semantic search — find documents by meaning, not just keyword overlap
• Retrieval-Augmented Generation (RAG) — fetch relevant context for an LLM
• Clustering and topic discovery — group similar items without labels
• Recommendations — find items "like this one" in user-facing applications
• Anomaly detection — flag inputs whose embeddings sit far from any cluster

Security implications

Embeddings are not one-way functions. Three concrete risks:

• Embedding inversion — research has demonstrated that with access to embeddings, attackers can reconstruct approximate original text. Embeddings of sensitive documents are themselves sensitive.
• Embedding-space poisoning — attackers craft adversarial documents whose embeddings cluster near a target query, manipulating what gets retrieved without the documents needing to be human-relevant.
• Privacy leakage via similarity — querying an embedding store and observing which results return reveals what's in the corpus. Stores containing PII or proprietary information need access controls at query time, not just at index time.

For long-form coverage including specific defenses, see Repello's research on vector embedding security (linked below).