Vector Databases — What they are and how they differ from SQL / NoSQL

TL;DR: Vector databases store and query high-dimensional embeddings produced by ML models. They provide fast approximate nearest neighbor (ANN) search across millions of vectors, enabling semantic search, recommendations, and retrieval-augmented generation (RAG). They are complementary to — not a replacement for — traditional SQL/NoSQL databases.

What is a vector database?

A vector database is a purpose-built system for storing, indexing and searching vector embeddings — dense numeric arrays that capture semantic meaning of text, images, audio or other data. Instead of matching keywords, vector search finds items whose embeddings are closest to the query embedding (using cosine similarity, dot product, or Euclidean distance).

Why embeddings?

Modern AI models turn raw content into vectors (embeddings). Nearby vectors imply semantic similarity — e.g. “car” and “automobile” have nearby embeddings even if the text differs. This property enables:

• Semantic search (query by meaning, not keywords)
• Document retrieval for RAG (chat-with-your-documents)
• Similarity-based recommendations (images, products, users)
• Anomaly detection and clustering in high dimensions

How vector DBs differ from traditional databases

Key concepts

• Embedding: vector representation of an item produced by an ML model (e.g., Sentence-BERT, CLIP)
• Distance metric: cosine similarity, dot product, or Euclidean distance
• ANN (Approximate Nearest Neighbor): algorithms that return near-neighbors quickly at scale (HNSW, IVF, PQ)
• Metadata filtering: combine vector similarity with structured filters (e.g., language='en' AND date>2024-01-01)

Popular vector databases & integrations

How indexing & search works (high level)

Brute-force nearest neighbor search (computing distances to every vector) is too slow at scale. Vector DBs use ANN indexes that trade a tiny amount of accuracy for big speed and memory gains. Common approaches:

• HNSW (Hierarchical Navigable Small World) — graph-based, excellent recall and latency for billions of vectors.
• IVF (Inverted File) + PQ (Product Quantization) — coarse clustering + compressed vectors for memory efficiency.
• OPQ and hybrid techniques — rotate/transform vector space for better compression.

Architectural patterns

Most production systems combine a vector DB with other components:

• Embedding service: model that converts text/images into vectors (hosted model or cloud API)
• Vector database: index & search vectors
• Metadata store: relational DB (Postgres) or document store for attributes
• Application layer: combining vector results + business logic (re-ranking, filtering, caching)

Getting started: Python examples

Below are two minimal examples: (A) local approach with sentence-transformers + FAISS (good for prototyping), and (B) using Qdrant (a full-featured vector DB) via its Python client.

A) Prototyping locally with Sentence-Transformers + FAISS

# pip install sentence-transformers faiss-cpu
from sentence_transformers import SentenceTransformer
import numpy as np
import faiss

# 1) Create embeddings
model = SentenceTransformer('all-MiniLM-L6-v2')
docs = ["How to cook rice", "Best way to learn python", "Introduction to vector databases"]
embs = model.encode(docs, convert_to_numpy=True)

# 2) Build FAISS index (L2)
d = embs.shape[1]
index = faiss.IndexFlatL2(d)        # brute-force (fast in C) — for small data
index.add(embs)                    # add vectors

# 3) Query
query = "vector search tutorial"
q_emb = model.encode([query])
k = 2
D, I = index.search(q_emb, k)
print("Top docs:", [docs[i] for i in I[0]])

B) Production-like flow with Qdrant

# pip install sentence-transformers qdrant-client
from sentence_transformers import SentenceTransformer
from qdrant_client import QdrantClient
from qdrant_client.http.models import Distance

# 1) Prepare embeddings
model = SentenceTransformer('all-MiniLM-L6-v2')
texts = ["How to cook rice", "Best way to learn python", "Introduction to vector databases"]
embs = model.encode(texts, convert_to_numpy=True)

# 2) Start Qdrant (local or cloud). Example assumes Qdrant running locally on 6333.
client = QdrantClient(url='http://localhost:6333')

# 3) Create collection and upload
collection_name = 'docs'
client.recreate_collection(collection_name=collection_name, vector_size=embs.shape[1], distance=Distance.COSINE)
payloads = [{"text": t} for t in texts]
client.upload_records(collection_name=collection_name, records=[(i, embs[i].tolist(), payloads[i]) for i in range(len(texts))])

# 4) Query
query = "vector database guide"
q_emb = model.encode([query])[0].tolist()
res = client.search(collection_name=collection_name, query_vector=q_emb, top=3)
for hit in res:
    print(hit.payload['text'], "score:", hit.score)

Best practices

• Normalize vectors when using cosine similarity (store unit vectors).
• Combine vector score + metadata filtering to reduce false positives (e.g., language, date).
• Re-rank top-k results with lightweight ML or BM25 to improve precision.
• Monitor drift and re-index periodically as your embedding model changes.
• Shard & replicate the index for high availability at scale (managed services make this easier).

When NOT to use a vector DB

If you only need transactional queries, strict ACID semantics, complex joins or analytics, a relational DB (Postgres, MySQL) or a document store (MongoDB) is still the right choice. Vector DBs complement—not replace—those systems.

Summary

Vector databases are a foundational piece of modern AI systems: they make semantic retrieval fast and scalable. Choose a local library (FAISS) to prototype, an open-source DB (Qdrant, Milvus, Weaviate) for control, or a managed service (Pinecone) for rapid production deployment. Combine vector search with metadata and re-ranking to deliver accurate, explainable results.