Vector Database Explained: How Similarity Search Works, From Cosine to HNSW

By Pushpjeet Cholkar, 17 years of corporate experience, starting in databases and now working in cloud, AI and ML.

Last updated: October 22, 2026

Short answer: A vector database stores embeddings (number lists that represent the meaning of text, images or other data) and returns the stored vectors closest to a query vector. Instead of exact matches, it ranks results by cosine similarity, dot product or Euclidean distance, and uses approximate indexes such as HNSW or IVF to avoid scanning every row.

If embeddings are new to you, read What are embeddings? first. This post picks up where that one stops: you have vectors, and now you need to store and search millions of them.

I started my career in databases, so my first reaction to vector databases was “why not just add an index?” The short version of this article is why that doesn’t work, and what these systems do instead.

What is a vector database?

Most business data used to be structured: rows and columns with a fixed schema, like a book catalog with ISBN, year and author, which fits PostgreSQL or MySQL. Semi-structured data, such as JSON documents with optional fields, fits document and key-value stores. Unstructured data (emails, support tickets, PDFs, images, audio, sensor logs) has no fixed format and can’t be searched by meaning with WHERE and ORDER BY.

Machine learning bridges that gap. An embedding model turns each piece of unstructured content into a vector, a fixed-length list of numbers, positioned so that similar meaning lands close together:

unstructured data → embedding model → vector → vector database

A vector database is the system that stores those vectors and answers one question fast: which stored vectors are closest to this query vector? In practice it stores three things per item:

1. The vector, for example 1,024 floating-point numbers.
2. The payload: the original text chunk, or a pointer to the image or document.
3. Metadata: tenant, source, date, access level, language. You filter on these.

Around that it adds what you expect from a database: inserts, updates and deletes, an index built for similarity search, filtering, access control, durability and scaling.

Why can’t an ordinary database index do this?

A B-tree index works because values can be sorted. WHERE price BETWEEN 10 AND 20 walks to 10 and reads forward. Even a composite index sorts by the first column, then the second.

Similarity search has no such order. The “closeness” of two 1,024-dimensional vectors depends on all 1,024 numbers at once, and two vectors can be close overall while differing a lot in any single dimension. Sorting by dimension 1, then dimension 2, tells you almost nothing about overall distance. Classic spatial indexes such as k-d trees and R-trees work well in two or three dimensions but lose their advantage as dimensions grow, which is often called the curse of dimensionality.

Without a special index, the only exact method is to compare the query with every stored vector. That costs $N \times d$ multiply-adds per query, where $N$ is the number of vectors and $d$ the dimension: 12.8 million for the 100,000 × 128 data set in the demo below. That’s fine for thousands of vectors and too slow for many millions at high query rates. Vector databases exist to avoid that full scan.

How do you measure “close”? Cosine, dot product and Euclidean distance

Every vector search uses a similarity or distance function. Three dominate. For two vectors $\mathbf{a}$ and $\mathbf{b}$ with $d$ dimensions:

Dot product (inner product)

$$\mathbf{a} \cdot \mathbf{b} = \sum_{i=1}^{d} a_i b_i$$

• $a_i$ and $b_i$ are the $i$-th numbers in each vector. Multiply matching positions and add them up.
• Higher means more similar. It grows with the vectors’ lengths as well as their direction.

Cosine similarity

$$\cos(\theta) = \frac{\mathbf{a} \cdot \mathbf{b}}{\lVert \mathbf{a} \rVert \, \lVert \mathbf{b} \rVert}, \qquad \lVert \mathbf{a} \rVert = \sqrt{\sum_{i=1}^{d} a_i^2}$$

• $\theta$ is the angle between the vectors, and $\lVert \mathbf{a} \rVert$ is the length (L2 norm) of $\mathbf{a}$.
• Dividing by both lengths keeps only direction. The result runs from $-1$ (opposite) to $1$ (same direction).
• Some systems report cosine distance, $1 – \cos(\theta)$, where lower is better. pgvector’s <=> operator returns cosine distance.

Euclidean (L2) distance

$$d(\mathbf{a}, \mathbf{b}) = \lVert \mathbf{a} – \mathbf{b} \rVert = \sqrt{\sum_{i=1}^{d} (a_i – b_i)^2}$$

• The straight-line distance between the two points. Lower means more similar.

Why the choice often doesn’t matter, and when it does. If every vector is normalized to length 1, then

$$\lVert \mathbf{a} – \mathbf{b} \rVert^2 = \lVert \mathbf{a} \rVert^2 + \lVert \mathbf{b} \rVert^2 – 2\, \mathbf{a} \cdot \mathbf{b} = 2 – 2\cos(\theta)$$

so all three rank results in the same order: the largest dot product is the largest cosine and the smallest distance. The demo checks this on a toy pair: cosine 0.960, squared Euclidean distance 0.080, and $2 – 2 \times 0.960 = 0.080$. When vectors are not normalized, the metrics disagree. In the same toy example, making one vector 10 times longer leaves cosine at 0.960 but multiplies the dot product by 10 and moves the Euclidean distance from 1.41 to 45.22. Use the metric your embedding model’s documentation recommends, and normalize if you plan to use the dot product as cosine. The dot product section of Mathematics for Machine Learning covers the underlying algebra.

Each engine names these differently, which trips people up:

Exact search vs approximate nearest neighbor (ANN) search

Exact k-nearest-neighbor (kNN) search scores every vector and returns the true top $k$. It is simple and always correct, and it is what pgvector does by default when there is no index.

Approximate nearest neighbor (ANN) search uses an index to score only a small, promising subset of vectors. It is much faster, but it can miss some true neighbors. The quality measure is recall@k:

$$\text{recall@}k = \frac{1}{|Q|} \sum_{q \in Q} \frac{|A_q \cap E_q|}{k}$$

• $Q$ is the set of test queries and $|Q|$ is how many there are.
• $E_q$ is the exact top $k$ for query $q$, and $A_q$ is the top $k$ the ANN index returned.
• $|A_q \cap E_q|$ counts how many true neighbors the index found. A recall@10 of 0.95 means that, on average, 9.5 of the true 10 came back.

Every ANN index has a knob that trades recall for speed (nprobe for IVF, ef_search for HNSW). Measure recall on your own data and queries; the right setting depends on both.

Vector index types: flat, IVF, HNSW and PQ

Flat (brute force)

No index structure: store the vectors and scan them all. Exact, no build time, and the right choice for small collections or as the ground truth when you measure recall.

IVF (inverted file index)

Cluster the vectors with k-means into nlist groups (lists). Each vector goes into the list of its nearest centroid. At query time, find the nprobe centroids closest to the query and scan only those lists. Per query, the work is roughly

$$\text{vectors scored} \approx \text{nlist} + \text{nprobe} \times \frac{N}{\text{nlist}}$$

With $N$ = 100,000, nlist = 316 and nprobe = 16 that is about 5,400 comparisons instead of 100,000; the demo measured 5,085 vectors scored, plus the 316 centroids. IVF builds fast and uses little extra memory, but recall drops when a true neighbor sits just across a cluster boundary, so it needs a reasonable nprobe. IVF needs a training step (the k-means), so train on data that looks like what you’ll store.

HNSW (hierarchical navigable small world)

Build a multi-layer graph. Every vector is a node linked to some of its near neighbors; upper layers hold a few nodes with long-range links, and the bottom layer holds everything. A search starts at the top, greedily hops toward the query, drops a layer, and repeats, keeping a candidate list of size ef_search at the bottom. The main parameters:

• M (pgvector calls it m): links per node. More links give better recall and more memory.
• ef_construction: candidate list size while building. Higher gives a better graph and a slower build.
• ef_search: candidate list size while querying. Higher gives better recall and slower queries.

HNSW usually gives the best recall-for-speed trade-off when memory allows. The costs are memory (the links) and build time. Default values differ between engines: pgvector uses m 16, ef_construction 64 and hnsw.ef_search 40; OpenSearch’s Faiss engine defaults to m 16, ef_construction 100 and ef_search 100.

PQ (product quantization)

A compression method rather than a search structure, usually combined with IVF (IVF-PQ) or HNSW. Split each $d$-dimensional vector into $m$ sub-vectors, run k-means on each sub-space with 256 centroids, and store each sub-vector as the 1-byte ID of its nearest centroid. A 128-dimensional float32 vector (512 bytes) with $m$ = 16 becomes 16 bytes. Distances are then estimated from the codes, so recall drops; the usual fix is to fetch extra candidates and re-rank them with the full vectors.

Index comparison

Python demo: exact search vs IVF vs HNSW, with recall and timing

The script below builds everything that matters from scratch in NumPy, then repeats the comparison with Faiss, the vector search library many engines build on. It uses synthetic data so it runs anywhere without downloading a model: 100,000 unit-length vectors with 128 dimensions, scattered around 1,000 random “topic” centers, plus 300 new query vectors drawn the same way. Real embeddings behave differently, so treat the recall numbers as an illustration of the trade-offs, not a benchmark.

It runs five experiments:

1. The three metrics on a toy pair of 2-D vectors.
2. Exact search: score all 100,000 vectors, keep the top 10. This is the ground truth.
3. An IVF index built from scratch: spherical k-means into 316 lists, then search with nprobe from 1 to 128.
4. Metadata filtering: about 10% of vectors belong to “tenant 7”; compare four ways to filter.
5. The same data in Faiss: Flat, IVF, HNSW and IVF-PQ, with recall, latency, build time and bytes per vector.

Install and run (Python 3.13.5, NumPy 2.5.4, faiss-cpu 1.15.1; the script runs parts 1 to 4 with NumPy alone and skips part 5 if Faiss isn’t installed):

pip install numpy faiss-cpu
python3 vector_db_demo.py

"""Vector database explained: exact search vs approximate (ANN) search, from scratch.

Part 1  The three distance metrics on two tiny vectors.
Part 2  Exact k-nearest-neighbor search (brute force) in NumPy: the ground truth.
Part 3  An IVF index built from scratch in NumPy (k-means lists + nprobe).
Part 4  Metadata filtering: post-filter vs filter-during-search vs exact pre-filter.
Part 5  Optional: the same data in Faiss (flat, IVF, HNSW, IVF-PQ), if faiss-cpu is installed.

Data is synthetic: 100,000 unit-length 128-D vectors drawn around 1,000 random
"topic" centers, plus 300 new query vectors from the same distribution.
Recall depends on the data; run it on your own embeddings before you trust any number.
Run: python3 vector_db_demo.py
"""
import os

for var in ("OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS"):
    os.environ.setdefault(var, "1")   # single thread, so timings compare like with like

import platform
import time

import numpy as np

SEED = 42
NOISE_SD = 1.5   # how loosely vectors sit around their topic; higher = harder search
N, D, TOPICS, N_QUERIES, K = 100_000, 128, 1_000, 300, 10
rng = np.random.default_rng(SEED)


def normalize(x):
    return (x / np.linalg.norm(x, axis=1, keepdims=True)).astype(np.float32)


def make_vectors(n, centers):
    topic = rng.integers(0, len(centers), n)
    return normalize(centers[topic] + rng.normal(0, NOISE_SD, (n, centers.shape[1])))


def top_k(scores, k):
    """Indices of the k largest scores, best first."""
    idx = np.argpartition(-scores, k)[:k]
    return idx[np.argsort(-scores[idx])]


def recall_at_k(found, truth):
    return np.mean([len(set(f) & set(t)) / len(t) for f, t in zip(found, truth)])


def ms_per_query(fn, queries):
    start = time.perf_counter()
    results = [fn(q) for q in queries]
    return results, (time.perf_counter() - start) * 1000 / len(queries)


print(f"Python {platform.python_version()}, NumPy {np.__version__}, single thread")

# Part 1: distance metrics ------------------------------------------------------
print("\n1) Three ways to compare two vectors")
a, b = np.array([3.0, 4.0]), np.array([4.0, 3.0])
b_long = 10 * b                      # same direction as b, 10x longer
for name, v in (("b = [4, 3]", b), ("10b = [40, 30]", b_long)):
    cos = a @ v / (np.linalg.norm(a) * np.linalg.norm(v))
    print(f"  a = [3, 4] vs {name:<15} cosine {cos:.3f} | dot {a @ v:7.1f} | euclidean {np.linalg.norm(a - v):6.2f}")
ua, ub = a / np.linalg.norm(a), b / np.linalg.norm(b)
print(f"  unit vectors: dot {ua @ ub:.3f}, squared euclidean {np.sum((ua - ub) ** 2):.3f}, "
      f"2 - 2*cos {2 - 2 * (ua @ ub):.3f}")

# Data -------------------------------------------------------------------------
centers = rng.normal(0, 1.0, (TOPICS, D))
X = make_vectors(N, centers)                 # the "database"
Q = make_vectors(N_QUERIES, centers)         # new questions, not in the database
print(f"\nDatabase: {N:,} vectors x {D} dims (float32, {X.nbytes / 2**20:.1f} MiB); {N_QUERIES} queries; k = {K}")

# Part 2: exact search ---------------------------------------------------------
print("\n2) Exact search (brute force): score every vector, keep the top k")
exact, exact_ms = ms_per_query(lambda q: top_k(X @ q, K), Q)
print(f"  {exact_ms:.2f} ms per query, {N:,} vectors scored per query, recall@{K} = 1.000 by definition")

# Part 3: IVF from scratch -----------------------------------------------------
NLIST = 316                                  # about sqrt(N) lists
print(f"\n3) IVF index from scratch: k-means into {NLIST} lists, search only the nprobe closest lists")
t0 = time.perf_counter()
centroids = X[rng.choice(N, NLIST, replace=False)].copy()
for _ in range(10):                          # spherical k-means (cosine)
    assign = np.argmax(X @ centroids.T, axis=1)
    for c in range(NLIST):
        members = X[assign == c]
        if len(members):
            centroids[c] = members.mean(axis=0)
    centroids = normalize(centroids)
assign = np.argmax(X @ centroids.T, axis=1)
order = np.argsort(assign, kind="stable")    # store each list contiguously
X_sorted, ids_sorted = X[order], order
offsets = np.concatenate([[0], np.cumsum(np.bincount(assign, minlength=NLIST))])
sizes = np.diff(offsets)
print(f"  build (10 k-means rounds): {time.perf_counter() - t0:.1f} s; list sizes min {sizes.min()}, "
      f"median {int(np.median(sizes))}, max {sizes.max()}")


def ivf_search(q, nprobe, k=K, mask=None):
    lists = top_k(centroids @ q, nprobe)
    rows = np.concatenate([np.arange(offsets[c], offsets[c + 1]) for c in lists])
    if mask is not None:                     # filter during search
        rows = rows[mask[ids_sorted[rows]]]
    scores = X_sorted[rows] @ q
    best = top_k(scores, k) if len(rows) > k else np.argsort(-scores)
    return ids_sorted[rows[best]], len(rows)


print(f"  {'nprobe':>6} {'recall@10':>9} {'ms/query':>9} {'vectors scored':>15} {'speed-up':>9}")
for nprobe in (1, 4, 16, 64, 128):
    res, ms = ms_per_query(lambda q: ivf_search(q, nprobe), Q)
    found, scanned = [r[0] for r in res], np.mean([r[1] for r in res])
    print(f"  {nprobe:>6} {recall_at_k(found, exact):>9.3f} {ms:>9.2f} {scanned:>15,.0f} {exact_ms / ms:>8.1f}x")

# Part 4: metadata filtering ---------------------------------------------------
print("\n4) Metadata filter: only vectors with tenant == 7 (10 tenants, so about 10% of rows)")
tenant = rng.integers(0, 10, N)
allowed = tenant == 7
truth_f = [top_k(np.where(allowed, X @ q, -np.inf), K) for q in Q]   # exact filtered answer
NPROBE = 64


def post_filter(q, fetch):
    ids, _ = ivf_search(q, NPROBE, k=fetch)
    return ids[allowed[ids]][:K]


strategies = (
    ("post-filter, fetch 10", lambda q: post_filter(q, 10)),
    ("post-filter, fetch 100", lambda q: post_filter(q, 100)),
    ("filter during search", lambda q: ivf_search(q, NPROBE, mask=allowed)[0]),
    ("exact pre-filter", lambda q: np.flatnonzero(allowed)[top_k(X[allowed] @ q, K)]),
)
print(f"  {'strategy (IVF nprobe=64 unless exact)':<38} {'avg results':>11} {'recall@10':>9} {'ms/query':>9}")
for name, fn in strategies:
    res, ms = ms_per_query(fn, Q)
    print(f"  {name:<38} {np.mean([len(r) for r in res]):>11.1f} {recall_at_k(res, truth_f):>9.3f} {ms:>9.2f}")

# Part 5: Faiss ----------------------------------------------------------------
try:
    import faiss
except ImportError:
    print("\n5) faiss-cpu not installed; skipping the library comparison (pip install faiss-cpu)")
    raise SystemExit
faiss.omp_set_num_threads(1)
print(f"\n5) Same data in Faiss {faiss.__version__} (inner product on unit vectors = cosine)")


def faiss_run(index, label, build_s):
    found, ms = ms_per_query(lambda q: index.search(q[None, :], K)[1][0], Q)
    size = faiss.serialize_index(index).nbytes / N
    print(f"  {label:<26} {recall_at_k(found, exact):>9.3f} {ms:>9.3f} {build_s:>8.1f} {size:>13.0f}")


print(f"  {'index':<26} {'recall@10':>9} {'ms/query':>9} {'build s':>8} {'bytes/vector':>13}")
t0 = time.perf_counter(); flat = faiss.IndexFlatIP(D); flat.add(X)
faiss_run(flat, "Flat (exact)", time.perf_counter() - t0)

t0 = time.perf_counter()
quantizer = faiss.IndexFlatIP(D)
ivf = faiss.IndexIVFFlat(quantizer, D, NLIST, faiss.METRIC_INNER_PRODUCT)
ivf.train(X); ivf.add(X)
build = time.perf_counter() - t0
for nprobe in (16, 64):
    ivf.nprobe = nprobe
    faiss_run(ivf, f"IVF{NLIST},Flat nprobe={nprobe}", build)

t0 = time.perf_counter()
hnsw = faiss.IndexHNSWFlat(D, 32, faiss.METRIC_INNER_PRODUCT)
hnsw.hnsw.efConstruction = 100
hnsw.add(X)
build = time.perf_counter() - t0
for ef in (16, 64, 256):
    hnsw.hnsw.efSearch = ef
    faiss_run(hnsw, f"HNSW32 efSearch={ef}", build)

t0 = time.perf_counter()
quantizer_pq = faiss.IndexFlatIP(D)
ivfpq = faiss.IndexIVFPQ(quantizer_pq, D, NLIST, 16, 8, faiss.METRIC_INNER_PRODUCT)
ivfpq.train(X)
refined = faiss.IndexRefineFlat(ivfpq)   # keeps full vectors to re-rank PQ candidates
refined.add(X)                           # adds to the PQ index and the full-vector store
build = time.perf_counter() - t0
ivfpq.nprobe = 64
faiss_run(ivfpq, f"IVF{NLIST},PQ16 nprobe=64", build)
refined.k_factor = 10                    # fetch 10 x k PQ candidates, re-score exactly
found, ms = ms_per_query(lambda q: refined.search(q[None, :], K)[1][0], Q)
size = (faiss.serialize_index(ivfpq).nbytes + X.nbytes) / N
print(f"  {'  + exact re-rank of 100':<26} {recall_at_k(found, exact):>9.3f} {ms:>9.3f} {build:>8.1f} {size:>13.0f}")

Output from this machine (a cloud VM with 8 vCPUs, Intel Xeon, one thread, run on October 22, 2026). Timings will differ on your hardware. The recall, “vectors scored” and bytes-per-vector columns came out identical on two runs and should match with the same library versions:

Python 3.13.5, NumPy 2.5.4, single thread

1) Three ways to compare two vectors
  a = [3, 4] vs b = [4, 3]      cosine 0.960 | dot    24.0 | euclidean   1.41
  a = [3, 4] vs 10b = [40, 30]  cosine 0.960 | dot   240.0 | euclidean  45.22
  unit vectors: dot 0.960, squared euclidean 0.080, 2 - 2*cos 0.080

Database: 100,000 vectors x 128 dims (float32, 48.8 MiB); 300 queries; k = 10

2) Exact search (brute force): score every vector, keep the top k
  1.86 ms per query, 100,000 vectors scored per query, recall@10 = 1.000 by definition

3) IVF index from scratch: k-means into 316 lists, search only the nprobe closest lists
  build (10 k-means rounds): 1.8 s; list sizes min 199, median 318, max 475
  nprobe recall@10  ms/query  vectors scored  speed-up
       1     0.594      0.04             323     47.7x
       4     0.668      0.10           1,279     19.0x
      16     0.770      0.39           5,085      4.8x
      64     0.901      1.42          20,252      1.3x
     128     0.965      2.68          40,508      0.7x

4) Metadata filter: only vectors with tenant == 7 (10 tenants, so about 10% of rows)
  strategy (IVF nprobe=64 unless exact)  avg results recall@10  ms/query
  post-filter, fetch 10                          1.0     0.104      1.43
  post-filter, fetch 100                         8.9     0.789      1.39
  filter during search                          10.0     0.811      0.32
  exact pre-filter                              10.0     1.000      1.08

5) Same data in Faiss 1.15.1 (inner product on unit vectors = cosine)
  index                      recall@10  ms/query  build s  bytes/vector
  Flat (exact)                   1.000     1.836      0.0           512
  IVF316,Flat nprobe=16          0.749     0.135      2.2           522
  IVF316,Flat nprobe=64          0.895     0.459      2.2           522
  HNSW32 efSearch=16             0.915     0.051     23.7           784
  HNSW32 efSearch=64             0.972     0.144     23.7           784
  HNSW32 efSearch=256            0.993     0.485     23.7           784
  IVF316,PQ16 nprobe=64          0.379     0.134      4.6            27
    + exact re-rank of 100       0.821     0.161      4.6           539

What the output shows

Part 1, metrics. Cosine ignores length; the dot product and Euclidean distance don’t. After normalization, squared Euclidean distance equals $2 – 2\cos(\theta)$ exactly (0.080 both ways), so the rankings agree.

Part 2, exact search. Scoring all 100,000 vectors took 1.86 ms per query here. That is already fast at this size, which is a useful reminder: below a few hundred thousand vectors, a flat index (or pgvector with no index) can be good enough. The cost grows linearly with $N$, so ten times the data means roughly ten times the time.

Part 3, IVF from scratch. With nprobe = 1 the index scored only 323 vectors per query and was about 48 times faster, but it found just 59.4% of the true top 10. Raising nprobe raised recall (77.0% at 16, 90.1% at 64, 96.5% at 128) and cost. At nprobe = 128 the NumPy version was slower than brute force (0.7x), because it still scored 40% of the data and paid Python overhead to gather the lists. An ANN index only pays off when it skips most of the data, and a tuned library implementation matters: Faiss’s IVF at nprobe = 64 reached 89.5% recall in 0.459 ms, against 1.42 ms for my NumPy version with similar recall.

Part 5, HNSW and PQ. HNSW gave the best trade-off: 91.5% recall at 0.051 ms (about 36 times faster than Faiss’s flat scan), 97.2% at 0.144 ms, and 99.3% at 0.485 ms. The price was a 23.7-second build, against 2.2 seconds for IVF, and 784 bytes per vector instead of 512 for the raw float32 vector, because of the graph links. IVF-PQ shrank each vector to 27 bytes (16 bytes of PQ code plus an 8-byte ID and a share of the codebooks and centroids), about 19 times smaller than raw, but recall fell to 37.9%. Re-ranking the top 100 PQ candidates with the full vectors brought it back to 82.1%; the remaining gap comes from the IVF step, which on its own reached 89.5% at the same nprobe.

Part 4, filtering, gets its own section next, because it’s the easiest mistake to miss.

How does metadata filtering work in a vector database?

Real queries are rarely “search everything”. They are “search this customer’s documents”, “only English”, “only documents this user may see”. There are three ways to combine a filter with vector search:

• Post-filtering: run the ANN search for the top $k$, then drop results that fail the filter. Simple, and dangerous with selective filters. In the demo, with a filter matching about 10% of rows, asking the IVF index for 10 results and then filtering left 1.0 result on average (recall 0.104). Fetching 100 and then filtering left 8.9 results (recall 0.789): better, but still short of 10, and about four times slower than filtering during the search.
• Filtering during the search: the index skips non-matching vectors while it searches, so it keeps going until it has $k$ matches. In the demo this returned all 10 results with recall 0.811, and it was the fastest option (0.32 ms), because it scored fewer vectors.
• Pre-filtering with exact search: select the matching rows first (for example with a normal B-tree or GIN index on the metadata), then score only those exactly. Recall is 1.000 by definition, and when the filter is selective enough the scan is small: about 10,000 rows (10% of the data) took 1.08 ms here.

Engines document the same behavior:

• The pgvector README states that with approximate indexes, filtering is applied after the index is scanned. With the default hnsw.ef_search of 40 and a condition matching 10% of rows, only about 4 rows match on average. Since pgvector 0.8.0, iterative index scans keep scanning until enough rows match. The Amazon Bedrock user guide recommends enabling them (hnsw.iterative_scan) when you use metadata filtering with an Aurora PostgreSQL knowledge base.
• The Amazon OpenSearch Service k-NN documentation warns that combining a knn query with a post_filter clause can return fewer than k results. In the open-source OpenSearch engine comparison, Lucene’s HNSW filters during the search, while the Faiss engine’s HNSW and IVF post-filter.
• Amazon S3 Vectors supports filtering on metadata attached to each vector. According to the S3 FAQ, on indexes whose index mode is ENHANCED, the filter is evaluated before the similarity search (pre-filtering) for higher recall on filtered queries.

Design rule: if a filter is selective (one tenant among many, a narrow date range), test recall with the filter on, not just on unfiltered queries. For hard multi-tenant isolation, consider a separate index per tenant rather than a filter. The pgvector README notes that tenants sharing one approximate index can affect each other’s recall and speed.

How does a vector database fit into a RAG pipeline?

Retrieval-augmented generation (RAG) is the most common reason teams adopt a vector database today. Documents are split into chunks, each chunk is embedded once and stored with its text and metadata, and every question is embedded with the same model and used as a k-NN query. The retrieved chunks go into the prompt.

The RAG tutorial in Python builds this pipeline end to end. Its step 3 uses a NumPy array as the vector index (exactly the flat index above), and its next step section moves the same index into Faiss or Chroma. If you’re still deciding whether retrieval is the right approach at all, RAG vs fine-tuning covers that decision.

Two practical notes:

• Store the chunk text and source next to the vector. The LLM needs the text, and users need citations. A vector without its payload is useless at answer time.
• Retrieval quality limits answer quality. If the right chunk isn’t in the top $k$, the LLM can’t use it. Measure hit rate or recall on real questions before tuning prompts.

Vector database vs vector library vs vector-capable relational database

“Vector database” is used loosely. There are three different kinds of tool:

Search engines such as OpenSearch sit between the last two columns: a document store with full-text search, aggregations and k-NN vector fields, which suits hybrid (keyword plus vector) search. My bias as a database person: if your vectors describe rows you already keep in PostgreSQL, start with pgvector and measure before adding another system. If vector search is the product, or the collection is huge, a dedicated store earns its keep. For the broader “which database on AWS” question, see DynamoDB vs RDS.

Vector database options on AWS

These are the main AWS options, as documented on October 22, 2026. Features move quickly here, so check the linked docs before you design around a limit.

Notes from the AWS documentation:

• OpenSearch Service memory: HNSW graphs live in native memory. OpenSearch Service gives half of an instance’s RAM to the Java heap (up to 32 GiB), and by default k-NN uses up to half of the remaining RAM, so an instance with 32 GiB of RAM can hold about 8 GiB of graphs. Size data nodes for this.
• S3 Vectors positioning: AWS describes S3 Vectors as cost-optimized vector storage with sub-second latency for infrequent queries and as low as 100 milliseconds for more frequent ones. The Bedrock guide calls it best suited for infrequent query workloads. An index’s name, dimension, distance metric and non-filterable metadata keys can’t be changed after creation; changing them means a new index.
• Bedrock Knowledge Bases vector stores: the Bedrock user guide lists Amazon OpenSearch Serverless, Amazon OpenSearch Service Managed Clusters, Amazon S3 Vectors, Amazon Aurora PostgreSQL, Amazon Neptune Analytics graphs (for GraphRAG), Pinecone and Redis Enterprise Cloud. Bedrock can quick-create a vector store for some of these. Only OpenSearch Serverless and OpenSearch Managed Clusters support binary vector embeddings; S3 Vectors supports floating-point embeddings only. For Aurora, Bedrock needs pgvector 0.5.0 or later and an HNSW index on the embedding column.

pgvector in SQL (not executed)

This is how the same ideas look in PostgreSQL with pgvector. It is not executed for this article; the syntax follows the pgvector README and the Aurora knowledge base guide. The table and column names are examples.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE doc_chunks (
    id         bigserial PRIMARY KEY,
    tenant_id  int       NOT NULL,
    chunk_text text      NOT NULL,
    embedding  vector(1024)          -- must match the embedding model's dimension
);

-- Approximate index for cosine distance (HNSW); without it, queries are exact scans
CREATE INDEX ON doc_chunks USING hnsw (embedding vector_cosine_ops);

-- Keep scanning the HNSW index until enough rows pass the filter (pgvector 0.8.0+)
SET hnsw.iterative_scan = relaxed_order;

-- Top 5 chunks for one tenant; <=> is cosine distance, so ascending order = most similar
SELECT id, chunk_text, 1 - (embedding <=> '[0.012, -0.034, ...]') AS cosine_similarity
FROM doc_chunks
WHERE tenant_id = 7
ORDER BY embedding <=> '[0.012, -0.034, ...]'
LIMIT 5;

Replace the shortened vector literal with a full 1,024-number query embedding from the same model you used for the documents.

Amazon S3 Vectors with the AWS CLI (not executed)

These commands are not executed against any AWS account. They were syntax-checked with AWS CLI 2.37.9 and all credentials removed, so the CLI accepted every option and then stopped with “Unable to locate credentials”. Bucket and index names are examples; the query vector is shortened.

# Create a vector index: 1,024 dimensions, cosine distance; keep the chunk text non-filterable
aws s3vectors create-index \
  --vector-bucket-name docs-vectors \
  --index-name kb-chunks \
  --data-type float32 \
  --dimension 1024 \
  --distance-metric cosine \
  --metadata-configuration '{"nonFilterableMetadataKeys":["chunk_text"]}' \
  --region us-east-1

# Top 5 nearest neighbors for one tenant, with distances and metadata
aws s3vectors query-vectors \
  --vector-bucket-name docs-vectors \
  --index-name kb-chunks \
  --query-vector '{"float32":[0.012,-0.034,0.051]}' \
  --top-k 5 \
  --filter '{"tenant":{"$eq":"acme"}}' \
  --return-distance \
  --return-metadata \
  --region us-east-1

Per the QueryVectors API reference, a query that uses a metadata filter or returns metadata needs both the s3vectors:QueryVectors and s3vectors:GetVectors permissions; without the second, the request fails with 403 Forbidden.

Which AWS option fits?

1. Your data already lives in PostgreSQL, and you need joins, transactions or SQL filters: Aurora PostgreSQL or RDS for PostgreSQL with pgvector.
2. You need hybrid keyword plus vector search, aggregations or low-latency search at high query rates: OpenSearch Service, as managed domains or a Serverless vector search collection.
3. Large collections that are queried infrequently, where storage cost matters more than latency: S3 Vectors.
4. You want AWS to run ingestion, chunking, embedding and retrieval for a RAG app: Bedrock Knowledge Bases on top of one of its supported stores.

Whichever you choose, run the demo’s method on your own data: build a small exact baseline, then measure recall@k and latency for the index settings you plan to use, with your real filters on.

Common mistakes with vector databases

1. Using a different embedding model (or version) for queries than for documents. Distances become meaningless, usually with no error. Store the model name and version with the index.
2. Mismatched distance metric. Indexing for Euclidean distance when the model was trained for cosine, or using the dot product on vectors that aren’t normalized, quietly changes the ranking.
3. Never measuring recall. An ANN index always returns $k$ results, even when they’re the wrong ones. Keep a flat index or an exact query as ground truth and track recall@k after every settings change.
4. Post-filtering selective filters. As the demo showed, top 10 then filter can leave one result. Filter during the search, enable iterative scans in pgvector, or over-fetch and verify.
5. Adding an ANN index too early. At tens of thousands of vectors, an exact scan may be fast enough and has perfect recall. Add HNSW or IVF when measurements say you need it.
6. Ignoring HNSW memory and build time. The graph adds memory on top of the raw vectors (784 bytes against 512 per vector in the demo) and took about ten times longer to build than IVF. Plan node sizes and re-index windows.
7. Training IVF or PQ on unrepresentative data. Centroids learned from last year’s documents can fit today’s data poorly. Retrain when the data distribution shifts.
8. Changing dimensions without re-indexing. An index (and an S3 vector index, explicitly) is fixed to one dimension. Switching models or dimensions means re-embedding everything into a new index.
9. Treating the vector database as the only search you need. Exact identifiers such as order numbers, error codes and SKUs still need keyword search; hybrid search combines both.

Frequently asked questions

What is a vector database in simple terms?

A vector database stores embeddings, which are lists of numbers that represent the meaning of text, images or other data, and finds the stored items most similar to a query. Instead of exact matches, it returns the nearest neighbors by a distance measure such as cosine similarity.

How is a vector database different from a relational database?

A relational database finds rows whose values match a condition, using indexes that sort values. A vector database finds rows whose vectors are closest to a query vector, using similarity indexes such as HNSW or IVF. Many systems now do both, for example PostgreSQL with the pgvector extension.

Do I need a vector database for RAG?

Not always. For a few thousand chunks, an in-memory NumPy array or a library such as Faiss is enough. You need a vector database when you want persistence, updates, metadata filtering, access control and scale. If your data is already in PostgreSQL, pgvector may be the simplest option.

What is the difference between cosine similarity, dot product and Euclidean distance?

Cosine similarity compares only the direction of two vectors. The dot product also grows with their lengths. Euclidean distance is the straight-line distance between them. For vectors normalized to length 1, all three give the same ranking, so use the metric your embedding model recommends and normalize when needed.

What is the difference between HNSW and IVF?

HNSW searches a layered graph of neighbor links and usually gives the best recall for the speed, at the cost of more memory and slower builds. IVF clusters vectors with k-means and scans only the nearest clusters, so it builds faster and uses less memory, but it typically needs more tuning to reach the same recall.

Which vector database should I use on AWS?

Use Aurora PostgreSQL or RDS for PostgreSQL with pgvector when vectors belong with relational data, OpenSearch Service for hybrid keyword and vector search at low latency, and S3 Vectors for large, infrequently queried collections at low storage cost. Bedrock Knowledge Bases can manage a RAG pipeline on top of these stores.

Where to go next

Start with What are embeddings? if the vectors themselves still feel abstract, then build a working retrieval pipeline with the RAG tutorial in Python. More GenAI explainers are collected in the Generative AI tutorials hub.

Sources

All checked on October 22, 2026.

• Amazon Bedrock User Guide: Prerequisites for using a vector store you created for a knowledge base (supported vector stores, quick-create, binary vector support, S3 Vectors float-only and metadata limits, Aurora table, HNSW index and iterative-scan recommendation)
• Amazon Aurora User Guide: Using Aurora PostgreSQL as a Knowledge Base for Amazon Bedrock (pgvector 0.5.0 or higher with HNSW)
• AWS Prescriptive Guidance: Retrievers for RAG workflows (pgvector on Aurora PostgreSQL and RDS for PostgreSQL; exact and approximate search; L2, inner product and cosine; IVFFlat and HNSW)
• Amazon S3 User Guide: Working with S3 Vectors and vector buckets, Vector indexes, Limitations and restrictions, Metadata filtering
• Amazon S3 API Reference: QueryVectors (approximate nearest neighbor query; permissions for filters and metadata)
• Amazon S3 FAQs (S3 Vectors getting started, querying, and metadata pre-filtering on ENHANCED indexes)
• Amazon OpenSearch Service Developer Guide: k-Nearest Neighbor (k-NN) search (knn_vector, maximum k, post_filter behavior, k-NN graph memory), Working with vector search collections (Serverless metrics, dimensions, filtering), Managing capacity limits for OpenSearch Serverless (OCUs), What is Amazon OpenSearch Service? (pricing model)
• Amazon Aurora User Guide: DB instance billing for Aurora (pricing components)
• Pricing pages (check live figures; this article states none): Amazon OpenSearch Service, Amazon S3 (Vectors tab), Amazon Aurora, Amazon RDS for PostgreSQL, Amazon Bedrock
• OpenSearch documentation (open source): Methods and engines (HNSW and IVF methods, Faiss and Lucene engines, space types, PQ encoder, filtering behavior, defaults)
• pgvector README (operators, exact search by default, HNSW and IVFFlat parameters and defaults, filtering and iterative index scans)
• Faiss and the Faiss wiki (IndexFlatIP, IndexIVFFlat, IndexHNSWFlat, IndexIVFPQ, IndexRefineFlat)
• Malkov and Yashunin: Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs (HNSW)
• Jégou, Douze and Schmid: Product quantization for nearest neighbor search, IEEE TPAMI 33(1), 2011 (PQ)

Learn vector search and RAG with guided practice. For live training on embeddings, vector databases, retrieval and LLM applications, see the RAG and LLM training page. If you’re in the United States, see online AWS and AI/ML training in the USA.