Google details zero-downtime embedding model upgrades for AlloyDB AI

Google has published a migration path for upgrading vector embedding models in AlloyDB AI without taking production systems offline, allowing teams running semantic search and other vector-based workloads to switch models while user queries continue to be served.

The stakes are practical rather than speculative. When a new embedding model ships, existing data usually has to be re-embedded, indexes rebuilt and query logic switched, because embeddings from different models cannot be compared directly. Google's guidance addresses that constraint head-on for live databases, using in-database model access, automated embedding generation, PostgreSQL functions and concurrent index creation.

What does the migration actually involve?

The process begins by adding a new vector column sized to match the output dimensions of the replacement model. In Google's worked example, a production table of product reviews moves from text-embedding-005, which produces 768-dimensional vectors, to gemini-embedding-001, which produces 3,072-dimensional vectors — a fourfold increase in dimensionality.

Auto-embeddings are then enabled on the new column so the database backfills vectors in the background. AlloyDB AI can generate embeddings at up to 3,000 per second and tracks changes to source data as rows are inserted or updated. The new column fills without blocking the writes that keep the application alive.

What are the two refresh modes?

The system supports two ways to keep vectors current, and the choice shapes latency across the entire migration.

• Transactional mode. Embeddings update as soon as source data changes. Vectors stay aligned with underlying records, but write latency rises because inference runs inside each write transaction.
• Manual mode. The database marks changed rows and lets operators refresh stale embeddings later in scheduled batches. This reduces pressure on write performance. It is the mode Google used in its migration example.

Google also quantified the operational trade-offs during backfill. Lowering the maximum number of parallel requests reduces the time concurrent writers may wait for a micro-batch to commit, but lengthens the overall backfill window. The benchmark numbers are concrete: on a 10,000-row table, reducing max_parallel_requests from 400 to 20 cut the longest concurrent writer wait from 5.1 seconds to 0.39 seconds, while increasing total backfill time from 5.2 seconds to 7.6 seconds.

Once the initial backfill completes, a refresh procedure catches up changed rows. A progress view shows job state, rows processed and estimated time remaining.

How different are the retrieval results?

This is the finding most likely to change how teams plan an upgrade. Before switching traffic, Google recommends checking for missing embeddings and validating the new vectors against fresh model invocations. Teams should also test application behaviour, because retrieval patterns can change sharply when moving to a different model.

In Google's internal tests, the top-10 nearest-neighbour results from text-embedding-005 and gemini-embedding-001 overlapped by only 31%. That figure means teams may need to review assumptions built into ranking, filtering or evaluation pipelines before routing live production traffic to the new model. A swap is not a drop-in improvement; it is a behavioural change in what the search returns.

How fast can the new index be built?

The migration includes building a new ScaNN vector index (Google's vector search index structure) on the replacement column, done concurrently to avoid blocking writes.

Google said its ScaNN index builds were up to 10 times faster than HNSW (Hierarchical Navigable Small World, the dominant open-source approximate nearest-neighbour index type) on open-source PostgreSQL. In tests on 100,000 rows, a 3,072-dimensional ScaNN index was built in about 50 seconds, while an HNSW index using halfvec took between seven and 13 minutes.

What are the cutover options?

For the final switch, Google outlined two main patterns.

• Atomic column rename plus query function. Applications need no code changes when the live embedding column is swapped. The pattern supports blue-green deployment, but not per-query canary testing.
• PostgreSQL session variables and a set-returning function. This allows canary releases and traffic splitting between old and new models, letting teams direct a percentage of sessions to the new model and revert quickly by changing a session parameter.

There are limits. The atomic rename pattern does not work with transactional auto-embeddings because managed triggers refer to fixed column names. Transactional mode also brings a marked increase in write latency while both models are active.

Google's figures for single-row update operations make the cost explicit. Median write latency ran about 37.5 milliseconds with text-embedding-005 alone, rose to roughly 191.0 milliseconds during migration when both models were active, then settled at about 172.0 milliseconds after cutover to gemini-embedding-001. The steady-state cost of the larger model alone is more than four times the original.

What happens to storage?

Storage use rises materially during migration because old and new vectors coexist. In Google's tests, text-embedding-005 used about 4.5KB per row, while gemini-embedding-001 used about 13.3KB per row.

Scaled out, a table with 10 million rows would need about 133GB for the new vector column and about 6GB for the new ScaNN index, before allowing extra headroom for write-ahead logging and page updates. Planning an upgrade without budgeting roughly triple the per-row footprint risks running out of space mid-migration.

After cutover, teams can remove the old embedding configuration, drop the legacy index and clear the retired vector column in batches before vacuuming and dropping it. In Google's 100,000-row tests, nulling out vectors and vacuuming before dropping the column reclaimed 381MB of TOAST space (PostgreSQL's storage mechanism for oversized values) for reuse in later model upgrades.

For teams betting their retrieval stack on Gemini-era embeddings, Google's guidance amounts to a working playbook: quantified latency costs, a 31% overlap warning on retrieval drift, and a cleanup procedure designed to be repeatable when the next embedding model arrives.