A search system can accept vectors from only one embedding model and still contain two incompatible meanings. That happens when a team changes the model used for new documents but leaves older vectors in place. Every vector has the right length, every write succeeds, and retrieval quietly becomes unreliable.
An embedding-model migration is therefore not a configuration edit. It is a data migration with a version boundary, a new index, a full re-embedding pass and a measured cutover.
Start with the representation, not the model name
An embedding model turns text into a numeric vector. Nearby vectors are meant to represent semantically related inputs, but the coordinates are meaningful only inside the space learned by that model and configuration.
Two models can both return 1,536 numbers without assigning the same meaning to coordinate 417. Even the same model family can produce a different shape when a dimensions parameter changes.
OpenAI’s current embeddings guide states that text-embedding-3-small returns 1,536 dimensions by default and text-embedding-3-large returns 3,072. It also allows shorter vectors through the dimensions parameter. That flexibility is useful, but it means the model name alone is not a complete version identifier.
Record at least the provider, exact model identifier, requested dimensions, input-normalization rules and chunking configuration. Treat that bundle as the embedding version.
Why mixing old and new vectors fails
A vector database compares query vectors with stored vectors using a distance or similarity metric. The comparison assumes both sides were produced in the same coordinate system.
If a new query uses Model B while half the corpus still contains Model A vectors, the database will still calculate a score when the dimensions match. The score is mathematically valid but semantically meaningless across the two spaces.
A dimension mismatch is easier because many systems reject it at the boundary. Pinecone’s indexing overview says an external embedding workflow creates an index that matches the characteristics of the model, then uses that same external model to turn queries into vectors.
The dangerous case is silent compatibility: equal dimensions, accepted writes, and incomparable representations.
Create a parallel index
The safest migration pattern is blue-green indexing. Keep the production index unchanged and build a second index for the new embedding version.
Pinecone’s index-creation guide distinguishes indexes with integrated embedding from indexes that store externally generated vectors. It also warns that document-schema migration is not supported: changing a schema requires a new index.
Even where a vendor permits some index settings to change, a parallel index remains the clearer operational boundary. It prevents accidental mixing and gives the team a rollback target.
Name the new index by purpose and embedding version rather than “new.” Store the version in deployment configuration and in record metadata. A human looking at either system should be able to tell which encoder produced a vector.
Keep source text outside the vector
Re-embedding requires the original text or a canonical representation of it. A vector is not reversible source storage.
For each chunk, retain:
- a stable document identifier;
- a stable chunk identifier;
- the source text or a reliable pointer to it;
- source revision or content hash;
- chunking policy version;
- embedding version;
- access-control metadata;
- deletion or retention state.
Stable identifiers let the migration overwrite retries safely and compare the same chunk across versions. They also make it possible to prove that the new index contains the same logical corpus, not merely the same record count.
This is where RAG data provenance becomes practical. The retrieval layer needs to know which document version produced the chunk, not just where the current document lives.
Re-embed in bounded batches
A migration job should read canonical chunks, call the new embedding model and write to the new index in repeatable batches. The unit of progress is a stable chunk ID, not an offset in a changing source table.
Pinecone’s upsert documentation says an upsert replaces an existing record with the same ID. That behavior supports idempotent retries when the migration always regenerates the full record.
Batch size should respect provider limits and make failures easy to retry. Save a checkpoint only after the destination confirms the write. Record rejected inputs separately instead of skipping them invisibly.
Do not continuously copy the live corpus without a change strategy. Choose one of two patterns:
- Take a snapshot, rebuild it, then replay changes made after the snapshot.
- Dual-write new and changed chunks to both indexes while the backfill runs.
The second pattern shortens the final cutover, but it adds temporary application complexity. The first is simpler when updates are infrequent and a brief write pause is acceptable.
Migrate deletions and permissions too
A new index can resurrect data that the old index correctly stopped returning.
If a source document was deleted during the backfill, the migration has to delete its chunks from the destination. If a user lost access, the new records must carry the current permission metadata before the index serves traffic.
Build a reconciliation pass from the source of truth. Compare active chunk IDs, content hashes and permission versions. A high record count is not evidence that the destination is safe.
For document extraction pipelines, uncertain or failed source parsing also needs an explicit state. The principles in document extraction confidence apply before embedding: low-quality text should not become a confidently indexed fact.
Test retrieval before cutover
Evaluate the new index with a fixed set of representative queries and judged relevant results. Include straightforward lookups, ambiguous language, rare terms, recent documents and permission-filtered cases.
Measure retrieval independently from the answer-generating model. Useful checks include recall at a chosen depth, the rank of the first relevant result, empty-result rate, latency and filter correctness.
Then shadow a sample of production queries. Send them to both indexes, return only the old result to the user, and compare the ranked sets offline. Large differences deserve inspection even if an aggregate score improves.
Do not require identical rankings. A new model is supposed to change retrieval. The goal is to understand whether the changes are useful, safe and explainable for the application.
Cut over with a reversible switch
Put the active index and embedding version behind one deployment setting. A request must use the query encoder that belongs to the selected index.
At cutover:
- Stop or complete the final change replay.
- Reconcile IDs, deletions and permissions.
- Run the evaluation suite against the final destination.
- Change the index and query encoder together.
- Monitor retrieval errors, empty results, latency and downstream answer quality.
- Keep the old index read-only for a defined rollback window.
Never switch the query encoder first and the index later. That creates the exact mixed-space failure the migration is meant to avoid.
After the rollback window, delete the old index only under the organization’s retention and recovery policy. Preserve the migration manifest and evaluation results so the next model change starts with evidence rather than memory.
Reporting & Sources
This field note uses the current OpenAI embeddings guide for model dimensions and dimension reduction, and Pinecone’s current documentation for index creation, indexing workflows and upserts. Product behavior described here is limited to those published interfaces; other vector databases may expose different migration operations.
