[kerem-acer]

Closes #80.

MigrateOptions.SyncIndexes, off by default, makes the server's indexes and search indexes exactly what the code declares:

await migrator.MigrateAllAsync(new MigrateOptions { SyncIndexes = true });

What it does

Once every vault's migrations have run, the migrator syncs each collection a migrated vault declares against what every registered vault declares of it. In addition to creating what's missing and changing in place what MongoDB can, as today:

• Plain on the server, declared unique: made unique in place, with collMod prepareUnique and then unique. If documents share its keys, the second step fails with CannotConvertIndexToUnique. MongoFlow then sets prepareUnique back to false, so the index is left exactly as it was. (prepareUnique alone already rejects new duplicates.)
• Differs in what MongoDB can't change in place: keys, unique to plain (collMod refuses unique: false), sparse, partial filter, collation, or an expiry it shouldn't have. It's dropped and created as declared.
• Declared keys and options under another name: dropped first, since MongoDB keeps only one of two equivalent indexes, then created under its declared name.
• No vault declares it: dropped last, after every other change across every collection. A failure earlier, such as a unique conversion over duplicates, leaves it in place. _id_ and Queryable Encryption's __safeContent__ index are never dropped.
• Search index of the other type: dropped, waited for until it's no longer listed (Atlas deletes in the background), and created anew. One no vault declares is dropped.

Planning before changing

Every collection is planned before any is changed. The sync fails, changing nothing, when:

• two vaults declare an index or search index of one name differently; the error names each vault and its definition;
• an index it would drop is the only one on a sharded collection's shard key;
• a collection declares search indexes on a server without Search.

Scope

• Synced collections: only collections a migrated vault maps. An orphan collection or the migration history collection keeps its indexes.
• Shared collections: a collection several vaults map is synced against all their declarations. MigrateAsync<TVault> syncs TVault's collections against every registered vault's declarations, so it never drops another vault's indexes, and creates them where they're missing.
• Views: skipped, since they have no indexes of their own. A collection that doesn't exist and declares nothing isn't created.

Logs and spans

• Logs: each drop is logged at Information as IndexDropped (27_002_013 under MongoFlow.Indexes, 27_008_009 under MongoFlow.Search), with the index as the server listed it, so it can be created again by hand. A recreate also logs IndexRecreated, and a unique conversion IndexUpdated with { "unique" : true }.
• Spans: MongoFlow.Index and MongoFlow.SearchIndex spans are tagged recreated or dropped.

Server behavior I checked first

• The issue's shard-key premise is only half true: MongoDB refuses to drop a ranged shard key's only index (CannotDropShardKeyIndex), but MongoDB 8.2 drops a hashed shard key's index without complaint. So the plan-time guard is what protects core's 34 AgentId_hashed… indexes on prod.
• Detecting shard keys: done with $listClusterCatalog. It works for a plain readWrite user, while config.collections is Unauthorized for that user. It needs MongoDB 8.0.10+. On an older server the sync fails, saying so, rather than guessing.
• Same keys, different options: indexes on the same keys coexist when they differ in unique or sparse. So a declared index can be created before the undeclared one it replaces is dropped. Only a rename, where both have the same identity, has to drop first.
• Search indexes after a drop: dropping and immediately recreating one works on both Atlas Local and community-search. The wait until it's gone is for real Atlas.

Tests

All on real servers, in CI:

• mongo 8.2, IndexTests.Sync.cs:
  • unique declared plain, recreated plain;
  • plain declared unique, converted in place, with and without another in-place change; over duplicates it fails and every index stays, including an undeclared one;
  • undeclared indexes dropped last, while another collection's and the history's are kept;
  • an index under another name recreated under its own;
  • a second run changes nothing;
  • two vaults keep each other's indexes, and their conflicting declarations fail before anything changes;
  • MigrateAsync of one vault sets up both vaults' indexes;
  • views left alone, and a collection that declares nothing not created;
  • a server without Search fails a declared search index before anything changes;
  • a drop failed by the failCommand failpoint throws, and its span is marked failed;
  • spans.
• Atlas Local and community-search 1.70.1 (SearchTests.Migrations.cs, every test on both servers): an undeclared search index dropped; one of the other type recreated, after which vector search works; a second synced run changes nothing; two vaults declaring one differently fail.
• Queryable Encryption: an encrypted collection keeps __safeContent___1. Without the exclusion, the server lets the sync drop it.
• Sharded cluster (new ShardedClusterFixture, a config server that is also a shard plus mongos):
  • an undeclared index on a hashed shard key fails the sync, naming it, and nothing changes;
  • with the shard key's index declared, the undeclared indexes are dropped.
• Unit: PlanSync for each outcome; Undeclared; Backs; the search drop waiting until the index is gone; $listClusterCatalog missing; search listing errors; a failed search-index drop.
• Results:
  • integration lane: 487/487 on net10.0 and net11.0;
  • unit lane: passes on both, except the two QueryAsync_JoinOn* tests, which fail only on my machine because of a local driver fork;
  • coverage: the new code is covered, except the compiler-generated rethrows already listed as known gaps.

Not in this PR

• No parent span: the sync's spans sit beside the vaults' MongoFlow.Migrate spans, not under a parent span of their own.
• No changelog entry: the changelog is updated by the release PR.

🤖 Generated with Claude Code