fix(plugin-mongodb): rename and remove MongoDB fields from the Structure tab, carrying the validator along and refusing what would break an index, a view or a document

[datlechin]

Follows #3145, now merged. Based on main at c21dc512e.

Closes the rename/remove half of #3132.

Root cause

Three defects made MongoDB fields impossible to rename or remove from the Structure tab.

1. Nothing could reach the server. MongoDB reported supportsSchemaEditing = false, the driver had no generateModifyColumnSQL or generateDropColumnSQL, and PluginSchemaOperation had no column cases, so no driver could refuse or answer a column change.
2. Whether a rename or removal is safe depends on server state the per-operation refusal hook cannot see: every index and search index, the validator, every view that reads the collection, whether the collection is capped, and the documents themselves. fix(plugin-mongodb): create collections from New Table with their fields as a validator, read the fields back, and write values in their declared types #3145 also puts every field of a New Table collection in $jsonSchema.properties, so renaming only on the documents leaves the validator requiring the old name and every renamed document invalid.
3. Safe Mode took a schema save's kind from the text classifier alone, which tiers updateMany(..., {$unset}) as a plain write, although the Structure tab already marks the removal destructive.

Older defects that affect every engine:

• A Structure save wrote the edits staged when it was pressed, and on success cleared whatever was staged by then. Staging stayed open for the whole save, so an edit made during it was missing from the script and then cleared with the edits that ran. On v0.75.0 that window is a long ALTER. On MongoDB it also covers the document checks, which can run for as long as the query timeout.
• A save that failed once its statements had started reloaded nothing but the catalog. MySQL, MariaDB and Oracle commit each DDL statement as it runs, and MongoDB keeps every document an updateMany changed before it stopped.
• A save told the windows about its rows with a scope-wide refreshData. That reloaded the selected tab of each window in the database whatever table it showed, and asked it to discard its edits, while a background tab on the saved table, and the saving tab's own Data view behind Structure, kept the rows from before the save. A table rebuild and a column reorder sent the same signal for the whole connection.
• updateOne and updateMany in the MongoDB shell dropped a writeConcern option, and mongoc_client_command_simple sends a command with no write concern of its own, so the update went out with the server's default.

Fix

Statements. A rename is db.c.updateMany({"F": {"$exists": true}, "G": {"$exists": false}}, {"$rename": {"F": "G"}}) and a removal is db.c.updateMany({"F": {"$exists": true}}, {"$unset": {"F": ""}}). The filters keep every document atomic and every run idempotent. A document that already holds the new name is skipped, never overwritten, and pressing Save again after a stop finishes only what is left.

Write concern. The connection's write concern is read from the libmongoc client once it connects (w, journal and wtimeoutMS, so an imported connection string counts), and every statement names it: updateMany(..., {"writeConcern": {"w": "majority"}}) and db.runCommand({"collMod": ..., "validator": ..., "writeConcern": ...}). SQL Preview shows it. A concern that asks for no answer (w: 0 without j: true) goes out as w: 1, because measured on 7.0.43 an unacknowledged update is answered n: 0 whatever it changed. The shell's update now puts a statement's writeConcern option on the command. With no write concern configured, nothing is added and the server default applies, as before.

Validator. When the validator is a $jsonSchema that declares the field in properties, required or property dependencies, the save starts with a collMod that renames or removes the field there too, keeping key order and the canonical EJSON values. A field only the validator declares can now be removed.

A validator that uses the field anywhere the rewrite cannot reach refuses the save:

• $expr and query operators;
• nested combinators;
• a document-level enum whose entries are documents naming the old or the new name. A document matches such an entry only with exactly its names in its order, and $rename moves the field to the end, so the rewrite cannot carry it along;
• anything that hands the whole document, $$ROOT or $$CURRENT, to something that reads its names (see the shared rule below). $where and $function already did;
• a patternProperties pattern that matches the old or the new name;
• an additionalProperties rule (anything but true or {}) that applies to the old or the new name because properties does not declare it.

One whole-document rule for validators and views (MongoWholeDocumentReads). A bare $$ROOT or $$CURRENT reads every field, except where it stays the document: $replaceRoot and $replaceWith of it, directly or through $mergeObjects, a $setField or $unsetField with a literal field name, or a branch of $cond, $switch, $ifNull or $let. A $getField with a literal name reads that one field. Anywhere else, placed under a name, in an array, in a variable or handed to $objectToArray or a comparison, it reads every field. A validator's $expr and a view's pipeline both answer through it, and a test holds them to the same answer for eleven expressions. This narrows one round-3 answer on purpose: a validator's $getField: {field: "a", input: "$$ROOT"} now reads a alone, as it does in a view.

Refusals, before anything is written.

• Per operation (sync): _id as source or target; empty, NUL, $-prefixed, dotted and __proto__ names; any change to type, nullability, default or comment. These run ahead of fix(plugin-mongodb): create collections from New Table with their fields as a validator, read the fields back, and write values in their declared types #3145's own refusal, which still answers everything else.
• Per save, from the catalog (SQL Preview and Save), through reviewSchemaChange:
  • a field named twice in one save;
  • a view, time series, system. or missing collection;
  • a rename to a longer name, in UTF-8 bytes, on a capped collection;
  • an index using either name (the message gives the dropIndex command);
  • an Atlas Search or Vector Search index whose definition mentions either name, read with $listSearchIndexes. The listing's failure is read by code. 6047401 (measured on 6.0.28 and 7.0.43 community) and 31082 SearchNotEnabled (measured on 8.2.12 community with no mongot) mean the server has no search: both refuse $search the same way. 40324, the unknown-stage code, no longer means none: an Atlas cluster older than the stage runs $search over indexes it cannot list. The driver then runs [{$search: {exists: {path: "_id"}}}, {$limit: 1}]. A server that answers it with 40324, 6047401 or 31082 has no search (measured: 5.0.33 and 6.0.5 community answer 40324 to both); any answer, even an empty one, stops the save with "This server runs Atlas Search but cannot list its search indexes, so one that uses the field cannot be ruled out." Any other failure stops it too;
  • an encrypted field;
  • a view reading the old name, found through viewOn chains and $lookup/$graphLookup/$unionWith at any depth, by any string or non-operator key naming it, or by the whole-document rule.
    The review keeps the collection's listCollections entry as its new basis.
• On Save only, through schemaChangeRefusalBeforeWriting, inside the lease that then writes:
  • the collection's entry is no longer the review's basis: validator, level, action, or a drop and recreate since the save was composed;
  • a document holding both names;
  • a validator simulation replayed step by step in the order the statements run;
  • when the save rewrites the validator, a pass over every document holding any changed name, checked against the rewritten validator. Under moderate, only a document the old validator accepted counts;
  • then the catalog is read once more, as the last thing before the first write, and any change to the entry, or an index, search index or view that now refuses, stops the save.

After writing, through schemaChangeShortfallAfterWriting, which now receives the review:

• the documents still holding each old name are counted, and any left fail the save with "The save did not finish: 3 documents still hold tmp ... Save again to finish.";
• when the save rewrote the validator, one more pass looks for a document holding a changed name that the rewritten validator rejects, with the validator, level and action taken from the review's basis. This is the target-only race: a document another client wrote with only the new name after the checks and before the collMod passed the old validator, which did not read that name, and fails the new one. The save fails with "The save did not finish: the updated validator of tr rejects the document with _id "late", most likely written by another client while the save ran. Fix that document, then save again." Under moderate, only a document the old validator accepts as it now stands counts, as before writing.

A read after writing that fails reports that the save ran and to save again.

Host.

• A save holds its edits from the press until it ends (StructureChangeManager.holdForSave()), and every staging path refuses while it does. The Structure tab shows Saving Changes….
• schemaChangeStatements returns a SchemaChangeScript (statements, operations, review). executeSchemaChanges asks the before-writing question just before the first statement and the after-writing question just after the last.
• The three save questions and the new tableDefinitionDidChange(table:schema:) are DatabaseDriver requirements with defaults, bridged in PluginDriverAdapter+SchemaChangeChecks.swift. The manager calls them on DatabaseDriver with no cast to the adapter, so any driver or test double answers them. withSchemaComposer keeps its existing cast, because SchemaStatementGenerator takes the plugin driver itself.
• Table-targeted invalidation. Once a save has written, finished or stopped partway, DatabaseManager.reportTableDefinitionChange(table:in:) tells the session's own driver tableDefinitionDidChange and sends DatabaseObjectChange(kind: .structure) for the saved table. A table rebuild and a column reorder send it too. The scope-wide refreshData is gone from all three. Each window, for every tab on that table:
  • forgets the table's cached schema columns, which a column-scoped reload builds its select list from;
  • drops a background tab's rows so it reloads when shown, unless they hold edits, are pinned or are loading;
  • reloads the selected tab's rows now, behind Structure too, unless they hold edits; a selected Data tab with edits asks first, as before;
  • refetches a structure no one has staged edits against, now for the one on screen and on next mount for the rest (markStructureStale()). The saving tab's structure holds staged edits for as long as its save runs, and keeps them when the save stops partway, so it is left alone.
    Tabs on other tables are untouched.
• Session driver caches. MongoDB's tableDefinitionDidChange drops the collection's inferred column kinds, field path kinds, identity kind and declared schema, in every database the driver keyed them under. Measured before: after a rename of a declared created on another connection, page 2 of the session's browse still listed created beside made, from the declared schema a later page reuses.
• A statement's isDestructive joins the text tier in the Safe Mode kind, so a field removal and a SQL column type change confirm like DROP COLUMN.
• Compare & Sync generates no structure script for an engine whose columns are sampled (new curated columnsAreSampled, true for MongoDB).

Edit surface. The Structure tab offers Name and row removal on a collection. Type, Nullable, adding a field and index editing stay read-only. An older installed plugin keeps its own supportsSchemaEditing = false.

PluginKit (additive, pending kit 33). Adds PluginSchemaOperation.modifyColumn and .dropColumn on the non-frozen enum, the PluginSchemaChangeReview struct (refusal, leadingStatements, basis), and four requirements with defaults: reviewSchemaChange(table:schema:operations:), schemaChangeRefusalBeforeWriting(table:schema:operations:review:), schemaChangeShortfallAfterWriting(table:schema:operations:review:) and tableDefinitionDidChange(table:schema:). None of these has shipped, so the after-writing requirement gaining review: changes nothing a built plugin references. scripts/check-pluginkit-abi.sh b8b2fc7b4 shows additions only. v0.75.0 ships kit 32, so there is no bump.

Overlap with open branches

• fix(tabs): reload only the tabs showing a changed table, never interrupt one holding edits, and read SQLite result columns after the first step #3148 (fix/refresh-written-table-tabs) makes the same move for every write: DatabaseObjectChange with .rows and .structure, refreshStructure(ofTabs:), markStructureStale(), forgetSchemaColumns, SchemaColumnStore.remove, plus freshness stamps and an origin tab. This branch adds the .structure case and those four pieces under the same names and semantics, without the stamps. On merge, take that branch's versions and keep reportTableDefinitionChange as the single sender, now feeding its .structure path. tableDefinitionDidChange stays: that branch does not reach the driver's caches.
• fix(plugin-mongodb): send the connection's write concern with every shell write and name the documents a failed write already changed #3151 (fix/mongodb-shell-write-concern) sends the connection's write concern with every shell write, updateMany included, and honours a statement's own. It does not add one to db.runCommand, so the collMod needs its explicit member either way. Both branches read writeConcernError in MongoWriteFailure; on merge keep fix(plugin-mongodb): send the connection's write concern with every shell write and name the documents a failed write already changed #3151's read(fromReply:) and this branch's concernFailure(fromReply:) beside it. On merge, take that branch's MongoScriptCommandBuilder.update and host write path over this branch's one-line pass-through; the statements' explicit concern is then the same one the connection would add. Both read the concern from libmongoc; folding MongoWriteConcern into its writeConcernJson(client:) is a follow-up.
• fix(plugin-mongodb): list views as views, keep index key order and options, and add db.createView to the shell #3150 (fix/mongodb-views-and-index-order) makes fetchTables report views as VIEW and system. collections as SYSTEM TABLE. Nothing here duplicates it.

Verified

Round 5. Codex found one P1 and four P2s:

• P1, bump the kit for the new API: not a defect here. CLAUDE.md bumps currentPluginKitVersion at most once per release cycle and every later change reuses the pending number. v0.75.0 shipped 32, main already holds 33 for this cycle, and no build has shipped with 33, so no host exists that accepts 33 and lacks these symbols.
• P2, the save hold did not reach the inspector. The inspector offered a row as editable while a save held the edits, and a commit then was dropped by the hold. The row is read-only while held (heldSaveMakesTheRowReadOnly, which fails without the change).
• P2, a collMod's writeConcernError passed. The shell's db.runCommand now throws on a reply carrying one, which is what mongosh's driver does for any command, so the save stops with the server's reason (commandConcernFailure). This replaces the round-4 note that it would not.
• P2, a longer name could take a document past 16 MB. Before writing, a rename to a longer name reads $bsonSize for a document the added bytes would take past MongoDB's limit and refuses with its _id, since updateMany would stop at it partway on every retry (oversizePassCountsGrowth). A server before 4.4 has no $bsonSize; there the check steps aside and the docs say so.
• P2, a dependency made during the save. After writing, the save reads the indexes, search indexes and views again and reports one another client made on either name while it ran, in words that say the save ran rather than asking to save again. The three checks now share one MongoFieldDependent, so before and after read the same rules (dependentAfterTheSave).

Round 5 checks, on main at c21dc512e: the 20 test files this branch changes plus StructureGridDelegateInspectorTests, MongoWriteFailureTests and StringCatalogIntegrityTests, 256 of 256; the app, MongoDBDriver and AllPlugins build; lint 0 violations on the changed Swift files; docs pass; four new plugin strings added through localization.py plugins --add. The 16 MB and dependency paths were not run live this round.

Round 4 and earlier:

This pass, on the tree amended into a5b3fc85b. One plugin-only line changed after the test runs (the search probe's success path returns false directly), and build MongoDBDriver compiled it again:

• verify.sh test, 43 suites, 666 of 666: the 8 suites this pass touches (MongoFieldChangeAssessmentTests, MongoFieldDataProbeTests, MongoFieldReferencesTests, MongoSearchIndexTests, MongoFieldChangeTests, MongoScriptCommandBuilderTests, DatabaseManagerSchemaChangeRoutingTests, CatalogChangeWindowTests) and 35 neighbours (SQLSchemaProviderTests, whose mock gained the hooks; EvictionTests, MainContentCoordinatorRefreshTests, MainContentCoordinatorLazyLoadTests, DataRefreshScopeTests, SchemaColumnStoreTests, SchemaColumnStoreCancellationTests, ColumnFetchScopeTests, the Structure, Safe Mode, registry, Compare and MongoDB DDL, schema, generator, query builder and write suites, and three materialized view suites).
• New cases: viewReadsEveryFieldThroughTheWholeDocument, viewPassingTheDocumentOn, validatorAndViewAgree, documentEnumEntriesNameFields, listingStageUnknown, serverWithoutSearch, statementCarriesWriteConcern, unacknowledgedConcernIsRaised, validatorStatementCarriesWriteConcern, collectionCacheKeys, catalogChangedSinceComposed, violationAfterWriting, updateWriteConcern, schemaChangeReportsItsTable, sessionDriverForgetsTheTableDefinition, checkAfterWritingGetsTheComposedReview, databaseDriverAnswersThroughTheProtocol, structureChangeReachesEveryTabOnTheTable, structureChangeKeepsEditedRows, structureChangeSparesStagedEdits, structureChangeForgetsCachedColumns. One round-3 expectation changed with the shared rule: a validator's $getField of a literal qty over $$ROOT no longer reads note.
• Mutation: with the old behaviour put back in 7 files (the scope-wide refresh, the .rows handling for a structure change, the adapter cast, no write concern on the statement or on the shell's update, the view walker without the whole-document rule, 40324 as no search, the cache key match, the enum rule, the after-writing pass replaying the steps), the 8 touched suites went 125 executed, 108 passed, 17 failed, each a new case above or a routing case the refresh change touches. The files were restored from a saved copy and checked by hash, and the suites then passed.
• verify.sh build (TablePro), build MongoDBDriver and plugins (all 40): pass.
• verify.sh lint on the 35 changed Swift files: 0 violations. verify.sh docs: pass. localization.py plugins --add added 3 strings, the key this branch no longer uses was removed, and plugins and verify are ok for both catalogs.
• verify.sh abi b8b2fc7b4 on the clean amended tree: additions only, no line removed.
• Live, through a swiftc harness linking the plugin sources against libmongoc, the committed code (1078c0ab0) against this code, in probe_field-rename-remove_r5, dropped afterwards:
  • Write concern, MongoDB 7.0.43, profiler at level 2 with slowms: -1. A Majority connection: before, the collMod in system.profile and the update in the server log carried no write concern; after, both carried {"w": "majority"}. A 2 connection on this standalone: before, the rename applied; after, the server refused the updateMany with "[2] cannot use 'w' > 1 when a host is not replicated" and no document changed.
  • w: 0 through mongoc_client_command_simple: {"n": 0, "nModified": 0, "ok": 1} for an update that changed 2 documents; w: 0, j: true: n: 2.
  • Target-only race: a second connection inserted {_id: "late", new: 42} between the check and the collMod. Before: "applied", the document then failed the validator, and its next update failed with 121. After: "The save did not finish: the updated validator of tr rejects the document with _id "late" ...". With the document fixed, saving again applied.
  • A view [{$project: {kv: {$objectToArray: "$$ROOT"}}}, {$project: {names: "$kv.k"}}]: before, renaming status applied and the view's names changed from [_id, status, qty] to [_id, qty, state]; after, refused with "View keys reads status." A view [{$replaceWith: {$mergeObjects: [{note: ""}, "$$ROOT"]}}] let qty rename in both.
  • {$jsonSchema: {enum: [{_id: 1, old: "a"}, {_id: 2, old: "b"}]}} under warn: before, the rename applied and both documents then failed the validator; after, the rename and the removal were refused.
  • Session caches: page 2 of the session's browse after another connection renamed created to made: before [_id, made, n, created], after [_id, made, n].
  • Search listing, measured with mongosh and then through the harness: 5.0.33 and 6.0.5 answer 40324 to $listSearchIndexes and to $search; 6.0.28 and 7.0.43 answer 6047401 to both; 8.2.12 answers 31082 to both. A rename applied on each, before and after (6.0.5 after only). The fail-closed answer needs a server that knows $search and not the listing stage, which no community build is; unit tests cover it.
  • The round-4 scenarios re-run on this code: both validator races refused with nothing written and the other client's validator or level kept (the message is now "vr changed while its documents were being checked"), W1 to W3, D1, N1 and N2 as in round 4. On 1,000,000 documents the check before writing took 1,347 to 1,400 ms and the count after writing 203 to 222 ms.
  • mongo:8.0 does not start on this host's kernel (SERVER-121912), so 8.2.12 stood in for it.

Deliberately not fixed here

Structure edit gate on views, time series and system. collections. The gate reads the object kind the sidebar listed, and on this base MongoDB lists every namespace as a table. Locking these before an edit needs a new per-object driver question asked when the Structure tab loads, plus plumbing into StructureEditGate and the grid delegate's own copy of it. fix/mongodb-views-and-index-order lists views as VIEW and system. collections as SYSTEM TABLE, which MongoDB's .table-only edit matrix then locks with no further change. A time series collection stays a table there, so its Name and row removal stay on offer, and the save refuses it at SQL Preview and at Save, before anything is written.

Residual races. MongoDB has no conditional collMod and no lock a client can take:

• A validator set in the last milliseconds before the collMod is replaced by the save's. The docs list this under Limitations.
• A validator set after the collMod is not overwritten. The updateMany statements are validated against it, and one that fails is reported as a failed save.
• A write after the checks after writing is not seen.
• A leftover document holding both names: the not-finished message says to save again, and that second save is refused with the both-names message. The user removes one of the two names, as that message says.
• Under moderate, the check after writing reads the old validator against a document as it now stands. A renamed document the old validator rejected only through a name it required, G among required while properties declared F, can be reported although the save left it as invalid as it found it.
• A collMod whose write concern was not met has already changed the validator when the save stops; the message gives the server's reason, and the documents were not touched.

Conservative refusals, accepted:

• A validator or view that reads the whole document refuses every rename and removal on its collection, {$push: "$$ROOT"} in a view included. Rewrite those from a query tab.
• Under a closed schema, removing a field properties does not declare is refused.

Carried from earlier rounds:

• MongoDB before 4.4 cannot measure a document, so a rename to a longer name is not checked against 16 MB there. The docs list it.
• A field name computed at run time by $getField in a view is covered by unit tests only: 7.0.43 rejects a view whose field argument is not a constant.
• The search-index mention check runs on fixtures built from the $listSearchIndexes output the MongoDB manual documents; no Atlas cluster was reachable.
• A column reorder that runs as metadata only keeps the grid editable while it runs.
• The in-app shell has no db.createView, and its createIndex drops wildcardProjection, language_override and other options it does not pass through.
• fetchIndexes types every MongoDB index BTREE and reads key paths in dictionary order, which is why index editing stays read-only.

No UI test. The rename flow needs a live mongod, and CI has no MongoDB server, so the live checks stand in for it. The tab refresh is covered at applyObjectChange and executeSchemaChanges; fix/refresh-written-table-tabs brings TableChangeReloadUITests for the same reload on SQLite and replaces this mechanism on merge. A save that fails raises an error alert with no window under XCTest, which hangs the test host, so the failure path is covered at the DatabaseManager level instead.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
TablePro	🟢 Ready	View Preview	Sep 26, 2026, 7:19 PM

💡 Tip: Enable Automations to automatically generate PRs for you.