fix(plugin-mongodb): store fields named "" and top-level $ names on insert

[datlechin]

Root cause

Two client-side rules refused documents the server stores on insert.

1. libmongoc's default insert check. Insert Document calls mongoc_collection_insert_one, and scriptInsert calls mongoc_collection_insert_many. scriptInsert is behind every shell insertOne, insertMany, insert and save, and therefore behind every grid new row, duplicate, paste and undo-delete restore. Both calls passed no options. libmongoc 1.28.1 then applies _mongoc_default_insert_vflags, which is BSON_VALIDATE_UTF8 | BSON_VALIDATE_UTF8_ALLOW_NULL | BSON_VALIDATE_EMPTY_KEYS. It refuses a field named "" at any depth with [22] invalid document for insert: empty key before anything is sent. The server has no such rule. MongoDB 7.0.43 stores the field through the raw insert command and through mongosh, and TablePro's own bulkWrite, which sends a raw command, already stored it.
2. MongoDocumentText refused a top-level name starting with $ as something "MongoDB reads as an operator". That is true of a replacement, not of an insert. MongoDB 5.0 and later stores the name as written: insert.cpp r5.0.0 gates the old refusal behind featureFlagDotsAndDollars, which is on by default. Servers before 5.0 refuse it themselves with BadValue. So Insert Document refused a document that its own insertOne statement stores when run in the query editor.

Fix

• New pure type MongoInsertOptions, compiled into TableProTests. It holds the options every insert hands libmongoc: {"validate":9}. That is libmongoc's default check without BSON_VALIDATE_EMPTY_KEYS, so a key that is not UTF-8 is still refused. The value is never false or 0, which turns every check off.
• insertDocument and scriptInsert parse that one text and pass it to insert_one and insert_many.
• The top-level $ check in MongoDocumentText.init(parsing:) is gone, along with Refusal.operatorField and its catalog string. Nothing else referenced them.
• The query editor reads each document a cursor returns, and the document findOneAndUpdate, findOneAndReplace and findOneAndDelete return, from its fields, never as one value. A document whose only field is named $oid, $date or another Extended JSON wrapper, which this change lets Insert Document store, used to come back from find({}, {_id: 0}) as that value, or fail when the text was not a valid one.
• Docs (databases/mongodb.mdx):
  • Field names are stored as typed. MongoDB before 5.0 refuses dotted and top-level $ names.
  • mongosh reads a top-level $ref plus $id as a DBRef.
  • New Limitations bullet for filtering, sorting and editing these names from the grid, with $getField/$setField as the workaround.
• CHANGELOG Fixed entry for the shell and grid half, which shipped in 0.75.0. Insert Document is unreleased, so its $ half gets no Fixed entry.

Verified

• verify.sh test, 8 suites (MongoInsertOptionsTests, MongoDocumentTextTests, MongoDocumentWritePlanTests, MongoDBStatementGeneratorTests, MongoDBWriteBackTypeTests, MongoScriptCommandBuilderTests, MongoScriptCursorOptionsTests, MongoWriteFailureTests): 132 executed, 132 passed.
• The new tests can fail. I set the value back to 25 (EMPTY_KEYS in) and put a top-level $ refusal back in the reader: 30 executed, 3 failed, namely dropsOnlyTheEmptyNameCheck, namesAreReadAsWritten and namesTheServerStores. MongoInsertOptionsTests does not compile on main.
• verify.sh build MongoDBDriver, plugins (AllPlugins), build (TablePro): all PASS.
• Codex review, round 1: a P2 that a wrapper-named document read back through a projection turned into the wrapper's value, and a P3 that the docs gave only $ names the 5.0 caveat when dotted names share it. Both fixed. wrapperNamedDocumentStaysADocument and findAndModifyDocumentStaysADocument fail against main's prelude and pass here.
• Rebased onto main at c21dc512e: MongoScriptPreludeTests, MongoDocumentTextTests, MongoDocumentWritePlanTests, MongoInsertOptionsTests and StringCatalogIntegrityTests, 64 of 64. MongoDBDriver and the app build.
• verify.sh lint on the changed Swift files: 0 violations. verify.sh docs: PASS. localization.py plugins and verify: ok.
• Live, MongoDB 7.0.43, the real plugin sources from main and from this branch, same inputs:
  • Insert Document: 2 of 12 stored on main, 10 of 12 on the branch. The two still refused are {"_id": {"$foo": 1}}, refused by the server with [52] _id fields may not contain '$'-prefixed fields, and {"": 1, "": 2}, refused by the reader as a repeated field.
  • Query editor: 1 of 7 inserts with a "" field stored on main (only bulkWrite), 7 of 7 on the branch. Cases: insertOne at top level, nested and inside an array element, insertMany, legacy insert, and save without _id.
  • Grid duplicate db.g.insertOne({"": "blank", "a": {"": 2}, "name": "n"}) and its undo-delete restore with _id 7: both [22] ... empty key on main, both stored on the branch.
  • Browse after the inserts: 13 columns, [_id] [] [a] [b] [$foo] [a.b] [$set] [ ] [$oid] [$date] [$numberInt] [$id] [$ref].

Composition with work in flight

Edit Document (#3152). Its replacement always starts with the stored _id. I measured on 7.0.43 with {"validate": false}:

• {"_id": 1, "$set": {"x": 1}} is refused by the server with code 52, and the stored document is unchanged.
• Only a replacement whose first key is $set is applied as an update: {"$set": {"x": 1}} turned {_id: 1, orig: 1} into {_id: 1, orig: 1, x: 1}. Edit Document cannot build that shape.

So removing the reader's refusal opens no silent write. What Edit Document loses is its open-time refusal: a document with a top-level $ field would open in the editor and then fail every save with code 52. After rebasing onto this, that branch needs:

• MongoDocumentReplacementTests.topLevelOperator, MongoEditableDocumentTests (line 84) and its copy of MongoDocumentTextTests.operatorFields fixed, since all three reference Refusal.operatorField and will not compile.
• A replace-only top-level $ refusal in MongoDocumentReplacement, next to emptyTimestampField, applied at save and when MongoEditableDocument opens a stored document, with a new string added through localization.py plugins --add.
• Optionally, validate 9 in place of false. Measured, it refuses $set client-side with Invalid key '$set': replace prohibits $ operators and still stores {"_id": 1, "": 1, "a": {"": 2}}.
• MongoDocumentWritePlanTests.namesTheServerStores rewritten against plan?.write == .insert(document:), since that branch replaces plan.document and makes make return an optional.
• An overlap in MongoDBConnection+Documents.swift with its replaceDocument and read code. My hunk there is the options parse inside insertDocument.

Shell write concern (#3151). Both branches change scriptInsert's insert_many options. libmongoc applies its EMPTY_KEYS default to any options document with no validate key. So its insertOptions(statementOptions:) must always return a document carrying "validate": MongoInsertOptions.validation beside writeConcern and ordered, never nil, with a test that says so.

Grid write values (fix/mongodb-grid-write-values) and null-vs-missing on top of it. The generator's insert refusal of a "" field existed only because of libmongoc's default. It can drop to __proto__ only, and insertRefusesUnwritableNames should change to match. Its $setField edits also make the editing half of the new Limitations bullet stale for $ and dotted names, so its docs change should update that bullet.

Deliberately not fixed here

• Grid filter, sort and edit on special names, measured the same on main and here:

  • "": filter and sort are refused by libmongoc's find ([16] Invalid filter: empty key, [16] Invalid opts: empty key), and an edit gets server code 56.
  • A name starting with $: the server refuses filter (2), sort (16410) and edit (52).
  • A dotted name: filter and sort use the nested path, and an edit sent as {"$set": {"a.b": "edited"}} changed b inside a and left the literal a.b field alone.

  All of this predates the change, since the shell and Insert Document already stored dotted names. It is now documented as a limitation. The edit half belongs to the grid write values branch.

• A top-level $ref and $id pair is stored as written. mongosh reads that document back as a DBRef, while TablePro shows two columns. It is kept for parity, since mongosh writes the same document, and the docs say so.

• MongoDBConnection.insertOne/updateOne/deleteOne(database:collection:...) and their *Sync helpers have no callers and still pass no options. Removing them is a separate refactor.

• The shell sends replaceOne(filter, {$set: ...}) as a raw update command, so the server applies it as an update, while mongosh refuses it. That belongs to the shell writes work.

• A document nested inside another whose only field is named like an Extended JSON wrapper still reads back in the query editor as that wrapper's value. The driver hands documents to JavaScript as Extended JSON, where the two spell the same, so telling them apart needs the BSON type carried alongside, which is its own change. The root of every returned document is fixed here.

• Servers before 5.0 are covered by the insert.cpp source only (r3.6.23 to r4.4.29), not measured live.

No UI test: CI has no MongoDB server, and Insert Document, the query editor insert and the grid insert all need a connected collection. Coverage is the unit tests above plus the live before and after run.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
TablePro	🟢 Ready	View Preview	Sep 26, 2026, 6:21 PM

💡 Tip: Enable Automations to automatically generate PRs for you.