fix(plugin-mongodb): send $type, $regex and $options objects as the operator documents they are written as

[datlechin]

Root cause

libbson's JSON reader (1.28.1) decides what an embedded object is from its first key alone. $type, $regex and $options open its legacy binary and regular expression values, and they are also MongoDB operators. Every shell payload reaches BSON through MongoDBConnection.jsonToBson, which passed the whole text to bson_new_from_json. So:

• {sig: {$type: "binData"}} failed with Missing "$binary" after "$type". So did any alias that starts with a hex digit (double, bool, date, decimal, array). A $type list failed with Invalid read of "[".
• {name: {$regex: "^a", $exists: true}} failed with Invalid key "$exists", and {name: {$regex: /^a/i}} failed with Unexpected nested object value.
• {$regex: "a", $options: "i"} became a regular expression value. mongosh never does that. An insert stored a regex where mongosh stores a subdocument. A stored subdocument of that shape turned into a regex when a script, or a rerun of Show DDL, wrote it back.

Only the first key of an object below the root triggers this, and libbson decodes escapes before it compares, so "$type" is misread too. libbson has no flag to turn the legacy grammar off.

Fix

• MongoBsonAssembly (new, pure) scans the UTF-8 bytes for an object below the root that opens with one of the three keys. When there is none, the text goes to libbson whole, as before. The scan takes 6 ms on a 6.5 MB text, against 149 to 201 ms for libbson's own parse.
• When there is one, and the text is well-formed JSON, the text is planned as parts:
  • A member that holds no such object becomes a one-member document, {"key": value}. libbson reads its value in nested context, so wrappers such as {"$oid": …} still work, and a root key is never special.
  • A container that holds such an object is built member by member.
  • Arrays are keyed 0, 1 and on.
  • A value wrapper ($code with $scope, and the rest) is never taken apart.
• jsonToBson keeps its signature and hands the text to MongoBsonBuilder, a file of its own so the probe below compiles the same code. It builds the parts with bson_new, bson_concat and bson_append_document/bson_append_array with explicit key lengths, so a key holding NUL is refused. Any failure discards the partial document.
• Every caller of jsonToBson was checked, and each now reads its text the way mongosh would:

Caller	Text	Reading
Script host: find, aggregate, runCommand, insert, update, count, explain, createIndex, bulkWrite	the prelude's serialization of script values	mongosh
Export streams	the script runtime's cursor plan	mongosh
Grid count (countDocuments)	the builder's filter, plus the Raw Filter row after the prelude	mongosh, because the same text runs as find(...) in the script for the rows
Metadata commands, ping, buildInfo, options	app-built constants	no operator-first object, so nothing changes

• MongoDBQueryBuilder: NOT CONTAINS and case-insensitive != send {$not: {$regularExpression: …}}. $not accepts a $regex operator document only from MongoDB 4.0.7 but accepts a regex value on every server, and the docs promise no minimum version. These arms send exactly what they sent before. The positive arms keep {$regex, $options}, which still pastes into mongosh.
• scripts/check-mongodb-filter-shapes.sh, which CI runs, compiled a C probe that sent every shape to bson_new_from_json whole, so it never ran the assembler it claimed to check. It now compiles MongoScriptJson, MongoBsonAssembly and MongoBsonBuilder with swiftc against Libs/libbson, builds every shape through them, and checks the BSON type libbson stored at each path where the two readings differ, read back from its canonical Extended JSON. Two raw shapes join the list: {sig: {$type: "binData"}} and {f: {$regex: "^a", $exists: true}}.
• Two force unwraps in buildUri are replaced with the same text as in fix(plugin-mongodb): list views as views, keep index key order and options, and add db.createView to the shell #3150, so the file lints clean.
• Docs: a paragraph under Writing queries and a Limitations line for a Code scope. CHANGELOG: one Fixed entry and one Changed entry.

Verified

• Tests: 216 executed, 216 passed across 8 suites, including the 12 new MongoBsonAssemblyTests cases. Among them:
  • one runs the real prelude and checks that its filter and insert output reach the planner as documents;
  • one does the same for the Raw Filter normalizer and the builder;
  • one checks that joining every part back gives the exact input.
• Builds: build MongoDBDriver and plugins (all 40) pass.
• Lint: 0 violations on the changed Swift files. Docs checks pass.
• check-mongodb-filter-shapes.sh: all 30 shapes build as the query means them. With the planner reduced to the old whole-text parse, 5 fail: three $regex shapes stored as regex values instead of documents, {$type: "binData"} refused with Missing "$binary", and $regex beside $exists refused with Invalid key "$exists". shellcheck and actionlint pass.
• Codex review, round 1: one P2, that the probe exercised libbson's whole-text parse and not the assembler. Fixed as described above.

Live on MongoDB 7.0.43, using a harness built from the plugin sources before and after the change, compared with mongosh 2.10.0:

• 14 script statements: before, 5 finds and countDocuments were refused. After, all 14 match mongosh. That includes {$in: [{$regex: "^a"}]}, which is now refused with the same cannot nest $ under $in. It used to return [1,3].
• Inserts and write-back: before, an insert holding {$type: "binData"}, {$regex: "a", $options: "i"} or {$type: "00", $binary: "AAAA"} was refused, find().forEach(replaceOne) over such a stored document was refused, and $set: {u: {$regex: "x"}} stored a regex. After, every stored document is byte-identical to what mongosh stores.
• A validator with $type: "binData", a view pipeline with $type: "array", a partialFilterExpression with $type: "double", a bulkWrite updateMany with a $type filter, export, explain and cursor count were all refused before and all succeed now.
• Grid filters: these 10 return the same rows and counts before and after: CONTAINS, NOT CONTAINS in both cases, REGEX, =, !=, IN and NOT IN ignoring case, STARTS WITH and ENDS WITH.
  • The profiler shows the $not arms unchanged on the wire.
  • The positive arms went from a regex value to the operator document.
  • Explain gives identical IXSCAN bounds for /^a/ and {$regex: "^a"}, with and without i.
• Raw Filter {sig: {$type: "binData"}} was refused for both rows and count before. Now it returns row 1 with count 1.
• Show DDL rerun of a validator {name: {$regex: "^a", $options: "i"}} used to store the validator back as a regex. It now stays the operator document.
• 19 ordinary statements give identical output before and after: dates, ObjectId, long, decimal, timestamp, MinKey, binary, updates, findOneAndUpdate, distinct, a collation index, delete, explain and ping.
• Worst case measured: a 6.5 MB array with one operator document among 50,000 siblings takes 1.1 s to plan. Most of that (580 ms) is MongoScriptJson.topLevelElements, which the host already runs on every insertMany and bulkWrite array.

Deliberately not fixed here

• Values pasted from the grid, and JSON import: a cell holding {"$options": "i", "$regex": "^a"} is now stored as an embedded document, not a regex value. That is exact for how the grid shows a stored subdocument. It changes a legacy EJSON v1 regex brought in from a pre-4.2 mongoexport file. fix/mongodb-grid-write-values rewrites that path and lists $regex in MongoExtendedJsonForm.wrapperKeys. Whichever of the two lands second should drop it there, so the grid and the shell agree.
• Insert Document and Edit Document read Extended JSON through bson_new_from_json directly, and are unchanged here. Edit Document (feat(plugin-mongodb): edit MongoDB documents as Extended JSON, found by their stored _id and saved under an exact guard #3152) should move both to the planner together.
• A Code scope holding an object that opens with $type, $regex or $options is still refused. It is documented, with the workaround.
• Number serialization, revive and the missing constructors (items 1 to 3 of the design) go in a separate PR.
• No server older than 4.0.7 was available to test. The $not arms send the same regex value as before.

No UI test: every path needs a connected MongoDB collection, and CI has no MongoDB server. The planner and the builder are covered in-process. TableProTests does not link CLibMongoc, so the libbson emitter is covered by the live checks above and by check-mongodb-filter-shapes.sh in CI.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
TablePro	🟢 Ready	View Preview	Sep 26, 2026, 6:11 PM

💡 Tip: Enable Automations to automatically generate PRs for you.