Skip to content

fix(plugin-mongodb): send the connection's write concern with every shell write and name the documents a failed write already changed - #3151

Open
datlechin wants to merge 1 commit into
mainfrom
fix/mongodb-shell-write-concern
Open

datlechin wants to merge 1 commit into
mainfrom
fix/mongodb-shell-write-concern

Conversation

@datlechin

Copy link
Copy Markdown
Member

Root cause

Write Concern was ignored by everything but inserts. The setting reaches libmongoc only through the URI (w=), and libmongoc applies it only on its CRUD calls. The shell sends every update, replace, delete, findAndModify and bulkWrite as a raw command through mongoc_client_command_simple, which applies no write concern. The command builders never added one, and the prelude dropped the options argument of insertOne, insertMany, insert and bulkWrite, so a statement's own writeConcern was dropped too. Grid edits and deletes run as the same shell statements, so they ignored Write Concern as well, while grid inserts obeyed it.

A write that stopped part-way never said so. MongoDB undoes nothing outside a transaction, but every multi-document path lost track of what it had written:

  • writeCommand threw the first writeErrors message.
  • scriptInsert threw away the reply that held insertedCount.
  • bulkWrite lost its running totals at the first throw.
  • A timeout or a killOp came back as a plain error, and mapExecutionError replaced any code-50 message with read-oriented "add an index" text.

Measured on 7.0.43: an updateMany that changed docs 1 and 2 and failed on doc 3 replies nModified: 0.

Found on the way: remove(filter, {justOne: true}) deleted every match, because the prelude only honoured a literal true. And an insertMany that stopped at an oversized document after libmongoc had already sent a batch failed with an empty message: libmongoc 1.28.1 clears its error when the last batch it sent succeeded (mongoc-write-command.c, end of _mongoc_write_opmsg).

Fix

Write concern

  • MongoDBConnection.writeConcernJson(client:) reads the client's write concern from libmongoc, so w, journal and wtimeoutMS from a pasted URL all come along. With Default it returns nil and nothing is sent.
  • MongoScriptCommandBuilder.writeConcern(statementOptions:connectionDefault:) takes the statement's own, else the connection's, the way mongosh does. The statement's is rebuilt from w, j and wtimeout, taking mongosh's journal, wtimeoutMS and fsync too. One that names none of them ({}, null, unknown keys) counts as unset. Measured: 7.0.43 refuses journal or wtimeoutMS in a command with [40415] IDLUnknownField, and libmongoc 1.28.1 drops them from an insert's options along with the connection's own concern. mongosh 2.10.0 sends {j, wtimeout} and falls back on {}.
  • Inserts keep libmongoc's own inheritance. insertOptions(statementOptions:) passes only the statement's rebuilt writeConcern and ordered. A statement's w: 0 beside j: true (or journal, or fsync) goes to the insert as w: 1, j: true: libmongoc 1.28.1 refuses the pair with [22] Invalid writeConcern (mongoc_write_concern_is_valid), and the server treats the two the same, since it leaves out a write's reply only for a w below 1 with neither j nor fsync (shouldSkipOutput, src/mongo/db/commands/write_commands.cpp on v7.0). Commands keep the pair as written.
  • isAcknowledged(writeConcern:): a numeric w of 0 or below without j: true. Measured: a w: 0 update, delete or insert command is answered n: 0 whatever it changed, and a duplicate key it hit is not reported. libmongoc sends an insert with its legacy w: -1 without waiting for any answer, and the server drops it (it refuses a negative w). Those writes return {acknowledged: false} with no counts (inserts keep their ids), as mongosh does. findAndModify is answered in full under w: 0, so it keeps its document. An update or delete with w: -1 fails with the server's own refusal, as it does in mongosh.
  • The prelude forwards options for insertOne, insertMany, insert and bulkWrite. remove takes true or {justOne, …} like mongosh.
  • Raw commands stay. Measured on libmongoc 1.28.1: mongoc_collection_update_one rejects a replacement document on the client, which legacy update(q, doc) relies on. The Bulk API rejects maxTimeMS, which is how the query timeout bounds a write.

Partial writes

  • MongoWriteFailure gains a stage: .document, .unconfirmed, .command, .unanswered or .notSent. It reads the CRUD reply's plural writeConcernErrors and errorReplies, and a top-level ok: 0. The code always comes from the reply, never from bson_error_t.
  • scriptWriteCommand and scriptInsert return a MongoWriteOutcome that keeps the reply even when the write failed.
  • The host enters every write in a statement-scoped MongoWriteLedger under its activity lock, and marks a write in flight so the silence watchdog can see it. An unacknowledged write counts as one that may have changed documents. Every stage counts what its reply says was written, .notSent included: a large insertMany goes out in batches, and one that stops at a document it cannot send has already inserted the batches before it. That insert now reports "The insert stopped at a document larger than MongoDB accepts." rather than an empty message.
  • A command stopped with ok: 0 and a code in the server's Interruption category may have written documents first: the server fails the whole batch for exactly those codes (write_ops_exec.cpp, handleError). MongoDBServerErrorCode.interruptionCategory is the union of that category on every release branch from 4.0 to 9.0, read from each branch's src/mongo/base/error_codes.yml (error_codes.err on 4.0 and 4.2). It adds 91331 (8.0), 10045600 (8.1), 453 (8.2), 471, 473 and 485 (8.3) and 509 (9.0 branch) to the 13 codes 5.1 to 7.3 share. scripts/check-mongodb-interruption-codes.sh diffs the set against every branch, the way the Redis and MySQL curated tables are checked.
  • A failed write travels as itself. The host throws the MongoWriteFailure, its stage crosses the bridge on the exception, and the driver reads it from the error that actually escaped. Before, it matched the last failed write by code and message, so a caught write timeout made a later count timeout (same [50] operation exceeded time limit) read as a write.
  • The driver builds every statement error in one place (reportedError): timeout wording first, write wording only for a write's own failure, the note after. A write-concern error keeps its "applied" text even when its code is 50. A cancel stays a quiet cancel.
  • An export whose statement wrote and then built a cursor carries those writes, and its use, into the stream. A stream that fails reports the note, and the driver follows the use on success and on failure.

Verified

  • Unit tests: 296 executed, 296 passed across the MongoDB suites, on current main.
  • The new assertions for this round (a .notSent insert keeping its count, w: 0 with j, journal or fsync on an insert, w: -1, and the seven newer interruption codes) fail 14 of 14 when compiled against the previous commit's sources and pass 14 of 14 against this one.
  • Builds: MongoDBDriver (only its 4 existing Sendable-capture warnings), AllPlugins (all 40), the app. Lint: 0 violations on the 19 changed Swift files. Docs checks pass. Plugin localization check passes. shellcheck --severity=warning passes on the new script, and the script reports ok: 20 codes, the union across v4.0 to v9.0 (a copy with 453 swapped for 11000 reports both).
  • Live, standalone MongoDB 7.0.43 through libmongoc 1.28.1, harness binaries built from the previous revision and from this one:

Write Concern 2

Statement Before After
updateOne, deleteMany, findOneAndUpdate, bulkWrite with {journal: true, wtimeoutMS: 1000} [40415] ... 'WriteConcernOptions.journal' is an unknown field. succeed, server receives {j: true, wtimeout: 1000}
insertOne with the same succeeds, no write concern sent succeeds with {j: true, wtimeout: 1000}
updateOne / insertOne with writeConcern: {} succeed, connection's w: 2 dropped fail with cannot use 'w' > 1 when a host is not replicated
{fsync: true, w: 1} sent as written sent as {w: 1, j: true}
Grid edit and grid delete of two rows fail with the w > 1 message, data unchanged; succeed under Default

Write Concern Majority with URL journal=true&wtimeoutMS=4321: update and insert go out as {w: "majority", j: true, wtimeout: 4321}.

w: 0 and w: -1

Statement Before After
updateOne w: 0 acknowledged: true, matched 0, modified 0, document changed {acknowledged: false}
insertMany w: 0 with a duplicate first acknowledged: true, insertedCount: 2, nothing inserted {acknowledged: false, insertedIds}
deleteMany, bulkWrite w: 0 zero counts, documents changed {acknowledged: false}
updateMany w: 0, then a duplicate insertOne E11000 only E11000 plus "Some documents may already have been changed"
URL w=0, insertOne acknowledged: true acknowledged: false; a statement's {w: 1} still gets real counts
insertOne / insertMany with {w: 0, j: true}, {w: 0, journal: true} [22] Invalid writeConcern inserted, real counts, wire {w: 1, j: true}
duplicate insertOne with {w: 0, fsync: true} [22] Invalid writeConcern E11000 duplicate key error
insertMany with {w: -1} acknowledged: true, insertedCount: 2, nothing inserted {acknowledged: false, insertedIds}
URL w=-1, insertOne acknowledged: true, nothing inserted acknowledged: false
insertOne w: -1, then a duplicate insertOne E11000 only E11000 plus "Some documents may already have been changed"
updateOne with {w: -1} [9] w has to be a non-negative number ... same, as mongosh 2.10.0

{w: 0, j: true} on updateOne keeps real counts, and findOneAndUpdate with w: 0 returns its document in both.

Partial writes, 1 s query timeout

Case Before After
Caught updateMany timeout, then a count timeout "The write did not finish…" "The query timed out…" plus the note
Same updateMany rethrown write wording plus the note same
Export of updateOne; find(slow) timeout text, no note timeout text plus "1 document(s) had already been changed"
Export of use("b"); db.c.find(), success and timeout driver stays on the old database driver follows to b
insertMany of four 15 MB documents and a 17 MB one (3 inserted) empty message, no note "The insert stopped at a document larger than MongoDB accepts." plus "3 document(s) had already been changed"
insertMany whose connection a proxy drops after forwarding it (both inserted) transport error plus "may already have been changed" same

The tables above hold each round's before and after. Rechecked on this final build: validator updateMany failing on its 3rd document gets the "may" note, ordered and unordered insertMany with duplicates report 2, a bulkWrite of insert, updateMany over 6, duplicate insert reports 7, remove({...}, {justOne: true}) deletes 1, the caught-timeout-then-count statement reads as a query, a lone updateMany timeout reads as a write, the updateOne; find(slow) export carries its count, the use export follows the database, and under Write Concern 2 the journal/wtimeoutMS, {} and fsync rows behave as tabled. The grid edit and delete rows were not rerun this round.

  • Codex review, round 3: an unacknowledged insert that stopped between batches dropped its "may already have been changed" note. It now keeps it (unacknowledgedStopBetweenBatchesMayHaveChanged).

Deliberately not fixed here

  • Unordered bulkWrite: operations still run one at a time and stop at the first failure. Documented as a limitation.
  • w: 0 changes what a command write reports, not how long it takes: mongoc_client_command_simple always waits for the server's reply.
  • A partial $merge/$out failure from aggregate gets no count. aggregate already applies the write concern through libmongoc.
  • JavaScript catch blocks see only the server message, with no result counts like mongosh's MongoBulkWriteError.
  • Index, collection and database commands and db.runCommand go to the server as written. The docs say so.
  • An insertMany with an oversized document still inserts the batches libmongoc sent before it; the error now says so and counts them. Refusing the whole call up front needs a size check against the server's limit before the insert, which is its own change.

No UI test: CI has no MongoDB server, so a flow that writes to a collection cannot run deterministically there. The grid and shell paths were checked live against 7.0.43 instead.

…hell write and name the documents a failed write already changed
@mintlify

mintlify Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
TablePro 🟢 Ready View Preview Sep 26, 2026, 4:52 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

This branch was successfully deployed

1 active deployment
staging - docs — 93fc1e9d Deployed Sep 26, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant