Skip to content

fix(core): accept IPC acks for void callbacks under zod 4.4+ - #4980

Open
NERLOE wants to merge 4 commits into
triggerdotdev:mainfrom
NERLOE:fix/zod-ipc-ack-optional-message
Open

NERLOE wants to merge 4 commits into
triggerdotdev:mainfrom
NERLOE:fix/zod-ipc-ack-optional-message

Conversation

@NERLOE

@NERLOE NERLOE commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Closes #4979

Credit to @marcus-dk for finding and root-causing this in #4979, including the zod version bisect and the end-to-end confirmation with a patched worker.

Summary

FLUSH and CANCEL declare callback: z.void(), so the task run process acks them with { type: "ACK", id, message: undefined }. Node's JSON IPC serialization drops the undefined message key. Under zod 4.4+ a z.any() object key is required at parse time (it was optional under zod 3 and 4.0–4.3), so Packet.safeParse rejects every such ack, #handlePacket returns silently, and sendWithAck waits out its full timeout.

Because zodIpc.ts imports zod/v4 and @trigger.dev/core resolves zod from the user's project, every project on zod ≥ 4.4 pays this on every run: ~6 s between the task finishing and the completion reaching the engine (FLUSH: 5 s + 1 s), and ~31 s on cancel.

The fix makes the ACK packet's message optional again (z.any().optional()). Only the ACK packet can legitimately carry undefined, so the other packets are unchanged.

✅ Checklist

  • I have followed every step in the contributing guide
  • The PR title follows the convention.
  • I ran and tested the code works

Testing

  • Added packages/core/test/zodIpc.test.ts: forks a real child process (test/fixtures/zodIpcChild.ts) and sends FLUSH (void callback) and PING (payload callback) over Node's IPC channel, so the ack goes through the same JSON serialization as between a worker and its task run process. On main (zod 4.5.4 in this repo) the void-callback test fails with sendWithAck() timeout; with the fix it resolves immediately.
  • pnpm run build --filter @trigger.dev/core passes.
  • Production evidence from our self-hosted instance (completed_at - started_at - usage_duration_ms on successful runs):
SDK runs p10 gap p50 gap
4.5.12 2.6M 179 ms 244 ms
4.5.15 5.0M 176 ms 251 ms
4.6.0 1.6M 6,162 ms 6,196 ms
4.6.2 4.0M 6,180 ms 6,240 ms

Changelog

Fix a ~6 second delay between a task finishing and its run completing (and a ~31 second delay when cancelling a run) in projects that use zod 4.4 or newer.

🤖 Generated with Claude Code

FLUSH and CANCEL declare `callback: z.void()`, so the child acks with
`message: undefined`. Node's JSON IPC serialization drops that key, and
zod 4.4+ treats a `z.any()` key as required, so the ACK packet failed to
parse and `sendWithAck` waited out its full timeout: ~6s on every run
completion and ~31s on cancel. Make the ACK packet's `message` optional.

Fixes triggerdotdev#4979

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 42f254f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 27 packages
Name Type
@trigger.dev/core Patch
@trigger.dev/build Patch
trigger.dev Patch
@trigger.dev/python Patch
@trigger.dev/redis-worker Patch
@trigger.dev/schema-to-json Patch
@trigger.dev/sdk Patch
@internal/clickhouse Patch
@internal/llm-model-catalog Patch
@internal/metrics-pipeline Patch
@trigger.dev/rbac Patch
@internal/redis Patch
@internal/replication Patch
@internal/run-engine Patch
@internal/run-store Patch
@internal/schedule-engine Patch
@internal/tracing Patch
@internal/webhook-engine Patch
@internal/webhook-sources Patch
@internal/dashboard-agent Patch
@internal/cache Patch
@trigger.dev/react-hooks Patch
@trigger.dev/rsc Patch
@trigger.dev/database Patch
@trigger.dev/otlp-importer Patch
@trigger.dev/sso Patch
@internal/testcontainers Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 7701ccf6-3b15-49da-8f76-3666e604fafa

📥 Commits

Reviewing files that changed from the base of the PR and between 41287e7 and 42f254f.

📒 Files selected for processing (1)
  • .changeset/void-ipc-acks.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • .changeset/void-ipc-acks.md

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout. (7)
  • GitHub Check: internal / 🧪 Unit Tests: Internal (1)
  • GitHub Check: e2e / 🧪 CLI v3 tests (warp-windows-latest-x64-8x - npm)
  • GitHub Check: packages / 🧪 Unit Tests: Packages (2, 3)
  • GitHub Check: e2e / 🧪 CLI v3 tests (warp-windows-latest-x64-8x - pnpm)
  • GitHub Check: packages / 🧪 Unit Tests: Packages (3, 3)
  • GitHub Check: internal / 🧪 Unit Tests: Internal (2)
  • GitHub Check: Analyze (javascript-typescript)
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings
Learnt from: NERLOE
URL: https://github.com/triggerdotdev/trigger.dev/pull/4980

Timestamp: 2026-09-25T11:45:20.033Z
Learning: In `packages/core/src/v3/zodIpc.ts`, ACK packets without `message` were accepted under zod 3 and zod 4.0–4.3 because the `z.any()` key was optional at parse time. PR `#4980` restores that behavior for zod 4.4+. Validating ACKs against their expected callback schemas would change `sendWithAck` semantics and is separate hardening work.

Walkthrough

The ACK packet schema now permits the message field to be absent. Tests use a forked child process to check void and payload acknowledgments. A patch changeset describes the task completion and cancellation delays reported for projects using Zod 4.4 or newer.

Priority: ⬆️ High

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 42f25

The change addresses delayed completion and cancellation acknowledgments. No concrete merge-blocking risk is identified; normal checks remain appropriate.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 41287

The change restores acknowledgments from child processes when callbacks return no value. No new externally reachable entrypoint or introduced security bypass was established, but the shared completion protocol warrants review.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The affected production boundary is communication with an IPC peer used by task-run processing, including FLUSH and CANCEL acknowledgments. The newly added fixture endpoints are reached by the test’s forked child, not an identified production caller.

Security Findings and Attack Paths

  • inferred — A peer able to send a matching ACK without a message could resolve a non-void callback as undefined. This is not an established new security bypass: the previous ACK field used z.any(), permitting that peer to send an arbitrary value instead. Whether an attacker can access the production IPC channel remains unestablished.

Trust Boundaries and Controls

  • observed — The connection reads process messages, parses their packet shape, and correlates ACKs by connection-local numeric ID. The inspected ACK handler has no operation-type, session, or callback-result check.

Resilience and Maintainability Implications

  • observed — Pending calls have separate IDs and timers; timeout rejects the call, while a matching ACK clears its timer. The inspected code does not remove the pending-map entry after either outcome, a behavior outside the schema edit.

Hardening Proposals

  • proposed — If IPC peers must be treated as untrusted, bind each pending ACK to its expected callback schema and validate the returned value before treating the operation as complete. This is hardening of the existing protocol, not an established vulnerability introduced by this PR.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main fix: accepting IPC acknowledgements for void callbacks under Zod 4.4 and newer.
Description check ✅ Passed The description includes the linked issue, completed checklist, technical summary, testing details, build result, and changelog entry. It omits the Screenshots section, but that omission is non-critic…
Linked Issues check ✅ Passed The PR meets the coding requirements in issue #4979. packages/core/src/v3/zodIpc.ts makes the ACK message field optional. This accepts ACK packets that omit message after JSON IPC serialization.…
Out of Scope Changes check ✅ Passed The changes stay within issue #4979. The schema change fixes rejected void IPC acknowledgements. The fixture changes and forked-process tests verify the affected serialization path. The changeset docu…
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 4 files. (1 skipped: 1 …
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]

This comment was marked as resolved.

The `resolves` assertion already fails when the ack is rejected, and a
wall-clock bound could flake on a slow runner. Also hyphenate the
changeset's compound adjectives.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@NERLOE

NERLOE commented Sep 25, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai On the retained reliability concern (an ACK without message can now resolve a non-void request): this is the behaviour zodIpc had under zod 3 and zod 4.0–4.3, where a z.any() key was already optional at parse time. This PR restores it for zod 4.4+ rather than introducing it. Validating each ack against its expected callback schema would be a reasonable hardening, but it changes sendWithAck semantics beyond this bug fix, so I'd keep it out of scope here.

Changeset wording fixed in d098714 as well.

For maintainers: the failing webapp unit/E2E checks are the fork-secrets issue (DockerHub login skipped because secrets are not available, then quay.io/minio/minio pull is unauthorized). code-quality, typecheck and all package unit-test shards (which run the new test) pass.

@NERLOE
NERLOE marked this pull request as ready for review September 25, 2026 11:43
devin-ai-integration[bot]

This comment was marked as resolved.

@coderabbitai

coderabbitai Bot commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
fd -a '^(zodIpc\.ts|zodIpc\.test\.ts|void-ipc-acks\.md)$'
ast-grep outline packages/core/src/v3/zodIpc.ts --items all
ast-grep outline packages/core/test/zodIpc.test.ts --items all
sed -n '1,280p' packages/core/src/v3/zodIpc.ts
sed -n '1,180p' packages/core/test/zodIpc.test.ts
cat .changeset/void-ipc-acks.md
git show HEAD^:packages/core/src/v3/zodIpc.ts | sed -n '1,110p' || true

Length of output: 16548


@NERLOE, thanks for the clarification. I agree that per-callback ACK validation is a separate hardening change, not a requirement for this fix. I withdraw the retained reliability concern as a finding on this PR.

I also see the corrected changeset wording. Thanks for the context on the webapp checks; I have not independently verified their failure logs. Would you like a follow-up issue for ACK callback validation?


✏️ Learnings added
Learnt from: NERLOE
URL: https://github.com/triggerdotdev/trigger.dev/pull/4980

Timestamp: 2026-09-25T11:45:19.903Z
Learning: In `packages/core/src/v3/zodIpc.ts`, ACK packets without `message` were accepted under zod 3 and zod 4.0–4.3 because the `z.any()` key was optional at parse time. PR `#4980` restores that behavior for zod 4.4+. Validating ACKs against their expected callback schemas would change `sendWithAck` semantics and is separate hardening work.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

NERLOE and others added 2 commits September 25, 2026 13:48
Replace the in-memory JSON channel with a forked child process so the ack
goes through Node's actual IPC serialization.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4.6.0+ with zod ≥ 4.4: every run completion waits a fixed 6 s (cancels 31 s) because FLUSH/CANCEL IPC acks are rejected

2 participants