Skip to content

fix: Check Any, not the delegating downcast, in FFI_ExecutionPlan::new's foreign-echo shortcut - #25645

Closed
namanjain24-sudo wants to merge 1 commit into
apache:mainfrom
namanjain24-sudo:fix-ffi-execution-plan-wrapper-downcast
Closed

namanjain24-sudo wants to merge 1 commit into
apache:mainfrom
namanjain24-sudo:fix-ffi-execution-plan-wrapper-downcast

Conversation

@namanjain24-sudo

Copy link
Copy Markdown
Contributor

related: #25155

This addresses only "Direction 3" (the FFI_ExecutionPlan::new shortcut fix)
from that issue, using the exact change the issue itself proposes. The issue
also covers a separate, larger problem ("Direction A") that needs its own
design discussion and is not touched here, so this should not close it.

What happened

FFI_ExecutionPlan::new short-circuits when the incoming plan is already a
ForeignExecutionPlan, handing back its original FFI handle instead of
re-wrapping it:

if let Some(plan) = plan.downcast_ref::<ForeignExecutionPlan>() {
    return plan.plan.clone();
}

That check goes through ExecutionPlan::downcast_ref, which follows
downcast_delegate when a plan opts in (added in #22557 so a transparent
wrapper can redirect public downcasts to its inner plan). A wrapper that
delegates to a ForeignExecutionPlan is therefore misclassified as being
one, and the shortcut discards the wrapper, returning the plan from before
it was ever applied.

What changes are included in this PR

Switch the check to a raw Any downcast, which only matches plan's own
concrete type rather than what downcast_delegate redirects to:

if let Some(plan) = (plan.as_ref() as &dyn std::any::Any).downcast_ref::<ForeignExecutionPlan>() {
    return plan.plan.clone();
}

For a plan whose own concrete type really is ForeignExecutionPlan (which
does not implement downcast_delegate), both checks agree, so this only
narrows the shortcut for the wrapper case. No ABI change.

What is the testing strategy for this PR

  • New unit test in datafusion/ffi/src/execution_plan.rs: a minimal
    downcast_delegate-opted-in wrapper around a plan mocked as foreign (the
    existing mock_foreign_marker_id pattern already used by sibling tests in
    this file) must survive FFI_ExecutionPlan::new. Verified it fails
    against the unpatched check — the wrapper is discarded and the round trip
    reports the inner plan's name instead of the wrapper's — and passes with
    the fix.
  • cargo test -p datafusion-ffi --lib (120 tests) and every cross-library
    integration test binary under --features integration-tests
    (ffi_execution_plan, ffi_query_planner, ffi_physical_optimizer,
    ffi_integration, ffi_udaf, ffi_udf, ffi_udtf, ffi_udwf,
    ffi_catalog, ffi_config; 31 tests), all passing.
  • cargo fmt --check and cargo clippy -p datafusion-ffi --all-targets --all-features report nothing for the changed file.

Are there any user-facing changes?

A transparent downcast_delegate wrapper around a plan that already crossed
the datafusion-ffi boundary is no longer silently discarded by
FFI_ExecutionPlan::new. No public API changes.

…w's foreign-echo shortcut

related: apache#25155 (Direction 3 only; Direction A in that issue is a separate,
larger problem needing its own design discussion, so this does not close it)

FFI_ExecutionPlan::new short-circuits when the incoming plan is already a
ForeignExecutionPlan, handing back its original FFI handle instead of
re-wrapping it. That check went through ExecutionPlan::downcast_ref, which
follows downcast_delegate when a plan opts in (added in apache#22557 so a
transparent wrapper can redirect public downcasts to its inner plan).

Because of that, a wrapper delegating to a ForeignExecutionPlan is
misclassified as being one, and the shortcut discards the wrapper, returning
the plan from before it was ever applied.

Switch the check to a raw Any downcast, which only matches plan's own
concrete type. This narrows the shortcut without an ABI change: for a plan
whose own concrete type really is ForeignExecutionPlan (which does not
implement downcast_delegate), both checks already agreed, so this only
changes behavior for the wrapper case.
@github-actions github-actions Bot added the ffi Changes to the ffi crate label Sep 23, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 30.64516% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.44%. Comparing base (7570366) to head (c570149).

Files with missing lines Patch % Lines
datafusion/ffi/src/execution_plan.rs 30.64% 41 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #25645      +/-   ##
==========================================
- Coverage   82.45%   82.44%   -0.02%     
==========================================
  Files        1140     1140              
  Lines      436589   436650      +61     
  Branches   436589   436650      +61     
==========================================
+ Hits       359996   360001       +5     
- Misses      54838    54889      +51     
- Partials    21755    21760       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ShayanGho

Copy link
Copy Markdown
Contributor

@namanjain24-sudo Thanks for working on this.

The constructor change looks correct, and all 120 FFI unit tests passed locally. #25717 also identifies the delegate-aware child check in pass_runtime_to_children, which remains unchanged here; the new test doesn’t exercise runtime propagation.

Since #25749 covers both checks, could we coordinate the overlapping PRs and clarify which one should land?

@namanjain24-sudo

Copy link
Copy Markdown
Contributor Author

Closing in favor of #25749, which does the same FFI_ExecutionPlan::new fix plus the pass_runtime_to_children fix this PR didn't cover, and is already approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ffi Changes to the ffi crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants