Skip to content

Conversation

@pepijnve
Copy link
Contributor

Which issue does this PR close?

Rationale for this change

The initial implementation of CooperativeExec#maintains_input_order is not correct and is likely to return a Vec with the wrong size.

What changes are included in this PR?

Update the implementation of maintains_input_order to vec![true; self.children().len()] since CooperativeExec does not modify input order in any way.

Are these changes tested?

Still working on unit test(s) that demonstrate the problem.

Are there any user-facing changes?

No

@github-actions github-actions bot added the physical-plan Changes to the physical-plan crate label Jul 31, 2025
@pepijnve
Copy link
Contributor Author

@alamb this one might require a 49.0.1 😬

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pepijnve

I have a suggestion on testing: #16994 (comment),

@alamb
Copy link
Contributor

alamb commented Jul 31, 2025

@alamb this one might require a 49.0.1 😬

I added it to a "Maybe even 49.0.1" list on

@pepijnve
Copy link
Contributor Author

pepijnve commented Aug 1, 2025

I added a number of invariant checks as suggested in #16994 (comment)
I had to introduce a separate 'default checks' function since it seems there's no way to call the default implementation of a trait function from an implementation that overrides it.

Add checks that verify the length of the vectors returned by methods that need to return a value per child.

/// Checks a set of invariants that apply to all ExecutionPlan implementations.
/// Returns an error if the given node does not conform.
pub fn check_default_invariants<P: ExecutionPlan + ?Sized>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 this is a good one

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @pepijnve

@alamb alamb merged commit 2968331 into apache:main Aug 4, 2025
27 checks passed
@alamb
Copy link
Contributor

alamb commented Aug 4, 2025

Thanks again @pepijnve and @xudong963

@pepijnve pepijnve deleted the issue_16994 branch August 7, 2025 10:26
pepijnve added a commit to pepijnve/datafusion that referenced this pull request Aug 7, 2025
…ec of the correct size (apache#16995)

* apache#16994 Ensure CooperativeExec#maintains_input_order returns a Vec of the correct size

* apache#16994 Extend default ExecutionPlan invariant checks

Add checks that verify the length of the vectors returned by methods that need to return a value per child.

(cherry picked from commit 2968331)
xudong963 pushed a commit that referenced this pull request Aug 7, 2025
…the correct size (#16995) (#17068)

* #16994 Ensure CooperativeExec#maintains_input_order returns a Vec of the correct size

* #16994 Extend default ExecutionPlan invariant checks

Add checks that verify the length of the vectors returned by methods that need to return a value per child.

(cherry picked from commit 2968331)
hknlof pushed a commit to hknlof/datafusion that referenced this pull request Aug 20, 2025
…ec of the correct size (apache#16995)

* apache#16994 Ensure CooperativeExec#maintains_input_order returns a Vec of the correct size

* apache#16994 Extend default ExecutionPlan invariant checks

Add checks that verify the length of the vectors returned by methods that need to return a value per child.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CooperativeExec incorrectly implements maintains_input_order

3 participants