Skip to content

Conversation

@allisonwang-db
Copy link
Contributor

What changes were proposed in this pull request?

This PR combines the current plan and the initial plan in the AQE query plan string when the two plans are the same. It also removes the == Current Plan == and == Initial Plan == headers:

Before

AdaptiveSparkPlan isFinalPlan=false
+- == Current Plan ==
   SortMergeJoin [key#13], [a#23], Inner
   :- Sort [key#13 ASC NULLS FIRST], false, 0
   :  +- Exchange hashpartitioning(key#13, 5), true, [id=#94]
            ...
+- == Initial Plan ==
   SortMergeJoin [key#13], [a#23], Inner
   :- Sort [key#13 ASC NULLS FIRST], false, 0
   :  +- Exchange hashpartitioning(key#13, 5), true, [id=#94]
            ...

After

AdaptiveSparkPlan isFinalPlan=false
+- SortMergeJoin [key#13], [a#23], Inner
   :- Sort [key#13 ASC NULLS FIRST], false, 0
   :  +- Exchange hashpartitioning(key#13, 5), true, [id=#94]
            ...

For SQL EXPLAIN output:
Before

AdaptiveSparkPlan (8)
+- == Current Plan ==
   Sort (7)
   +- Exchange (6)
      ...
+- == Initial Plan ==
   Sort (7)
   +- Exchange (6)
      ...

After

AdaptiveSparkPlan (8)
+- Sort (7)
   +- Exchange (6)
      ...

Why are the changes needed?

To simplify the AQE plan string by removing the redundant plan information.

Does this PR introduce any user-facing change?

Yes.

How was this patch tested?

Modified the existing unit test.

@SparkQA
Copy link

SparkQA commented Sep 30, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33900/

@SparkQA
Copy link

SparkQA commented Sep 30, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33900/

@SparkQA
Copy link

SparkQA commented Oct 1, 2020

Test build #129284 has finished for PR 29915 at commit fe4873e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

@gatorsmile gatorsmile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gatorsmile gatorsmile closed this in 14aeab3 Oct 5, 2020
@gatorsmile
Copy link
Member

Thanks! Merged to master.

@allisonwang-db allisonwang-db deleted the aqe-explain branch January 19, 2024 01:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants