Skip to content

Conversation

@viirya
Copy link
Member

@viirya viirya commented Oct 13, 2015

JIRA: https://issues.apache.org/jira/browse/SPARK-11036

Related to #9093, SPARK-11036 is proposed to not allow AttributeReference to be created in executors. #9093 solves part of SPARK-11036. But the method withExprId will possibly assign new expression id to AttributeReference too. This patch tries to prevent that by using the same mechanism of #9093.

@SparkQA
Copy link

SparkQA commented Oct 13, 2015

Test build #43636 has finished for PR 9094 at commit 3df9212.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@davies
Copy link
Contributor

davies commented Oct 13, 2015

@viirya Since the newExprId is created in driver, I think it's OK to call withExprId in tasks. #9093 should be enough to fix SPARK-11036, I will close that as duplicated.

@viirya
Copy link
Member Author

viirya commented Oct 13, 2015

@davies Indeed. Looks like exprId given to withExprId is from other Attribute and is not created in tasks. I close this now.

@viirya viirya closed this Oct 13, 2015
@viirya viirya deleted the not-newexprid-on-executor branch December 27, 2023 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants