Skip to content

Conversation

@HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Apr 2, 2019

What changes were proposed in this pull request?

This is a followup of #23285. This PR adds the notes into PySpark and SparkR documentation as well.

While I am here, I revised the doc a bit to make it sound a bit more neutral

How was this patch tested?

Manually built the doc and verified.

@HyukjinKwon HyukjinKwon changed the title [SPARK-26224][SQL][PYTHON][R] Add notes about many projects in withColumn at SparkR and PySpark as well [SPARK-26224][SQL][PYTHON][R][FOLLOW-UP] Add notes about many projects in withColumn at SparkR and PySpark as well Apr 2, 2019
@HyukjinKwon
Copy link
Member Author

cc @cloud-fan and @mgaido91

Copy link
Member

@felixcheung felixcheung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. is this also applicable to withColumnRenamed?

@SparkQA
Copy link

SparkQA commented Apr 2, 2019

Test build #104184 has finished for PR 24272 at commit 9f6c9ae.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member Author

retest this please

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Apr 2, 2019

The reported case was a usecase making sense with withColumn but I am not sure with withColumnRenamed ... Hope we can handle that differently later.

@mgaido91
Copy link
Contributor

mgaido91 commented Apr 2, 2019

LGTM, thanks @HyukjinKwon . I don't think withColumnRenamed is used by users in big for loops as I have seen for withColumn, so I think it is fine not to handle it. Thanks.

@SparkQA
Copy link

SparkQA commented Apr 2, 2019

Test build #104190 has finished for PR 24272 at commit 9f6c9ae.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member Author

retest this please

@SparkQA
Copy link

SparkQA commented Apr 2, 2019

Test build #104206 has finished for PR 24272 at commit 9f6c9ae.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member Author

Thanks, @felixcheung, @mgaido91, @srowen and @viirya.

Merged to master.

@HyukjinKwon HyukjinKwon deleted the SPARK-26224 branch March 3, 2020 01:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants