Skip to content

Conversation

@HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Nov 4, 2016

What changes were proposed in this pull request?

This PR proposes to improve documentation and fix some descriptions equivalent to several minor fixes identified in #15677

Also, this suggests to change Note: and NOTE: to .. note:: consistently with the others which marks up pretty.

How was this patch tested?

Jenkins tests and manually.

For PySpark, Note: and NOTE: to .. note:: make the document as below:

From

2016-11-04 6 53 35
2016-11-04 6 53 45
2016-11-04 6 54 11
2016-11-04 6 53 51
2016-11-04 6 53 51

To

2016-11-04 7 03 48
2016-11-04 7 04 03
2016-11-04 7 04 28
2016-11-04 7 33 39
2016-11-04 7 33 39

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Nov 4, 2016

It seems something wrongly gone in my local so I can't run cran-check for SparkR in my local. Please let me run this via Jenkins.

@HyukjinKwon
Copy link
Member Author

Please let me cc @felixcheung and @srowen though if you don't mind.

@HyukjinKwon
Copy link
Member Author

Oh, actually, it seems I should fix the ones in functions.scala too.

@SparkQA
Copy link

SparkQA commented Nov 4, 2016

Test build #68124 has finished for PR 15765 at commit c8fafbe.

  • This patch fails Python style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

def randn(): Column = randn(Utils.random.nextLong)

/**
* Partition ID of the Spark task.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one was missed in 02f2031 and a08463b

@HyukjinKwon HyukjinKwon changed the title [MINOR][Documentation][PySpark][SparkR] Fix minor descriptions in functions for SparkR and PySpark [MINOR][Documentation] Fix minor descriptions in functions for SparkR and PySpark Nov 4, 2016
@HyukjinKwon HyukjinKwon changed the title [MINOR][Documentation] Fix minor descriptions in functions for SparkR and PySpark [MINOR][Documentation] Fix some minor descriptions in functions consistently with expressions Nov 4, 2016
@SparkQA
Copy link

SparkQA commented Nov 4, 2016

Test build #68125 has finished for PR 15765 at commit 4d8e829.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Nov 4, 2016

It seems the newly added test seems too flaky. the specific test seems often failed. I saw this failure in series randomized aggregation test - [typed, with partial + safe] - with grouping keys - with non-empty input cc @liancheng

[1]https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68125/consoleFull
[2]https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68112/consoleFull
[3]https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68119/consoleFull
[4]https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68121/consoleFull

@HyukjinKwon
Copy link
Member Author

retest this please

@SparkQA
Copy link

SparkQA commented Nov 4, 2016

Test build #68126 has finished for PR 15765 at commit 9b5b8eb.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 4, 2016

Test build #68128 has finished for PR 15765 at commit 9b5b8eb.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 4, 2016

Test build #3414 has finished for PR 15765 at commit 9b5b8eb.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@felixcheung
Copy link
Member

changes LGTM. thanks.
to nit pick, I find this "(Signed) shift the given value numBits right." confusing. Perhaps we should clarify this across all doc?

@srowen
Copy link
Member

srowen commented Nov 4, 2016

How would you write it? we do need to specify that it's a signed shift.

@felixcheung
Copy link
Member

hmm, like "shift the signed value numBits to the right"?

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Nov 5, 2016

Ah, IMHO, it seems both are fine but actually it seems there is an unsigned shift one which is described as..

Unsigned shift the given value numBits right.

So.. if any of you does not feel strongly which one is better, I hope I could leave as it is because it seems at least consistent.

(Signed) shift the given value numBits right.

@felixcheung
Copy link
Member

I think both sound odd, but it's a nit

@srowen
Copy link
Member

srowen commented Nov 5, 2016

It's not a shift of a signed value, but a 'signed' shift, meaning one that preserves the sign bit. I recognize the terms "unsigned shift" and "signed shift", FWIW.

asfgit pushed a commit that referenced this pull request Nov 6, 2016
…stently with expressions

## What changes were proposed in this pull request?

This PR proposes to improve documentation and fix some descriptions equivalent to several minor fixes identified in #15677

Also, this suggests to change `Note:` and `NOTE:` to `.. note::` consistently with the others which marks up pretty.

## How was this patch tested?

Jenkins tests and manually.

For PySpark, `Note:` and `NOTE:` to `.. note::` make the document as below:

**From**

![2016-11-04 6 53 35](https://cloud.githubusercontent.com/assets/6477701/20002648/42989922-a2c5-11e6-8a32-b73eda49e8c3.png)
![2016-11-04 6 53 45](https://cloud.githubusercontent.com/assets/6477701/20002650/429fb310-a2c5-11e6-926b-e030d7eb0185.png)
![2016-11-04 6 54 11](https://cloud.githubusercontent.com/assets/6477701/20002649/429d570a-a2c5-11e6-9e7e-44090f337e32.png)
![2016-11-04 6 53 51](https://cloud.githubusercontent.com/assets/6477701/20002647/4297fc74-a2c5-11e6-801a-b89fbcbfca44.png)
![2016-11-04 6 53 51](https://cloud.githubusercontent.com/assets/6477701/20002697/749f5780-a2c5-11e6-835f-022e1f2f82e3.png)

**To**

![2016-11-04 7 03 48](https://cloud.githubusercontent.com/assets/6477701/20002659/4961b504-a2c5-11e6-9ee0-ef0751482f47.png)
![2016-11-04 7 04 03](https://cloud.githubusercontent.com/assets/6477701/20002660/49871d3a-a2c5-11e6-85ea-d9a5d11efeff.png)
![2016-11-04 7 04 28](https://cloud.githubusercontent.com/assets/6477701/20002662/498e0f14-a2c5-11e6-803d-c0c5aeda4153.png)
![2016-11-04 7 33 39](https://cloud.githubusercontent.com/assets/6477701/20002731/a76e30d2-a2c5-11e6-993b-0481b8342d6b.png)
![2016-11-04 7 33 39](https://cloud.githubusercontent.com/assets/6477701/20002731/a76e30d2-a2c5-11e6-993b-0481b8342d6b.png)

Author: hyukjinkwon <[email protected]>

Closes #15765 from HyukjinKwon/minor-function-doc.

(cherry picked from commit 15d3926)
Signed-off-by: Felix Cheung <[email protected]>
@felixcheung
Copy link
Member

Sure then.
Merged to master/branch-2.1

@asfgit asfgit closed this in 15d3926 Nov 6, 2016
@HyukjinKwon
Copy link
Member Author

Thank you both!

uzadude pushed a commit to uzadude/spark that referenced this pull request Jan 27, 2017
…stently with expressions

## What changes were proposed in this pull request?

This PR proposes to improve documentation and fix some descriptions equivalent to several minor fixes identified in apache#15677

Also, this suggests to change `Note:` and `NOTE:` to `.. note::` consistently with the others which marks up pretty.

## How was this patch tested?

Jenkins tests and manually.

For PySpark, `Note:` and `NOTE:` to `.. note::` make the document as below:

**From**

![2016-11-04 6 53 35](https://cloud.githubusercontent.com/assets/6477701/20002648/42989922-a2c5-11e6-8a32-b73eda49e8c3.png)
![2016-11-04 6 53 45](https://cloud.githubusercontent.com/assets/6477701/20002650/429fb310-a2c5-11e6-926b-e030d7eb0185.png)
![2016-11-04 6 54 11](https://cloud.githubusercontent.com/assets/6477701/20002649/429d570a-a2c5-11e6-9e7e-44090f337e32.png)
![2016-11-04 6 53 51](https://cloud.githubusercontent.com/assets/6477701/20002647/4297fc74-a2c5-11e6-801a-b89fbcbfca44.png)
![2016-11-04 6 53 51](https://cloud.githubusercontent.com/assets/6477701/20002697/749f5780-a2c5-11e6-835f-022e1f2f82e3.png)

**To**

![2016-11-04 7 03 48](https://cloud.githubusercontent.com/assets/6477701/20002659/4961b504-a2c5-11e6-9ee0-ef0751482f47.png)
![2016-11-04 7 04 03](https://cloud.githubusercontent.com/assets/6477701/20002660/49871d3a-a2c5-11e6-85ea-d9a5d11efeff.png)
![2016-11-04 7 04 28](https://cloud.githubusercontent.com/assets/6477701/20002662/498e0f14-a2c5-11e6-803d-c0c5aeda4153.png)
![2016-11-04 7 33 39](https://cloud.githubusercontent.com/assets/6477701/20002731/a76e30d2-a2c5-11e6-993b-0481b8342d6b.png)
![2016-11-04 7 33 39](https://cloud.githubusercontent.com/assets/6477701/20002731/a76e30d2-a2c5-11e6-993b-0481b8342d6b.png)

Author: hyukjinkwon <[email protected]>

Closes apache#15765 from HyukjinKwon/minor-function-doc.
@HyukjinKwon HyukjinKwon deleted the minor-function-doc branch January 2, 2018 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants