Skip to content

Conversation

@shivaram
Copy link
Contributor

What changes were proposed in this pull request?

When we build the docs separately we don't have the JAR files from the Spark build in
the same tree. As the SparkR vignettes need to launch a SparkContext to be built, we skip building them if JAR files don't exist

How was this patch tested?

To test this we can run the following:

build/mvn -DskipTests -Psparkr clean
./R/create-docs.sh

You should see a line Skipping R vignettes as Spark JARs not found at the end

@shivaram
Copy link
Contributor Author

The only downside of this is that we won't be uploading our vignette to the website. I think this is fine -- Any thoughts @junyangq @felixcheung

cc @rxin

@SparkQA
Copy link

SparkQA commented Sep 22, 2016

Test build #65778 has finished for PR 15200 at commit eb7e8b8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor

rxin commented Sep 22, 2016

I'm going to merge this in master/branch-2.0 because I need this to cut an rc.

@asfgit asfgit closed this in 9f24a17 Sep 22, 2016
asfgit pushed a commit that referenced this pull request Sep 22, 2016
## What changes were proposed in this pull request?

When we build the docs separately we don't have the JAR files from the Spark build in
the same tree. As the SparkR vignettes need to launch a SparkContext to be built, we skip building them if JAR files don't exist

## How was this patch tested?

To test this we can run the following:
```
build/mvn -DskipTests -Psparkr clean
./R/create-docs.sh
```
You should see a line `Skipping R vignettes as Spark JARs not found` at the end

Author: Shivaram Venkataraman <[email protected]>

Closes #15200 from shivaram/sparkr-vignette-skip.

(cherry picked from commit 9f24a17)
Signed-off-by: Reynold Xin <[email protected]>
@felixcheung
Copy link
Member

we could hand write the result instead of running test/Spark to generate the vignettes?
It seems that could be problematic if output are getting out of sync - and similar problem if we build doc without jar and then just skip the vignettes?

maybe we should add vignettes to profile -Psparkr in Maven?

@shivaram
Copy link
Contributor Author

Yeah - so I'm thinking we should just auto-generate this and check in the html file in git. Its not that big. When somebody updates the vignette we need to remind them to regenerate it though as a part of the PR ?

@felixcheung
Copy link
Member

if it's part of the -Psparkr profile of the build it will be regenerated by default. If it's changed and not in .gitignore it should be flagged for commit..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants