Skip to content

Commit f14f47f

Browse files
shivaramrxin
authored andcommitted
Skip building R vignettes if Spark is not built
## What changes were proposed in this pull request? When we build the docs separately we don't have the JAR files from the Spark build in the same tree. As the SparkR vignettes need to launch a SparkContext to be built, we skip building them if JAR files don't exist ## How was this patch tested? To test this we can run the following: ``` build/mvn -DskipTests -Psparkr clean ./R/create-docs.sh ``` You should see a line `Skipping R vignettes as Spark JARs not found` at the end Author: Shivaram Venkataraman <[email protected]> Closes #15200 from shivaram/sparkr-vignette-skip. (cherry picked from commit 9f24a17) Signed-off-by: Reynold Xin <[email protected]>
1 parent b25a8e6 commit f14f47f

File tree

1 file changed

+22
-3
lines changed

1 file changed

+22
-3
lines changed

R/create-docs.sh

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,13 @@ set -e
3030

3131
# Figure out where the script is
3232
export FWDIR="$(cd "`dirname "$0"`"; pwd)"
33+
export SPARK_HOME="$(cd "`dirname "$0"`"/..; pwd)"
34+
35+
# Required for setting SPARK_SCALA_VERSION
36+
. "${SPARK_HOME}"/bin/load-spark-env.sh
37+
38+
echo "Using Scala $SPARK_SCALA_VERSION"
39+
3340
pushd $FWDIR
3441

3542
# Install the package (this will also generate the Rd files)
@@ -45,9 +52,21 @@ Rscript -e 'libDir <- "../../lib"; library(SparkR, lib.loc=libDir); library(knit
4552

4653
popd
4754

48-
# render creates SparkR vignettes
49-
Rscript -e 'library(rmarkdown); paths <- .libPaths(); .libPaths(c("lib", paths)); Sys.setenv(SPARK_HOME=tools::file_path_as_absolute("..")); render("pkg/vignettes/sparkr-vignettes.Rmd"); .libPaths(paths)'
55+
# Find Spark jars.
56+
if [ -f "${SPARK_HOME}/RELEASE" ]; then
57+
SPARK_JARS_DIR="${SPARK_HOME}/jars"
58+
else
59+
SPARK_JARS_DIR="${SPARK_HOME}/assembly/target/scala-$SPARK_SCALA_VERSION/jars"
60+
fi
61+
62+
# Only create vignettes if Spark JARs exist
63+
if [ -d "$SPARK_JARS_DIR" ]; then
64+
# render creates SparkR vignettes
65+
Rscript -e 'library(rmarkdown); paths <- .libPaths(); .libPaths(c("lib", paths)); Sys.setenv(SPARK_HOME=tools::file_path_as_absolute("..")); render("pkg/vignettes/sparkr-vignettes.Rmd"); .libPaths(paths)'
5066

51-
find pkg/vignettes/. -not -name '.' -not -name '*.Rmd' -not -name '*.md' -not -name '*.pdf' -not -name '*.html' -delete
67+
find pkg/vignettes/. -not -name '.' -not -name '*.Rmd' -not -name '*.md' -not -name '*.pdf' -not -name '*.html' -delete
68+
else
69+
echo "Skipping R vignettes as Spark JARs not found in $SPARK_HOME"
70+
fi
5271

5372
popd

0 commit comments

Comments
 (0)