From f705e9bbe7f1e6a6393062c07e239b23ebf53ac8 Mon Sep 17 00:00:00 2001 From: Amit Dev Date: Fri, 12 Feb 2016 13:13:13 +0530 Subject: [PATCH 1/2] Added pygments.rb dependancy --- docs/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/README.md b/docs/README.md index 1f4fd3e56ed5f..53f2afef6fa0f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -15,6 +15,7 @@ Python and R. To get started you can run the following commands $ sudo gem install jekyll $ sudo gem install jekyll-redirect-from $ sudo pip install Pygments + $ sudo gem install pygments.rb $ sudo pip install sphinx $ Rscript -e 'install.packages(c("knitr", "devtools"), repos="http://cran.stat.ucla.edu/")' From 044b0d83f12b48fcf535141fb129dcf46d02b026 Mon Sep 17 00:00:00 2001 From: Amit Dev Date: Sun, 14 Feb 2016 13:00:26 +0530 Subject: [PATCH 2/2] [Documentation] Added more details about setup --- docs/README.md | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/docs/README.md b/docs/README.md index 53f2afef6fa0f..bcea93e1f3b6d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -10,16 +10,18 @@ whichever version of Spark you currently have checked out of revision control. ## Prerequisites The Spark documentation build uses a number of tools to build HTML docs and API docs in Scala, -Python and R. To get started you can run the following commands +Python and R. - $ sudo gem install jekyll - $ sudo gem install jekyll-redirect-from +You need to have [Ruby](https://www.ruby-lang.org/en/documentation/installation/) and +[Python](https://docs.python.org/2/using/unix.html#getting-and-installing-the-latest-version-of-python) +installed. Also install the following libraries: +```sh + $ sudo gem install jekyll jekyll-redirect-from pygments.rb $ sudo pip install Pygments - $ sudo gem install pygments.rb + # Following is needed only for generating API docs $ sudo pip install sphinx $ Rscript -e 'install.packages(c("knitr", "devtools"), repos="http://cran.stat.ucla.edu/")' - - +``` ## Generating the Documentation HTML We include the Spark documentation as part of the source (as opposed to using a hosted wiki, such as @@ -39,14 +41,16 @@ compiled files. $ jekyll build You can modify the default Jekyll build as follows: - +```sh # Skip generating API docs (which takes a while) $ SKIP_API=1 jekyll build + # Serve content locally on port 4000 $ jekyll serve --watch + # Build the site with extra features used on the live page $ PRODUCTION=1 jekyll build - +``` ## API Docs (Scaladoc, Sphinx, roxygen2) @@ -60,7 +64,7 @@ When you run `jekyll` in the `docs` directory, it will also copy over the scalad Spark subprojects into the `docs` directory (and then also into the `_site` directory). We use a jekyll plugin to run `build/sbt unidoc` before building the site so if you haven't run it (recently) it may take some time as it generates all of the scaladoc. The jekyll plugin also generates the -PySpark docs [Sphinx](http://sphinx-doc.org/). +PySpark docs using [Sphinx](http://sphinx-doc.org/). NOTE: To skip the step of building and copying over the Scala, Python, R API docs, run `SKIP_API=1 jekyll`.