Skip to content

Commit ab8177d

Browse files
LewuatheJoshRosen
authored andcommitted
[SPARK-4652][DOCS] Add docs about spark-git-repo option
There might be some cases when WIPS spark version need to be run on EC2 cluster. In order to setup this type of cluster more easily, add --spark-git-repo option description to ec2 documentation. Author: lewuathe <[email protected]> Author: Josh Rosen <[email protected]> Closes apache#3513 from Lewuathe/doc-for-development-spark-cluster and squashes the following commits: 6dae8ee [lewuathe] Wrap consistent with other descriptions cfaf9be [lewuathe] Add docs about spark-git-repo option (Editing / cleanup by Josh Rosen)
1 parent 743a889 commit ab8177d

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

docs/ec2-scripts.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,11 @@ another.
8585
specified version of Spark. The `<version>` can be a version number
8686
(e.g. "0.7.3") or a specific git hash. By default, a recent
8787
version will be used.
88+
- `--spark-git-repo=<repository url>` will let you run a custom version of
89+
Spark that is built from the given git repository. By default, the
90+
[Apache Github mirror](https://github.com/apache/spark) will be used.
91+
When using a custom Spark version, `--spark-version` must be set to git
92+
commit hash, such as 317e114, instead of a version number.
8893
- If one of your launches fails due to e.g. not having the right
8994
permissions on your private key file, you can run `launch` with the
9095
`--resume` option to restart the setup process on an existing cluster.

0 commit comments

Comments
 (0)