Skip to content

[SPARK-4889] update history server example cmds #3736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

[SPARK-4889] update history server example cmds #3736

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9ca7629
[SPARK-4889] update history server example cmds
ryan-williams Dec 18, 2014
a2d0f82
code review feedback
ryan-williams Dec 18, 2014
76d6a4c
remove hdfs example
ryan-williams Dec 18, 2014
421d8ff
add another random typo fix
ryan-williams Dec 18, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions conf/metrics.properties.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,8 @@
# sample false Whether to show entire set of samples for histograms ('false' or 'true')
#
# * Default path is /metrics/json for all instances except the master. The master has two paths:
# /metrics/aplications/json # App information
# /metrics/master/json # Master information
# /metrics/applications/json # App information
# /metrics/master/json # Master information

# org.apache.spark.metrics.sink.GraphiteSink
# Name: Default: Description:
Expand Down
9 changes: 5 additions & 4 deletions core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,11 +158,12 @@ class HistoryServer(

/**
* The recommended way of starting and stopping a HistoryServer is through the scripts
* start-history-server.sh and stop-history-server.sh. The path to a base log directory
* is must be specified, while the requested UI port is optional. For example:
* start-history-server.sh and stop-history-server.sh. The path to a base log directory,
* as well as any other relevant history server configuration, should be specified via
* the $SPARK_HISTORY_OPTS environment variable. For example:
*
* ./sbin/spark-history-server.sh /tmp/spark-events
* ./sbin/spark-history-server.sh hdfs://1.2.3.4:9000/spark-events
* export SPARK_HISTORY_OPTS="-Dspark.history.fs.logDirectory=/tmp/spark-events"
* ./sbin/start-history-server.sh
Copy link
Contributor

@andrewor14 andrewor14 Dec 18, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make this more terse

SPARK_HISTORY_OPTS=" -Dspark.history.fs.logDirectory=/tmp/spark-events" ./sbin/start-history-server.sh

I think it's OK to not show the hdfs path as well, since we document that elsewhere

Copy link
Contributor Author

@ryan-williams ryan-williams Dec 18, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, but

 *   SPARK_HISTORY_OPTS="-Dspark.history.fs.logDirectory=/tmp/spark-events" ./sbin/start-history-server.sh

is 107 chars long; preferences on wrapping?

*
* This launches the HistoryServer as a Spark daemon.
*/
Expand Down