From eee8ee699ddeeb0ba755fd54c8d82f5722c8137e Mon Sep 17 00:00:00 2001 From: Kent Yao Date: Mon, 9 Nov 2020 20:54:35 +0800 Subject: [PATCH 1/2] [SPARK-33397][DOC] Fix generating md to html for available-patterns-for-shs-custom-executor-log-url --- docs/running-on-yarn.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 5e8eb48093c8a..e1403f6b4eaa4 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -584,48 +584,48 @@ To use a custom metrics.properties for the application master and executors, upd - - + + - + - + - + - + - - + + - + - - + + - - + +
PatternMeaning
{{HTTP_SCHEME}}`http://` or `https://` according to YARN HTTP policy. (Configured via `yarn.http.policy`){{HTTP_SCHEME}}http:// or https:// according to YARN HTTP policy. (Configured via yarn.http.policy)
{{NM_HOST}}{{NM_HOST}} The "host" of node where container was run.
{{NM_PORT}}{{NM_PORT}} The "port" of node manager where container was run.
{{NM_HTTP_PORT}}{{NM_HTTP_PORT}} The "port" of node manager's http server where container was run.
{{NM_HTTP_ADDRESS}}{{NM_HTTP_ADDRESS}} Http URI of the node on which the container is allocated.
{{CLUSTER_ID}}The cluster ID of Resource Manager. (Configured via `yarn.resourcemanager.cluster-id`){{CLUSTER_ID}}The cluster ID of Resource Manager. (Configured via yarn.resourcemanager.cluster-id)
{{CONTAINER_ID}}{{CONTAINER_ID}} The ID of container.
{{USER}}'SPARK_USER' on system environment.{{USER}}SPARK_USER on system environment.
{{FILE_NAME}}`stdout`, `stderr`.{{FILE_NAME}}stdout, stderr.
For example, suppose you would like to point log url link to Job History Server directly instead of let NodeManager http server redirects it, you can configure `spark.history.custom.executor.log.url` as below: - `{{HTTP_SCHEME}}:/jobhistory/logs/{{NM_HOST}}:{{NM_PORT}}/{{CONTAINER_ID}}/{{CONTAINER_ID}}/{{USER}}/{{FILE_NAME}}?start=-4096` +{{HTTP_SCHEME}}:/jobhistory/logs/{{NM_HOST}}:{{NM_PORT}}/{{CONTAINER_ID}}/{{CONTAINER_ID}}/{{USER}}/{{FILE_NAME}}?start=-4096 - NOTE: you need to replace `` and `` with actual value. +NOTE: you need to replace `` and `` with actual value. # Resource Allocation and Configuration Overview From 078bae1a8bb879d7c37ecb44209eadef2642ff37 Mon Sep 17 00:00:00 2001 From: Kent Yao Date: Mon, 9 Nov 2020 21:11:05 +0800 Subject: [PATCH 2/2] fix --- docs/running-on-yarn.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index e1403f6b4eaa4..f19ce3de5e018 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -623,7 +623,7 @@ To use a custom metrics.properties for the application master and executors, upd For example, suppose you would like to point log url link to Job History Server directly instead of let NodeManager http server redirects it, you can configure `spark.history.custom.executor.log.url` as below: -{{HTTP_SCHEME}}:/jobhistory/logs/{{NM_HOST}}:{{NM_PORT}}/{{CONTAINER_ID}}/{{CONTAINER_ID}}/{{USER}}/{{FILE_NAME}}?start=-4096 +{{HTTP_SCHEME}}<JHS_HOST>:<JHS_PORT>/jobhistory/logs/{{NM_HOST}}:{{NM_PORT}}/{{CONTAINER_ID}}/{{CONTAINER_ID}}/{{USER}}/{{FILE_NAME}}?start=-4096 NOTE: you need to replace `` and `` with actual value.