You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-29070][CORE] Make SparkLauncher log full spark-submit command line
Log the full spark-submit command in SparkSubmit#launchApplication
Adding .python-version (pyenv file) to RAT exclusion list
### What changes were proposed in this pull request?
Original motivation [here](http://apache-spark-user-list.1001560.n3.nabble.com/Is-it-possible-to-obtain-the-full-command-to-be-invoked-by-SparkLauncher-td35144.html), expanded in the [Jira](https://issues.apache.org/jira/browse/SPARK-29070).. In essence, we want to be able to log the full `spark-submit` command being constructed by `SparkLauncher`
### Why are the changes needed?
Currently, it is not possible to directly obtain this information from the `SparkLauncher` instance, which makes debugging and customer support more difficult.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
`core` `sbt` tests were executed. The `SparkLauncherSuite` (where I added assertions to an existing test) was also checked. Within that, `testSparkLauncherGetError` is failing, but that appears not to have been caused by this change (failing for me even on the parent commit of c18f849).
Closes#25777 from jeff303/SPARK-29070.
Authored-by: Jeff Evans <[email protected]>
Signed-off-by: Marcelo Vanzin <[email protected]>
0 commit comments