diff --git a/docs/configuration.md b/docs/configuration.md index 0c7c4472be643..0c1a6c9edf181 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -456,6 +456,40 @@ Apart from these, the following properties are also available, and may be useful from JVM to Python worker for every task. + + spark.python.task.killTimeout + 2s + + Timeout to wait before killing the python worker when a task cannot be interrupted + + + + spark.sql.repl.eagerEval.enabled + false + + Enable eager evaluation or not. If true and the REPL you are using supports eager evaluation, + Dataset will be ran automatically. The HTML table which generated by _repl_html_ + called by notebooks like Jupyter will feedback the queries user have defined. For plain Python + REPL, the output will be shown like dataframe.show() + (see SPARK-24215 for more details). + + + + spark.sql.repl.eagerEval.maxNumRows + 20 + + Default number of rows in eager evaluation output HTML table generated by _repr_html_ or plain text, + this only take effect when spark.sql.repl.eagerEval.enabled is set to true. + + + + spark.sql.repl.eagerEval.truncate + 20 + + Default number of truncate in eager evaluation output HTML table generated by _repr_html_ or + plain text, this only take effect when spark.sql.repl.eagerEval.enabled set to true. + + spark.files @@ -541,6 +575,13 @@ Apart from these, the following properties are also available, and may be useful Python binary executable to use for PySpark in both driver and executors. + + spark.worker.driverTerminateTimeout + 10s + + Timeout to wait for when trying to terminate a driver. + + ### Shuffle Behavior @@ -771,6 +812,13 @@ Apart from these, the following properties are also available, and may be useful Buffer size to use when writing to output streams, in KiB unless otherwise specified. + + spark.ui.consoleProgress.update.interval + 200 + + Update period of progress bar, in milliseconds + + spark.ui.enabled true