Skip to content

[SPARK-29542][FOLLOW-UP] Keep the description of spark.sql.files.* in tuning guide be consistent with that in SQLConf. #27545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

[SPARK-29542][FOLLOW-UP] Keep the description of spark.sql.files.* in tuning guide be consistent with that in SQLConf. #27545

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
13a4758
[SPARK-29542][FOLLOW-UP] Keep the description of spark.sql.files.* co…
turboFei Feb 12, 2020
25b3feb
remove blank line
turboFei Feb 12, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/sql-performance-tuning.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ that these options will be deprecated in future release as more optimizations ar
<td>134217728 (128 MB)</td>
<td>
The maximum number of bytes to pack into a single partition when reading files.
This configuration is effective only when using file-based sources such as Parquet, JSON and ORC.
</td>
</tr>
<tr>
Expand All @@ -76,7 +77,8 @@ that these options will be deprecated in future release as more optimizations ar
The estimated cost to open a file, measured by the number of bytes could be scanned in the same
time. This is used when putting multiple files into a partition. It is better to over-estimated,
then the partitions with small files will be faster than partitions with bigger files (which is
scheduled first).
scheduled first). This configuration is effective only when using file-based sources such as Parquet,
JSON and ORC.
</td>
</tr>
<tr>
Expand Down