Skip to content

Commit 8b18397

Browse files
turboFeiHyukjinKwon
authored andcommitted
[SPARK-29542][FOLLOW-UP] Keep the description of spark.sql.files.* in tuning guide be consistent with that in SQLConf
### What changes were proposed in this pull request? This pr is a follow up of #26200. In this PR, I modify the description of spark.sql.files.* in sql-performance-tuning.md to keep consistent with that in SQLConf. ### Why are the changes needed? To keep consistent with the description in SQLConf. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Existed UT. Closes #27545 from turboFei/SPARK-29542-follow-up. Authored-by: turbofei <[email protected]> Signed-off-by: HyukjinKwon <[email protected]>
1 parent f5026b1 commit 8b18397

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

docs/sql-performance-tuning.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ that these options will be deprecated in future release as more optimizations ar
6767
<td>134217728 (128 MB)</td>
6868
<td>
6969
The maximum number of bytes to pack into a single partition when reading files.
70+
This configuration is effective only when using file-based sources such as Parquet, JSON and ORC.
7071
</td>
7172
</tr>
7273
<tr>
@@ -76,7 +77,8 @@ that these options will be deprecated in future release as more optimizations ar
7677
The estimated cost to open a file, measured by the number of bytes could be scanned in the same
7778
time. This is used when putting multiple files into a partition. It is better to over-estimated,
7879
then the partitions with small files will be faster than partitions with bigger files (which is
79-
scheduled first).
80+
scheduled first). This configuration is effective only when using file-based sources such as Parquet,
81+
JSON and ORC.
8082
</td>
8183
</tr>
8284
<tr>

0 commit comments

Comments
 (0)