Skip to content

Commit 8df1da3

Browse files
marmbrusgatorsmile
authored andcommitted
[SPARK-22862] Docs on lazy elimination of columns missing from an encoder
This behavior has confused some users, so lets clarify it. Author: Michael Armbrust <[email protected]> Closes #20048 from marmbrus/datasetAsDocs.
1 parent 22e1849 commit 8df1da3

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -401,6 +401,10 @@ class Dataset[T] private[sql](
401401
* If the schema of the Dataset does not match the desired `U` type, you can use `select`
402402
* along with `alias` or `as` to rearrange or rename as required.
403403
*
404+
* Note that `as[]` only changes the view of the data that is passed into typed operations,
405+
* such as `map()`, and does not eagerly project away any columns that are not present in
406+
* the specified class.
407+
*
404408
* @group basic
405409
* @since 1.6.0
406410
*/

0 commit comments

Comments
 (0)