Skip to content

Commit 97ee85d

Browse files
shijinkuiJoshRosen
authored andcommitted
[SPARK-12953][EXAMPLES] RDDRelation writer set overwrite mode
https://issues.apache.org/jira/browse/SPARK-12953 fix error when run RDDRelation.main(): "path file:/Users/sjk/pair.parquet already exists" Set DataFrameWriter's mode to SaveMode.Overwrite Author: shijinkui <[email protected]> Closes #10864 from shijinkui/set_mode.
1 parent 1eac380 commit 97ee85d

File tree

1 file changed

+3
-4
lines changed

1 file changed

+3
-4
lines changed

examples/src/main/scala/org/apache/spark/examples/sql/RDDRelation.scala

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,7 @@
1919
package org.apache.spark.examples.sql
2020

2121
import org.apache.spark.{SparkConf, SparkContext}
22-
import org.apache.spark.sql.SQLContext
23-
import org.apache.spark.sql.functions._
22+
import org.apache.spark.sql.{SaveMode, SQLContext}
2423

2524
// One method for defining the schema of an RDD is to make a case class with the desired column
2625
// names and types.
@@ -58,8 +57,8 @@ object RDDRelation {
5857
// Queries can also be written using a LINQ-like Scala DSL.
5958
df.where($"key" === 1).orderBy($"value".asc).select($"key").collect().foreach(println)
6059

61-
// Write out an RDD as a parquet file.
62-
df.write.parquet("pair.parquet")
60+
// Write out an RDD as a parquet file with overwrite mode.
61+
df.write.mode(SaveMode.Overwrite).parquet("pair.parquet")
6362

6463
// Read in parquet file. Parquet files are self-describing so the schmema is preserved.
6564
val parquetFile = sqlContext.read.parquet("pair.parquet")

0 commit comments

Comments
 (0)