Skip to content

Commit 63ebf1a

Browse files
committed
Merge pull request #103 from falaki/issue-79
Updated README with better R example and inferSchema flag
2 parents 7c6cce3 + cedd7c8 commit 63ebf1a

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ When reading files the API accepts several options:
4545
* `DROPMALFORMED`: drops lines which have fewer or more tokens than expected
4646
* `FAILFAST`: aborts with a RuntimeException if encounters any malformed line
4747
* `charset`: defaults to 'UTF-8' but can be set to other valid charset names
48+
* `inferSchema`: automatically infers column types. It requires one extra pass over the data and is false by default
4849

4950
The package also support saving simple (non-nested) DataFrame. When saving you can specify the delimiter and whether we should generate a header row for the table. See following examples for more details.
5051

@@ -136,6 +137,9 @@ df.select("year", "model").save("newcars.csv", "com.databricks.spark.csv")
136137
### R API
137138
Spark 1.4+:
138139
```R
140+
library(SparkR)
141+
142+
Sys.setenv('SPARKR_SUBMIT_ARGS'='"--packages" "com.databricks:spark-csv_2.10:1.1.0" "sparkr-shell"')
139143
sqlContext <- sparkRSQL.init(sc)
140144
df <- read.df(sqlContext, "cars.csv", source = "com.databricks.spark.csv")
141145

0 commit comments

Comments
 (0)