From 99b23365ceccb1a2dde4ac4dc500f9ff9aedb239 Mon Sep 17 00:00:00 2001 From: luogankun Date: Mon, 22 Dec 2014 16:01:31 +0800 Subject: [PATCH 1/3] [SPARK-4916]Update SQL programming guide --- docs/sql-programming-guide.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index 2aea8a8aedafc..710fe6fbd6497 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -835,8 +835,7 @@ Spark SQL can cache tables using an in-memory columnar format by calling `sqlCon Then Spark SQL will scan only required columns and will automatically tune compression to minimize memory usage and GC pressure. You can call `sqlContext.uncacheTable("tableName")` to remove the table from memory. -Note that if you call `schemaRDD.cache()` rather than `sqlContext.cacheTable(...)`, tables will _not_ be cached using -the in-memory columnar format, and therefore `sqlContext.cacheTable(...)` is strongly recommended for this use case. +Note that you call schemaRDD.cache() alike sqlContext.cacheTable(...) in 1.2 release of Spark, tables will be cached using the in-memory columnar format. Configuration of in-memory caching can be done using the `setConf` method on SQLContext or by running `SET key=value` commands using SQL. From 0b93785a3d68ccdc4825244eda439d441fab0672 Mon Sep 17 00:00:00 2001 From: luogankun Date: Mon, 22 Dec 2014 16:11:42 +0800 Subject: [PATCH 2/3] [SPARK-4916]Update SQL programming guide --- docs/sql-programming-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index 710fe6fbd6497..bfb008edb05af 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -835,7 +835,7 @@ Spark SQL can cache tables using an in-memory columnar format by calling `sqlCon Then Spark SQL will scan only required columns and will automatically tune compression to minimize memory usage and GC pressure. You can call `sqlContext.uncacheTable("tableName")` to remove the table from memory. -Note that you call schemaRDD.cache() alike sqlContext.cacheTable(...) in 1.2 release of Spark, tables will be cached using the in-memory columnar format. +Note that you call `schemaRDD.cache()` alike `sqlContext.cacheTable(...)` in 1.2 release of Spark, tables will be cached using the in-memory columnar format. Configuration of in-memory caching can be done using the `setConf` method on SQLContext or by running `SET key=value` commands using SQL. From 7b39864636135a5d97766a47259101c4694ed719 Mon Sep 17 00:00:00 2001 From: luogankun Date: Tue, 23 Dec 2014 15:14:38 +0800 Subject: [PATCH 3/3] [SPARK-4916]Update SQL programming guide --- docs/sql-programming-guide.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index bfb008edb05af..1b5fde991e405 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -831,12 +831,10 @@ turning on some experimental options. ## Caching Data In Memory -Spark SQL can cache tables using an in-memory columnar format by calling `sqlContext.cacheTable("tableName")`. +Spark SQL can cache tables using an in-memory columnar format by calling `sqlContext.cacheTable("tableName")` or `schemaRDD.cache()`. Then Spark SQL will scan only required columns and will automatically tune compression to minimize memory usage and GC pressure. You can call `sqlContext.uncacheTable("tableName")` to remove the table from memory. -Note that you call `schemaRDD.cache()` alike `sqlContext.cacheTable(...)` in 1.2 release of Spark, tables will be cached using the in-memory columnar format. - Configuration of in-memory caching can be done using the `setConf` method on SQLContext or by running `SET key=value` commands using SQL.