From eb3d6cbf3f96f1457062cd526033d1ebbde17525 Mon Sep 17 00:00:00 2001 From: Gabor Somogyi Date: Mon, 4 Mar 2019 14:16:21 +0100 Subject: [PATCH] [SPARK-27046][DSTREAMS] Remove SPARK-19185 related references from documentation --- docs/streaming-kafka-0-10-integration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/streaming-kafka-0-10-integration.md b/docs/streaming-kafka-0-10-integration.md index c78459cd27d1..975adacca2f1 100644 --- a/docs/streaming-kafka-0-10-integration.md +++ b/docs/streaming-kafka-0-10-integration.md @@ -96,7 +96,7 @@ In most cases, you should use `LocationStrategies.PreferConsistent` as shown abo The cache for consumers has a default maximum size of 64. If you expect to be handling more than (64 * number of executors) Kafka partitions, you can change this setting via `spark.streaming.kafka.consumer.cache.maxCapacity`. -If you would like to disable the caching for Kafka consumers, you can set `spark.streaming.kafka.consumer.cache.enabled` to `false`. Disabling the cache may be needed to workaround the problem described in SPARK-19185. This property may be removed in later versions of Spark, once SPARK-19185 is resolved. +If you would like to disable the caching for Kafka consumers, you can set `spark.streaming.kafka.consumer.cache.enabled` to `false`. The cache is keyed by topicpartition and group.id, so use a **separate** `group.id` for each call to `createDirectStream`.