Skip to content

Conversation

@rxin
Copy link
Contributor

@rxin rxin commented Nov 11, 2015

If it returns Text, we can reuse this in Spark SQL to provide a WholeTextFile data source and directly convert the Text into UTF8String without extra string decoding and encoding.

@SparkQA
Copy link

SparkQA commented Nov 11, 2015

Test build #45610 has finished for PR 9622 at commit 4d412dd.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * public class JavaLBFGSExample\n * class LDA @Since(\"1.6.0\") (\n * case class Metadata(\n * require(className == expectedClassName, s\"Error loading metadata: Expected class name\" +\n * class SlidingRDDPartition[T](val idx: Int, val prev: Partition, val tail: Seq[T], val offset: Int)\n * class SlidingRDD[T: ClassTag](@transient val parent: RDD[T], val windowSize: Int, val step: Int)\n

@JoshRosen
Copy link
Contributor

Jenkins, retest this please.

@srowen
Copy link
Member

srowen commented Nov 11, 2015

LGTM since it's internal

@SparkQA
Copy link

SparkQA commented Nov 11, 2015

Test build #45617 has finished for PR 9622 at commit 4d412dd.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@davies
Copy link
Contributor

davies commented Nov 11, 2015

Merging this into master and 1.6.

@asfgit asfgit closed this in 95daff6 Nov 11, 2015
asfgit pushed a commit that referenced this pull request Nov 11, 2015
If it returns Text, we can reuse this in Spark SQL to provide a WholeTextFile data source and directly convert the Text into UTF8String without extra string decoding and encoding.

Author: Reynold Xin <[email protected]>

Closes #9622 from rxin/SPARK-11646.

(cherry picked from commit 95daff6)
Signed-off-by: Reynold Xin <[email protected]>
@rxin
Copy link
Contributor Author

rxin commented Nov 11, 2015

(racing. I just merge this too)

dskrvk pushed a commit to dskrvk/spark that referenced this pull request Nov 13, 2015
If it returns Text, we can reuse this in Spark SQL to provide a WholeTextFile data source and directly convert the Text into UTF8String without extra string decoding and encoding.

Author: Reynold Xin <[email protected]>

Closes apache#9622 from rxin/SPARK-11646.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants