Skip to content

Conversation

@rxin
Copy link
Contributor

@rxin rxin commented Nov 24, 2015

This patch makes it consistent to use varargs in all DataFrameReader methods, including Parquet, JSON, text, and the generic load function.

Also added a few more API tests for the Java API.

@rxin rxin changed the title [SPARK-11967][SQL] Use varargs for multiple paths in DataFrameReader [SPARK-11967][SQL] Consistent use of varargs for multiple paths in DataFrameReader Nov 24, 2015
@rxin
Copy link
Contributor Author

rxin commented Nov 24, 2015

cc @koertkuipers since you did the change for load(Array) - I'm changing it to varargs here.

@SparkQA
Copy link

SparkQA commented Nov 24, 2015

Test build #46623 has finished for PR 9945 at commit 01897a8.

  • This patch fails PySpark unit tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 25, 2015

Test build #46638 has finished for PR 9945 at commit bd9a538.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * public final class SortedIterator extends UnsafeSorterIterator\n * case class GetStructField(child: Expression, ordinal: Int, name: Option[String] = None)\n * case class PrettyAttribute(name: String, dataType: DataType = NullType)\n

@SparkQA
Copy link

SparkQA commented Nov 25, 2015

Test build #46641 has finished for PR 9945 at commit f97193c.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 25, 2015

Test build #46642 has finished for PR 9945 at commit c824f24.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46641/
Test FAILed.

asfgit pushed a commit that referenced this pull request Nov 25, 2015
…taFrameReader

This patch makes it consistent to use varargs in all DataFrameReader methods, including Parquet, JSON, text, and the generic load function.

Also added a few more API tests for the Java API.

Author: Reynold Xin <[email protected]>

Closes #9945 from rxin/SPARK-11967.

(cherry picked from commit 25bbd3c)
Signed-off-by: Reynold Xin <[email protected]>
@asfgit asfgit closed this in 25bbd3c Nov 25, 2015
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no @Test here, is it by means?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh we should add it. can you add it in ur next pr?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants