Skip to content

[SPARK-2287] [SQL] Make ScalaReflection be able to handle Generic case classes. #1226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 5 commits into from
Closed

[SPARK-2287] [SQL] Make ScalaReflection be able to handle Generic case classes. #1226

wants to merge 5 commits into from

Conversation

@ueshin
Copy link
Member

@ueshin ueshin commented Jun 26, 2014

No description provided.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build started.

@ueshin
Copy link
Member Author

ueshin commented Jun 26, 2014

This will cause merge conflict with #1193, but I can fix it soon.

@ueshin
Merge branch 'master' into issues/SPARK-2287
d306e60 
Conflicts:
	sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
	sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build started.

@ueshin
Copy link
Member Author

ueshin commented Jun 26, 2014

Fixed merge conflict with #1193.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build finished.

1 similar comment
@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build finished.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16143/

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16140/

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build finished.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16147/

@ueshin
Copy link
Member Author

ueshin commented Jun 26, 2014

Could you please retest this?
Previous tests seemed like Hive metastore was something wrong.
(Can I let Jenkins do retest?)

@marmbrus
Copy link
Contributor

marmbrus commented Jun 26, 2014

test this please

@marmbrus
Copy link
Contributor

marmbrus commented Jun 26, 2014

I think Jenkins only listens to committers, but you could always just push a new commit (or the same commit with something like git commit --amend; git push <remote> +<branchname>)

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Merged build finished.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 26, 2014

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16165/

@ueshin
Copy link
Member Author

ueshin commented Jun 27, 2014

@marmbrus I see, I will do it next time, thanks.
But why failed...

@marmbrus
Copy link
Contributor

marmbrus commented Jun 27, 2014

Yeah, I don't know. Seems to work for me locally and you didn't touch anything hive related...

test this please

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 27, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 27, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 27, 2014

Merged build finished.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 27, 2014

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16203/

@ueshin
Copy link
Member Author

ueshin commented Jun 27, 2014

Passed Hive tests. Why? Just merged master.
And Python tests failed...

@marmbrus
Copy link
Contributor

marmbrus commented Jun 27, 2014

test this please.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 27, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 27, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 27, 2014

Merged build finished.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 27, 2014

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16210/

@ueshin
Copy link
Member Author

ueshin commented Jun 28, 2014

Hi @marmbrus,

I found that there is a case at my local throwing the same exception with test-only like:

testOnly org.apache.spark.sql.hive.execution.PruningSuite

even if my patch is not applied.

And I added a line to PruningSuite

+  TestHive.loadTestTable("src")
  TestHive.reset()

then it worked.

Would it be a help for you?

@ueshin
Copy link
Member Author

ueshin commented Jun 28, 2014

Ah, we must call SHOW TABLES before TestHive.reset() the same as here, right?

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 28, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 28, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 28, 2014

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

AmplabJenkins commented Jun 28, 2014

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16225/

@ueshin ueshin mentioned this pull request Jun 30, 2014
Closed
@asfgit asfgit closed this in bc7041a Jul 2, 2014
asfgit pushed a commit that referenced this pull request Jul 2, 2014
@ueshin @marmbrus
[SPARK-2287] [SQL] Make ScalaReflection be able to handle Generic cas…
cf0d14b 
…e classes.

Author: Takuya UESHIN <[email protected]>

Closes #1226 from ueshin/issues/SPARK-2287 and squashes the following commits:

32ef7c3 [Takuya UESHIN] Add execution of `SHOW TABLES` before `TestHive.reset()`.
541dc8d [Takuya UESHIN] Merge branch 'master' into issues/SPARK-2287
fac5fae [Takuya UESHIN] Remove unnecessary method receiver.
d306e60 [Takuya UESHIN] Merge branch 'master' into issues/SPARK-2287
7de5706 [Takuya UESHIN] Make ScalaReflection be able to handle Generic case classes.

(cherry picked from commit bc7041a)
Signed-off-by: Michael Armbrust <[email protected]>
asfgit pushed a commit that referenced this pull request Jul 2, 2014
@ueshin @marmbrus
[SPARK-2287] [SQL] Make ScalaReflection be able to handle Generic cas…
87b74a9 
…e classes.

Author: Takuya UESHIN <[email protected]>

Closes #1226 from ueshin/issues/SPARK-2287 and squashes the following commits:

32ef7c3 [Takuya UESHIN] Add execution of `SHOW TABLES` before `TestHive.reset()`.
541dc8d [Takuya UESHIN] Merge branch 'master' into issues/SPARK-2287
fac5fae [Takuya UESHIN] Remove unnecessary method receiver.
d306e60 [Takuya UESHIN] Merge branch 'master' into issues/SPARK-2287
7de5706 [Takuya UESHIN] Make ScalaReflection be able to handle Generic case classes.

(cherry picked from commit bc7041a)
Signed-off-by: Michael Armbrust <[email protected]>
@marmbrus
Copy link
Contributor

marmbrus commented Jul 2, 2014

Thanks! I've merged this into master and the 1.0 branches.

xiliu82 pushed a commit to xiliu82/spark that referenced this pull request Sep 4, 2014
@ueshin @conviva-zz
[SPARK-2287] [SQL] Make ScalaReflection be able to handle Generic cas…
3ae19a1 
…e classes.

Author: Takuya UESHIN <[email protected]>

Closes apache#1226 from ueshin/issues/SPARK-2287 and squashes the following commits:

32ef7c3 [Takuya UESHIN] Add execution of `SHOW TABLES` before `TestHive.reset()`.
541dc8d [Takuya UESHIN] Merge branch 'master' into issues/SPARK-2287
fac5fae [Takuya UESHIN] Remove unnecessary method receiver.
d306e60 [Takuya UESHIN] Merge branch 'master' into issues/SPARK-2287
7de5706 [Takuya UESHIN] Make ScalaReflection be able to handle Generic case classes.
@schlosna schlosna mentioned this pull request Feb 8, 2018
Closed
sunchao pushed a commit to sunchao/spark that referenced this pull request Dec 8, 2021
@aokolnychyi
rdar://84031596 Add Migrate and Snapshot Catalog Support (apache#1226)
aff93bb 
### What changes were proposed in this pull request?

This PR cherry-picks the code for SNAPSHOT and MIGRATE commands to Spark 3.2.

### Why are the changes needed?

A catalog can now implement MigrateSupport and SnapshotSupport
which are mapped to SQL that matches the ADT grammar
used previously.

### Does this PR introduce _any_ user-facing change?

Yes but the changes are isolated.

### How was this patch tested?

This PR comes with tests. More tests in Iceberg.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ueshin @AmplabJenkins @marmbrus