Skip to content

[SPARK-31476][SQL] Add an ExpressionInfo entry for EXTRACT #28251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

[SPARK-31476][SQL] Add an ExpressionInfo entry for EXTRACT #28251

wants to merge 2 commits into from

Conversation

@maropu
Copy link
Member

@maropu maropu commented Apr 18, 2020

What changes were proposed in this pull request?

This PR intends to add an ExpressionInfo entry for EXTRACT for better documentations.
This PR comes from the comment in #21479 (comment)

Why are the changes needed?

To make SQL documentations complete.

Does this PR introduce any user-facing change?

Yes, this PR updates the Spark SQL, Built-in Functions page.

How was this patch tested?

Run the example tests.

dongjoon-hyun
dongjoon-hyun approved these changes Apr 18, 2020
View reviewed changes
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM.

@SparkQA
Copy link

SparkQA commented Apr 18, 2020

Test build #121436 has finished for PR 28251 at commit 933c89f.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • case class Extract(field: Expression, source: Expression, child: Expression)

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Apr 18, 2020

Could you regenerate date.sql, too?

sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException: postgreSQL/date.sql
Expected "struct<[date_part('EPOCH',] DATE '1970-01-01'):...", but got "struct<[extract('EPOCH' ```

@maropu
Copy link
Member Author

maropu commented Apr 18, 2020

Ah, yes. I'm updating now. Thanks for the check, anyway.

@SparkQA
Copy link

SparkQA commented Apr 18, 2020

Test build #121444 has finished for PR 28251 at commit 5579736.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@maropu
Copy link
Member Author

maropu commented Apr 18, 2020

retest this please

@SparkQA
Copy link

SparkQA commented Apr 18, 2020

Test build #121446 has finished for PR 28251 at commit 5579736.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Apr 18, 2020

cc @gatorsmile

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Apr 18, 2020

Merged to master/3.0. Thank you, @maropu .

dongjoon-hyun pushed a commit that referenced this pull request Apr 18, 2020
@maropu @dongjoon-hyun
[SPARK-31476][SQL] Add an ExpressionInfo entry for EXTRACT
43b5116 
### What changes were proposed in this pull request?

This PR intends to add an ExpressionInfo entry for EXTRACT for better documentations.
This PR comes from the comment in #21479 (comment)

### Why are the changes needed?

To make SQL documentations complete.

### Does this PR introduce any user-facing change?

Yes, this PR updates the `Spark SQL, Built-in Functions` page.

### How was this patch tested?

Run the example tests.

Closes #28251 from maropu/AddExtractExpr.

Authored-by: Takeshi Yamamuro <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 74aed8c)
Signed-off-by: Dongjoon Hyun <[email protected]>
@maropu
Copy link
Member Author

maropu commented Apr 18, 2020

Thanks for the quick response, @dongjoon-hyun ! cc: @cloud-fan

cloud-fan
cloud-fan reviewed Apr 20, 2020
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
expression[MakeTimestamp]("make_timestamp"),
expression[MakeInterval]("make_interval"),
expression[DatePart]("date_part"),
expression[Extract]("extract"),
Copy link
Contributor

@cloud-fan cloud-fan Apr 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one side effect is now we support extract(field, source) other than extract(field from source). Not a big deal but better if we can avoid exposing more APIs,

Copy link
Member Author

@maropu maropu Apr 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a big deal but better if we can avoid exposing more APIs

Yea, +1 .

btw, its better to add tests for the case extract(field, source)?

Copy link
Contributor

@cloud-fan cloud-fan Apr 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea if we decide to support it.

Copy link
Member Author

@maropu maropu Apr 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I'll do follow-up later.

Copy link
Member Author

@maropu maropu Apr 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened #28276

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cloud-fan cloud-fan cloud-fan left review comments

@dongjoon-hyun dongjoon-hyun dongjoon-hyun approved these changes

Assignees

No one assigned

Labels

SQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@maropu @SparkQA @dongjoon-hyun @cloud-fan