Skip to content

Conversation

@HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Jul 24, 2018

What changes were proposed in this pull request?

It's minor and trivial but looks 2000 input is good enough to reproduce and test in SPARK-22499.

How was this patch tested?

Manually brought the change and tested.

Locally tested:

Before: 3m 21s 288ms
After: 1m 29s 134ms

Given the latest successful build took:

ArithmeticExpressionSuite:
- SPARK-22499: Least and greatest should not generate codes beyond 64KB (7 minutes, 49 seconds)

I expect it's going to save 4ish mins.

@HyukjinKwon HyukjinKwon changed the title [SPARK-22499][FOLLOWUP][SQL] Reduce input string expressions for Least and Greatest to reduce time [SPARK-22499][FOLLOWUP][SQL] Reduce input string expressions for Least and Greatest to reduce time in its test Jul 24, 2018
@SparkQA
Copy link

SparkQA commented Jul 24, 2018

Test build #93483 has finished for PR 21855 at commit 0efe041.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@kiszk
Copy link
Member

kiszk commented Jul 24, 2018

retest this please

@kiszk
Copy link
Member

kiszk commented Jul 24, 2018

LGTM.

With Spark 2.2, I have just confirmed the followings:

N=1500: passed
N=1800: exception
N=2000: exception

@HyukjinKwon
Copy link
Member Author

Thanks, @kiszk!

@SparkQA
Copy link

SparkQA commented Jul 24, 2018

Test build #93489 has finished for PR 21855 at commit 0efe041.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Jul 24, 2018

Merged to master, branch-2.3 and branch-2.2.

asfgit pushed a commit that referenced this pull request Jul 24, 2018
…t and Greatest to reduce time in its test

## What changes were proposed in this pull request?

It's minor and trivial but looks 2000 input is good enough to reproduce and test in SPARK-22499.

## How was this patch tested?

Manually brought the change and tested.

Locally tested:

Before: 3m 21s 288ms
After: 1m 29s 134ms

Given the latest successful build took:

```
ArithmeticExpressionSuite:
- SPARK-22499: Least and greatest should not generate codes beyond 64KB (7 minutes, 49 seconds)
```

I expect it's going to save 4ish mins.

Author: hyukjinkwon <[email protected]>

Closes #21855 from HyukjinKwon/minor-fix-suite.

(cherry picked from commit 3d5c61e)
Signed-off-by: hyukjinkwon <[email protected]>
asfgit pushed a commit that referenced this pull request Jul 24, 2018
…t and Greatest to reduce time in its test

## What changes were proposed in this pull request?

It's minor and trivial but looks 2000 input is good enough to reproduce and test in SPARK-22499.

## How was this patch tested?

Manually brought the change and tested.

Locally tested:

Before: 3m 21s 288ms
After: 1m 29s 134ms

Given the latest successful build took:

```
ArithmeticExpressionSuite:
- SPARK-22499: Least and greatest should not generate codes beyond 64KB (7 minutes, 49 seconds)
```

I expect it's going to save 4ish mins.

Author: hyukjinkwon <[email protected]>

Closes #21855 from HyukjinKwon/minor-fix-suite.

(cherry picked from commit 3d5c61e)
Signed-off-by: hyukjinkwon <[email protected]>
@asfgit asfgit closed this in 3d5c61e Jul 24, 2018
MatthewRBruce pushed a commit to Shopify/spark that referenced this pull request Jul 31, 2018
…t and Greatest to reduce time in its test

## What changes were proposed in this pull request?

It's minor and trivial but looks 2000 input is good enough to reproduce and test in SPARK-22499.

## How was this patch tested?

Manually brought the change and tested.

Locally tested:

Before: 3m 21s 288ms
After: 1m 29s 134ms

Given the latest successful build took:

```
ArithmeticExpressionSuite:
- SPARK-22499: Least and greatest should not generate codes beyond 64KB (7 minutes, 49 seconds)
```

I expect it's going to save 4ish mins.

Author: hyukjinkwon <[email protected]>

Closes apache#21855 from HyukjinKwon/minor-fix-suite.

(cherry picked from commit 3d5c61e)
Signed-off-by: hyukjinkwon <[email protected]>
@HyukjinKwon HyukjinKwon deleted the minor-fix-suite branch October 16, 2018 12:46
Willymontaz pushed a commit to criteo-forks/spark that referenced this pull request Sep 26, 2019
…t and Greatest to reduce time in its test

## What changes were proposed in this pull request?

It's minor and trivial but looks 2000 input is good enough to reproduce and test in SPARK-22499.

## How was this patch tested?

Manually brought the change and tested.

Locally tested:

Before: 3m 21s 288ms
After: 1m 29s 134ms

Given the latest successful build took:

```
ArithmeticExpressionSuite:
- SPARK-22499: Least and greatest should not generate codes beyond 64KB (7 minutes, 49 seconds)
```

I expect it's going to save 4ish mins.

Author: hyukjinkwon <[email protected]>

Closes apache#21855 from HyukjinKwon/minor-fix-suite.

(cherry picked from commit 3d5c61e)
Signed-off-by: hyukjinkwon <[email protected]>
Willymontaz pushed a commit to criteo-forks/spark that referenced this pull request Sep 27, 2019
…t and Greatest to reduce time in its test

## What changes were proposed in this pull request?

It's minor and trivial but looks 2000 input is good enough to reproduce and test in SPARK-22499.

## How was this patch tested?

Manually brought the change and tested.

Locally tested:

Before: 3m 21s 288ms
After: 1m 29s 134ms

Given the latest successful build took:

```
ArithmeticExpressionSuite:
- SPARK-22499: Least and greatest should not generate codes beyond 64KB (7 minutes, 49 seconds)
```

I expect it's going to save 4ish mins.

Author: hyukjinkwon <[email protected]>

Closes apache#21855 from HyukjinKwon/minor-fix-suite.

(cherry picked from commit 3d5c61e)
Signed-off-by: hyukjinkwon <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants