Skip to content

Remove extra semicolon in import statement and unused import in ApplicationMaster #358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

Remove extra semicolon in import statement and unused import in ApplicationMaster #358

wants to merge 1 commit into from

Conversation

@hsaputra
Copy link
Contributor

@hsaputra hsaputra commented Apr 8, 2014

Small nit cleanup to remove extra semicolon and unused import in Yarn's stable ApplicationMaster (it bothers me every time I saw it)

@rxin
Copy link
Contributor

rxin commented Apr 8, 2014

bothers me too. will merge once jenkins is happy.

@AmplabJenkins
Copy link

AmplabJenkins commented Apr 8, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Apr 8, 2014

Merged build started.

@srowen
Copy link
Member

srowen commented Apr 8, 2014

Meta question - all of the unused imports can be zapped in one go with an IDE and semicolons too. Is that worth one big PR here?

@rxin
Copy link
Contributor

rxin commented Apr 8, 2014

It might be worth it. But be careful sometimes IDE gives you false signals. Also for the imports to conform to Spark's coding standard, we need @aarondav's import organizer plugin for IntelliJ.

@hsaputra
Copy link
Contributor Author

hsaputra commented Apr 8, 2014

@srowen there has been PR (I think I sent it) to remove unnecessary semicolons and imports in Scala before and AFAIK we have removed most of them.

Probably the best way to have some kind of check executed by Jenkins in addition to style check.

@AmplabJenkins
Copy link

AmplabJenkins commented Apr 8, 2014

Merged build finished.

@AmplabJenkins
Copy link

AmplabJenkins commented Apr 8, 2014

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13882/

@hsaputra
Copy link
Contributor Author

hsaputra commented Apr 8, 2014

Seems like Jenkins had build error related to some PySpark file parsing.
Can we try it again?

@rxin
Copy link
Contributor

rxin commented Apr 8, 2014

Jenkins, retest this please.

@AmplabJenkins
Copy link

AmplabJenkins commented Apr 8, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Apr 8, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented Apr 8, 2014

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

AmplabJenkins commented Apr 8, 2014

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13908/

@rxin
Copy link
Contributor

rxin commented Apr 8, 2014

ok i've merged this. thanks

@asfgit asfgit closed this in 3bc0548 Apr 8, 2014
@hsaputra
Copy link
Contributor Author

hsaputra commented Apr 8, 2014

Thx!

pdeyhim pushed a commit to pdeyhim/spark-1 that referenced this pull request Jun 25, 2014
@hsaputra @pdeyhim
Remove extra semicolon in import statement and unused import in Appli…
a34d958 
…cationMaster

Small nit cleanup to remove extra semicolon and unused import in Yarn's stable ApplicationMaster (it bothers me every time I saw it)

Author: Henry Saputra <[email protected]>

Closes apache#358 from hsaputra/nitcleanup_removesemicolon_import_applicationmaster and squashes the following commits:

bffb685 [Henry Saputra] Remove extra semicolon in import statement and unused import in ApplicationMaster.scala
mccheah pushed a commit to mccheah/spark that referenced this pull request Oct 3, 2018
@robert3005
Merge pull request apache#358 from palantir/rk/more-upstream
5ccc040
bzhaoopenstack pushed a commit to bzhaoopenstack/spark that referenced this pull request Sep 11, 2019
@liusheng @kiwik
Fix manageiq job against Mitaka devstack (apache#358)
2506b02 
* Fix manageiq job against Mitaka devstack
fishcus added a commit to fishcus/spark that referenced this pull request Jan 10, 2022
@fishcus
KY-895, Spark driver pod cleanup when application is completed (apach…
1d41f03 
…e#358)

* KY-895, Spark driver pod cleanup when application is completed

* KY-895, release 3.1.1-kylin-4.x-r42-xuanwu
RolatZhang pushed a commit to RolatZhang/spark that referenced this pull request Apr 10, 2023
@frearb @fishcus @RolatZhang
KY-895 Spark driver pod cleanup when application is completed (apache…
e9b02ff 
…#528)

* KY-895, Spark driver pod cleanup when application is completed (apache#358)

* KY-895, Spark driver pod cleanup when application is completed

* KY-895, release 3.1.1-kylin-4.x-r42-xuanwu

* KY-895, add config KUBERNETES_DELETE_DRIVER (apache#376)

Co-authored-by: Feng Zhu <[email protected]>
RolatZhang added a commit to RolatZhang/spark that referenced this pull request Aug 18, 2023
@RolatZhang @leejaywei
@frearb @fishcus
cherry-pick 3.3 commits (apache#616)
cbbea03 
* KE-39980 configMap Binds the KE pod

* minor fix dockerfile java version and host user (apache#510)

* KY-895 Spark driver pod cleanup when application is completed (apache#528)

* KY-895, Spark driver pod cleanup when application is completed (apache#358)

* KY-895, Spark driver pod cleanup when application is completed

* KY-895, release 3.1.1-kylin-4.x-r42-xuanwu

* KY-895, add config KUBERNETES_DELETE_DRIVER (apache#376)

Co-authored-by: Feng Zhu <[email protected]>

---------

Co-authored-by: Jiawei Li <[email protected]>
Co-authored-by: Zhiting Guo <[email protected]>
Co-authored-by: Feng Zhu <[email protected]>
turboFei pushed a commit to turboFei/spark that referenced this pull request Nov 6, 2025
@wangyum @kazuyukitanimura
[SPARK-45786][SQL] Fix inaccurate Decimal multiplication and division…
94f10e1 
… results (apache#358)

* [SPARK-45786][SQL] Fix inaccurate Decimal multiplication and division results

### What changes were proposed in this pull request?
This PR fixes inaccurate Decimal multiplication and division results.

### Why are the changes needed?
Decimal multiplication and division results may be inaccurate due to rounding issues.
#### Multiplication:
```
scala> sql("select  -14120025096157587712113961295153.858047 * -0.4652").show(truncate=false)
+----------------------------------------------------+
|(-14120025096157587712113961295153.858047 * -0.4652)|
+----------------------------------------------------+
|6568635674732509803675414794505.574764              |
+----------------------------------------------------+
```
The correct answer is `6568635674732509803675414794505.574763`

Please note that the last digit is `3` instead of `4` as

```
scala> java.math.BigDecimal("-14120025096157587712113961295153.858047").multiply(java.math.BigDecimal("-0.4652"))
val res21: java.math.BigDecimal = 6568635674732509803675414794505.5747634644
```
Since the factional part `.574763` is followed by `4644`, it should not be rounded up.

#### Division:
```
scala> sql("select -0.172787979 / 533704665545018957788294905796.5").show(truncate=false)
+-------------------------------------------------+
|(-0.172787979 / 533704665545018957788294905796.5)|
+-------------------------------------------------+
|-3.237521E-31                                    |
+-------------------------------------------------+
```
The correct answer is `-3.237520E-31`

Please note that the last digit is `0` instead of `1` as

```
scala> java.math.BigDecimal("-0.172787979").divide(java.math.BigDecimal("533704665545018957788294905796.5"), 100, java.math.RoundingMode.DOWN)
val res22: java.math.BigDecimal = -3.237520489418037889998826491401059986665344697406144511563561222578738E-31
```
Since the factional part `.237520` is followed by `4894...`, it should not be rounded up.

### Does this PR introduce _any_ user-facing change?
Yes, users will see correct Decimal multiplication and division results.
Directly multiplying and dividing with `org.apache.spark.sql.types.Decimal()` (not via SQL) will return 39 digit at maximum instead of 38 at maximum and round down instead of round half-up

### How was this patch tested?
Test added

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#43678 from kazuyukitanimura/SPARK-45786.

Authored-by: Kazuyuki Tanimura <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 5ef3a84)
Signed-off-by: Dongjoon Hyun <[email protected]>

* [SPARK-45786][SQL][FOLLOWUP][TEST] Fix Decimal random number tests with ANSI enabled

### What changes were proposed in this pull request?
This follow-up PR fixes the test for SPARK-45786 that is failing in GHA with SPARK_ANSI_SQL_MODE=true

### Why are the changes needed?
The issue discovered in apache#43678 (comment)

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Test updated

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#43853 from kazuyukitanimura/SPARK-45786-FollowUp.

Authored-by: Kazuyuki Tanimura <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 949de34)
Signed-off-by: Dongjoon Hyun <[email protected]>

---------

Signed-off-by: Dongjoon Hyun <[email protected]>
Co-authored-by: Kazuyuki Tanimura <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hsaputra @rxin @AmplabJenkins @srowen