Skip to content

Conversation

@amanomer
Copy link
Contributor

@amanomer amanomer commented Sep 23, 2019

What changes were proposed in this pull request?

This PR add support sorting Duration columns for ThriftServerSessionPage.

Why are the changes needed?

Previously, it's not sorted correctly.

Does this PR introduce any user-facing change?

Yes.

How was this patch tested?

Manually do the following and test sorting on those columns in the Spark Thrift Server Session Page.

$ sbin/start-thriftserver.sh
$ bin/beeline -u jdbc:hive2://localhost:10000
0: jdbc:hive2://localhost:10000> create table t(a int);
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (0.521 seconds)
0: jdbc:hive2://localhost:10000> select * from t;
+----+--+
| a  |
+----+--+
+----+--+
No rows selected (0.772 seconds)
0: jdbc:hive2://localhost:10000> show databases;
+---------------+--+
| databaseName  |
+---------------+--+
| default       |
+---------------+--+
1 row selected (0.249 seconds)

Sorted by Duration column:
image

… for ThriftServerSessionPage

This PR add support sorting `Execution Time` and `Duration` columns for `ThriftServerSessionPage`.

Previously, it's not sorted correctly.

Yes.

Manually do the following and test sorting on those columns in the Spark Thrift Server Session Page.
```
$ sbin/start-thriftserver.sh
$ bin/beeline -u jdbc:hive2://localhost:10000
0: jdbc:hive2://localhost:10000> create table t(a int);
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (0.521 seconds)
0: jdbc:hive2://localhost:10000> select * from t;
+----+--+
| a  |
+----+--+
+----+--+
No rows selected (0.772 seconds)
0: jdbc:hive2://localhost:10000> show databases;
+---------------+--+
| databaseName  |
+---------------+--+
| default       |
+---------------+--+
1 row selected (0.249 seconds)
```

**Sorted by `Execution Time` column**:
![image](https://user-images.githubusercontent.com/5399861/65387476-53038900-dd7a-11e9-885c-fca80287f550.png)

**Sorted by `Duration` column**:
![image](https://user-images.githubusercontent.com/5399861/65387481-6e6e9400-dd7a-11e9-9318-f917247efaa8.png)

Closes apache#25892 from wangyum/SPARK-28599.

Authored-by: Yuming Wang <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
@amanomer
Copy link
Contributor Author

cc @dongjoon-hyun @wangyum @srowen

Copy link
Member

@srowen srowen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks OK pending tests as a back-port.

@SparkQA
Copy link

SparkQA commented Sep 23, 2019

Test build #4882 has finished for PR 25906 at commit da43acc.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @amanomer . This backporting is correctly made by using @wangyum 's authorship.
I fixed the followings because this PR still ignored my advice on the original PR.

  • I fixed the title because this PR doesn't fix Execution Time column because it doesn't exist.
  • I fixed the PR description with the same reason, also removed the first image Sorted by Execution Time column:

The renaming image is technically wrong because it's a screenshot of 3.0.0-SNAPSHOT. We had better replace it with 2.4.5-SNAPSHOT. However, it's understandable.

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-28599][SQL][2.4] Fix Execution Time and Duration column sorting for ThriftServerSessionPage [SPARK-28599][SQL][2.4] Fix Duration column sorting for ThriftServerSessionPage Sep 23, 2019
@dongjoon-hyun
Copy link
Member

Merged to branch-2.4. Thank you, @amanomer and @srowen .

dongjoon-hyun pushed a commit that referenced this pull request Sep 23, 2019
…rSessionPage

### What changes were proposed in this pull request?
This PR add support sorting `Duration` columns for `ThriftServerSessionPage`.

### Why are the changes needed?
Previously, it's not sorted correctly.

### Does this PR introduce any user-facing change?
Yes.

### How was this patch tested?
Manually do the following and test sorting on those columns in the Spark Thrift Server Session Page.
```
$ sbin/start-thriftserver.sh
$ bin/beeline -u jdbc:hive2://localhost:10000
0: jdbc:hive2://localhost:10000> create table t(a int);
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (0.521 seconds)
0: jdbc:hive2://localhost:10000> select * from t;
+----+--+
| a  |
+----+--+
+----+--+
No rows selected (0.772 seconds)
0: jdbc:hive2://localhost:10000> show databases;
+---------------+--+
| databaseName  |
+---------------+--+
| default       |
+---------------+--+
1 row selected (0.249 seconds)
```

**Sorted by `Duration` column**:
![image](https://user-images.githubusercontent.com/5399861/65387481-6e6e9400-dd7a-11e9-9318-f917247efaa8.png)

Closes #25906 from amanomer/BP_28599.

Authored-by: Yuming Wang <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
@wangyum
Copy link
Member

wangyum commented Sep 23, 2019

Thank you all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants