[SPARK-28004][UI] Update jquery to 3.4.1 #24843

srowen · 2019-06-11T19:39:21Z

What changes were proposed in this pull request?

We're using an old-ish jQuery, 1.12.4, and should probably update for Spark 3 to keep up in general, but also to keep up with CVEs. In fact, we know of at least one resolved in only 3.4.0+ (https://nvd.nist.gov/vuln/detail/CVE-2019-11358). They may not affect Spark, but, if the update isn't painful, maybe worthwhile in order to make future 3.x updates easier.

jQuery 1 -> 2 doesn't sound like a breaking change, as 2.0 is supposed to maintain compatibility with 1.9+ (https://blog.jquery.com/2013/04/18/jquery-2-0-released/)

2 -> 3 has breaking changes: https://jquery.com/upgrade-guide/3.0/. It's hard to evaluate each one, but the most likely area for problems is in ajax(). However, our usage of jQuery (and plugins) is pretty simple.

Update jquery to 3.4.1; update jquery blockUI and mustache to latest

How was this patch tested?

Manual testing of docs build (except R docs), worker/master UI, spark application UI.
Note: this really doesn't guarantee it works, as our tests can't test javascript, and this is merely anecdotal testing, although I clicked about every link I could find. There's a risk this breaks a minor part of the UI; it does seem to work fine in the main.

SparkQA · 2019-06-11T21:05:35Z

Test build #106394 has finished for PR 24843 at commit 185cac1.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

srowen · 2019-06-11T21:16:55Z

core/src/main/resources/org/apache/spark/ui/static/jquery.mustache.js

    return obj != null && typeof obj === 'object' && (propName in obj);
  }

+  /**


These changes are all from the source jquery.mustache.js, not custom modifications

I also checked this. This file is identical with mustache.js v3.0.1. Only the trailing spaces are removed in a few lines.

srowen · 2019-06-11T21:17:09Z

docs/js/main.js

  // Scroll now too in case we had opened the page on a hash, but wait a bit because some browsers
  // will try to do *their* initial scroll after running the onReady handler.
-  $(window).load(function() { setTimeout(function() { maybeScrollToHash(); }, 25); }); 
+  $(window).on('load', function() { setTimeout(function() { maybeScrollToHash(); }, 25); });


This is actually the only change I seemed to need to make to the Spark javascript.

Yep. This is the correct update based on the upgrade-guide.

https://jquery.com/upgrade-guide/3.0/#breaking-change-load-unload-and-error-removed

srowen · 2019-06-12T01:11:48Z

Ah right. The history server. There's a problem there. I'll look into it.

SparkQA · 2019-06-12T04:13:33Z

Test build #106400 has finished for PR 24843 at commit 8fe52eb.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-06-12T20:42:14Z

Test build #106441 has finished for PR 24843 at commit 48d3ca4.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-06-12T22:39:44Z

Test build #106443 has finished for PR 24843 at commit 3ea4db7.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-06-13T14:14:31Z

Test build #106465 has finished for PR 24843 at commit ebe85ba.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

srowen · 2019-06-13T14:18:39Z

Hm, for anyone following along, these tests pass locally. Maybe I'm doing something wrong running locally. Looks like an httpclient version mismatch that I'll have to run down: Cannot locate declared field class : org.apache.http.impl.client.HttpClientBuilder.dnsResolver

SparkQA · 2019-06-13T17:19:25Z

Test build #106471 has finished for PR 24843 at commit f1e02e4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2019-06-15T05:05:22Z

core/src/main/resources/org/apache/spark/ui/static/historypage.js


    var historySummary = $("#history-summary");
-    var searchString = historySummary["context"]["location"]["search"];
+    var searchString = window.location.search;


I also verified that this works in the same way.

dongjoon-hyun

+1, LGTM. Thank you, @srowen !

This should be tested in the browser's private mode to avoid old javascript cache.
I verified this PR with the followings.

Spark shell application UI
Thrift JDBC/ODBC Server application UI
Spark History Server UI on my several Spark history logs.

All tabs look working correctly. I might miss a few tiny UI features like hover info, but the navigation and the info looks correct in general. I hope we can merge this and proceed.

Merged to master.

dongjoon-hyun · 2019-06-15T05:28:07Z

cc @gatorsmile and @gengliangwang

## What changes were proposed in this pull request? We're using an old-ish jQuery, 1.12.4, and should probably update for Spark 3 to keep up in general, but also to keep up with CVEs. In fact, we know of at least one resolved in only 3.4.0+ (https://nvd.nist.gov/vuln/detail/CVE-2019-11358). They may not affect Spark, but, if the update isn't painful, maybe worthwhile in order to make future 3.x updates easier. jQuery 1 -> 2 doesn't sound like a breaking change, as 2.0 is supposed to maintain compatibility with 1.9+ (https://blog.jquery.com/2013/04/18/jquery-2-0-released/) 2 -> 3 has breaking changes: https://jquery.com/upgrade-guide/3.0/. It's hard to evaluate each one, but the most likely area for problems is in ajax(). However, our usage of jQuery (and plugins) is pretty simple. Update jquery to 3.4.1; update jquery blockUI and mustache to latest ## How was this patch tested? Manual testing of docs build (except R docs), worker/master UI, spark application UI. Note: this really doesn't guarantee it works, as our tests can't test javascript, and this is merely anecdotal testing, although I clicked about every link I could find. There's a risk this breaks a minor part of the UI; it does seem to work fine in the main. Closes apache#24843 from srowen/SPARK-28004. Authored-by: Sean Owen <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>

srowen · 2019-06-15T14:44:38Z

core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala

-    implicit val webDriver: WebDriver = new HtmlUnitDriver(true) {
-      getWebClient.getOptions.setThrowExceptionOnScriptError(false)
-    }
+    implicit val webDriver: WebDriver =


One last clarifying comment: This change slightly improves the test to not ignore javascript errors, and as part of that, needed to test as a modern browser (IE 11 vs IE 8).

gengliangwang · 2019-06-17T05:52:32Z

A late LGTM. I also tried grep -r "jquery-1.12.4.min.js" * and there is no matched results.

gabrieljones · 2019-10-07T19:05:48Z

any chance this will be back ported to 2.3? 2.3 is currently quarantined at my site because of this.

srowen · 2019-10-07T19:11:37Z

No, 2.3 is EOL. You should update to 2.4.x

We're using an old-ish jQuery, 1.12.4, and should probably update for Spark 3 to keep up in general, but also to keep up with CVEs. In fact, we know of at least one resolved in only 3.4.0+ (https://nvd.nist.gov/vuln/detail/CVE-2019-11358). They may not affect Spark, but, if the update isn't painful, maybe worthwhile in order to make future 3.x updates easier. jQuery 1 -> 2 doesn't sound like a breaking change, as 2.0 is supposed to maintain compatibility with 1.9+ (https://blog.jquery.com/2013/04/18/jquery-2-0-released/) 2 -> 3 has breaking changes: https://jquery.com/upgrade-guide/3.0/. It's hard to evaluate each one, but the most likely area for problems is in ajax(). However, our usage of jQuery (and plugins) is pretty simple. Update jquery to 3.4.1; update jquery blockUI and mustache to latest Manual testing of docs build (except R docs), worker/master UI, spark application UI. Note: this really doesn't guarantee it works, as our tests can't test javascript, and this is merely anecdotal testing, although I clicked about every link I could find. There's a risk this breaks a minor part of the UI; it does seem to work fine in the main. Closes apache#24843 from srowen/SPARK-28004. Authored-by: Sean Owen <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>

Update jquery to 3.4.1; update jquery blockUI and mustache to latest

185cac1

srowen self-assigned this Jun 11, 2019

srowen changed the title ~~[SPARK-28004][UI]~~ [SPARK-28004][UI] Update jquery to 3.4.1 Jun 11, 2019

srowen commented Jun 11, 2019

View reviewed changes

Fix history server access to query string

8fe52eb

dongjoon-hyun added the IMPROVEMENT label Jun 12, 2019

Attempt to fix JS Selenium problem with htmlunit update

48d3ca4

Oops, build fix

3ea4db7

Try to get SBT to honor the htmlunit dependency update

ebe85ba

Also coerce httpclient version up for SBT

f1e02e4

dongjoon-hyun added WEB UI and removed IMPROVEMENT labels Jun 14, 2019

dongjoon-hyun reviewed Jun 15, 2019

View reviewed changes

dongjoon-hyun approved these changes Jun 15, 2019

View reviewed changes

dongjoon-hyun closed this in 15462e1 Jun 15, 2019

srowen commented Jun 15, 2019

View reviewed changes

srowen deleted the SPARK-28004 branch June 19, 2019 13:55

srowen mentioned this pull request Sep 19, 2019

[MINOR][BUILD] Fix about 15 misc build warnings #25852

Closed

[SPARK-28004][UI] Update jquery to 3.4.1 #24843

[SPARK-28004][UI] Update jquery to 3.4.1 #24843

Uh oh!

Conversation

srowen commented Jun 11, 2019

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

SparkQA commented Jun 11, 2019

Uh oh!

srowen Jun 11, 2019

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun Jun 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

srowen Jun 11, 2019

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun Jun 15, 2019

Choose a reason for hiding this comment

Uh oh!

srowen commented Jun 12, 2019

Uh oh!

SparkQA commented Jun 12, 2019

Uh oh!

SparkQA commented Jun 12, 2019

Uh oh!

SparkQA commented Jun 12, 2019

Uh oh!

SparkQA commented Jun 13, 2019

Uh oh!

srowen commented Jun 13, 2019

Uh oh!

SparkQA commented Jun 13, 2019

Uh oh!

dongjoon-hyun Jun 15, 2019

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Jun 15, 2019

Uh oh!

srowen Jun 15, 2019

Choose a reason for hiding this comment

Uh oh!

gengliangwang commented Jun 17, 2019

Uh oh!

gabrieljones commented Oct 7, 2019

Uh oh!

srowen commented Oct 7, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

dongjoon-hyun Jun 15, 2019 •

edited

Loading

dongjoon-hyun left a comment •

edited

Loading