[SPARK-26650][CORE]Yarn Client throws 'ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration'

[shahidki31]

What changes were proposed in this pull request?

When we launch an application in yarn client without any hbase related libraries, spark is throwing a misleading exception.

Added a separate warning message for "NoClassFoundException".

How was this patch tested?

Test steps:
bin/spark-shell --master yarn
Before fix:

After fix:

[HeartSaVioR]

Unlike Hadoop and Hive, Spark will not be built with HBase libs. It will only work when end users link the HBase libraries or contain HBase libraries in their app jar.

So the message doesn't look correct, though I think Fail to invoke HBaseConfiguration is more misleading.

Actually the root issue is that HBaseDelegationToken is activated even end users don't use HBase in their apps, which #23499 is trying to avoid with service loader.

[shahidki31]

Thanks I will check the PR

[SparkQA]

Test build #101491 has finished for PR 23605 at commit 2b59fef.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

Also, please describe how you met this issue, reproducible steps, analysis of the problem and how the PR fixes the PR in JIRA and/or PR.

[shahidki31]

@HyukjinKwon The issue is pretty easy to reproduce in the latest code, if we dont give any configs related to hbase, which was not happening previously. Thanks

[HyukjinKwon]

It's easy to reproduce but I would show what you tried then making people try it by themselves to deduplicate efforts and time. Please fill the details up.

[shahidki31]

Thanks @HyukjinKwon . updated

[SparkQA]

Test build #101508 has finished for PR 23605 at commit aa6fef9.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.