[SPARK-10427][SQL]Respect -S option for spark-sql

[zhichao-li]

"-S" option is ignored by the conf setting logic at the moment, we need to pick up this to complete the "silent" function.

Before:
lizhichao@lizhichao-desktop:~/repo/spark/bin$ ./spark-sql -S -e "show databases"
SET hive.support.sql11.reserved.keywords=false
SET spark.sql.hive.version=1.2.1
SET spark.sql.hive.version=1.2.1
OK
default

After:
lizhichao@lizhichao-desktop:~/repo/spark/bin$ ./spark-sql -S -e "show databases"
default

[zhichao-li]

It is redundant here since the contexClasstloader would be reset correctly within the constructor of ClientWrapper, and setting the contextClassLoader too early here would prevent us to retrieve the other classLoader within the same thread which having SessionState containing the user input option.

[zhichao-li]

ping @chenghao-intel @yhuai @liancheng

[chenghao-intel]

This probably causes ClassNotFoundException or ClassCastException if we removed this line, as we do need to load the specified version of Hive Metastore jars and its dependencies.

[SparkQA]

Test build #43011 has finished for PR 8915 at commit b9dc8d1.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #43014 has finished for PR 8915 at commit 4596424.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[andrewor14]

@zhichao-li can you rebase to master?
@liancheng please have a look. Does this still apply?

[zhichao-li]

@andrewor14 @liancheng , These options like "-S" "-v" still be discarded since they are all stored within SessionState which would be reconstructed by HiveContext while creating ClientWrapper.

The logic flow kind of like:
SessionState is stored in a static field.
1)sparkClassLoader(SparkSQLCLIDriver) -> SessionState (containing user parameters)
2)sparkClassLoader(executationHive) -> reconstruct SessionState
3)MutableURLClassLoader(metaDataHive) -> reconstruct SessionState

Somehow we need to pass the parameters from sparkClassLoader to MutableURLClassLoader.

[SparkQA]

Test build #47988 has finished for PR 8915 at commit 7844933.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #47992 has finished for PR 8915 at commit 506cc13.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #47994 has finished for PR 8915 at commit 7844933.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

@zhichao-li My feeling is that it's probably not worth so much effort for such a tiny feature...

[chenghao-intel]

revert the change

[chenghao-intel]

@liancheng some of our customer complaints that Spark SQL produces too many logs in their cronjob when immigrate the existing scripts from the legacy product. As Hive will suppress the log when given the option -S.

[SparkQA]

Test build #48148 has finished for PR 8915 at commit 0d3c596.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #48149 has finished for PR 8915 at commit c1a7ef5.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #48152 has finished for PR 8915 at commit 592ed63.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[zhichao-li]

retest this please.

[SparkQA]

Test build #48160 has finished for PR 8915 at commit 592ed63.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[zhichao-li]

retest this please.

[SparkQA]

Test build #48173 has finished for PR 8915 at commit 592ed63.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[zhichao-li]

retest this please.

[SparkQA]

Test build #48295 has finished for PR 8915 at commit 592ed63.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

Thanks for the pull request. I'm going through a list of pull requests to cut them down since the sheer number is breaking some of the tooling we have. Due to lack of activity on this pull request, I'm going to push a commit to close it. Feel free to reopen it or create a new one.