[SPARK-38770][K8S] Remove renameMainAppResource from baseDriverContainer

### What changes were proposed in this pull request?

This PR aims to simply steps to re-write primary resource in k8s spark application.

### Why are the changes needed?

Re-write primary resource uses `renameMainAppResource` twice.
* First `renameMainAppResource` in `baseDriverContainer` in  is introduced by #23546
* #25870 refactors `renameMainAppResource` and introduces `renameMainAppResource` in `configureForJava`.

Refactoring and `renameMainAppResource` in `configureForJava` makes `renameMainAppResource` in `baseDriverContainer` useless.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

* Pass the GA.
* Pass k8s IT.
```
$ build/sbt -Pkubernetes -Pkubernetes-integration-tests -Dtest.exclude.tags=r -Dspark.kubernetes.test.imageRepo=kubespark "kubernetes-integration-tests/test"
[info] KubernetesSuite:
[info] - Run SparkPi with no resources (17 seconds, 443 milliseconds)
[info] - Run SparkPi with no resources & statefulset allocation (17 seconds, 858 milliseconds)
[info] - Run SparkPi with a very long application name. (30 seconds, 450 milliseconds)
[info] - Use SparkLauncher.NO_RESOURCE (18 seconds, 596 milliseconds)
[info] - Run SparkPi with a master URL without a scheme. (18 seconds, 534 milliseconds)
[info] - Run SparkPi with an argument. (21 seconds, 853 milliseconds)
[info] - Run SparkPi with custom labels, annotations, and environment variables. (14 seconds, 285 milliseconds)
[info] - All pods have the same service account by default (13 seconds, 800 milliseconds)
[info] - Run extraJVMOptions check on driver (7 seconds, 825 milliseconds)
[info] - Run SparkRemoteFileTest using a remote data file (15 seconds, 242 milliseconds)
[info] - Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j2.properties (15 seconds, 491 milliseconds)
[info] - Run SparkPi with env and mount secrets. (26 seconds, 967 milliseconds)
[info] - Run PySpark on simple pi.py example (20 seconds, 318 milliseconds)
[info] - Run PySpark to test a pyfiles example (25 seconds, 659 milliseconds)
[info] - Run PySpark with memory customization (25 seconds, 608 milliseconds)
[info] - Run in client mode. (14 seconds, 620 milliseconds)
[info] - Start pod creation from template (19 seconds, 916 milliseconds)
[info] - SPARK-38398: Schedule pod creation from template (19 seconds, 966 milliseconds)
[info] - PVs with local hostpath storage on statefulsets (22 seconds, 380 milliseconds)
[info] - PVs with local hostpath and storageClass on statefulsets (26 seconds, 935 milliseconds)
[info] - PVs with local storage (30 seconds, 75 milliseconds)
[info] - Launcher client dependencies (2 minutes, 48 seconds)
[info] - SPARK-33615: Launcher client archives (1 minute, 26 seconds)
[info] - SPARK-33748: Launcher python client respecting PYSPARK_PYTHON (1 minute, 47 seconds)
[info] - SPARK-33748: Launcher python client respecting spark.pyspark.python and spark.pyspark.driver.python (1 minute, 51 seconds)
[info] - Launcher python client dependencies using a zip file (1 minute, 51 seconds)
[info] - Test basic decommissioning (59 seconds, 765 milliseconds)
[info] - Test basic decommissioning with shuffle cleanup (1 minute, 3 seconds)
[info] - Test decommissioning with dynamic allocation & shuffle cleanups (2 minutes, 58 seconds)
[info] - Test decommissioning timeouts (58 seconds, 754 milliseconds)
[info] - SPARK-37576: Rolling decommissioning (1 minute, 15 seconds)
[info] Run completed in 29 minutes, 15 seconds.
[info] Total number of tests run: 31
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 31, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 2020 s (33:40), completed 2022-4-2 12:35:52
```
PS. #23546 introduces deleted code and `DepsTestsSuite`. `DepsTestsSuite` can check re-write primary resource. This PR can pass `DepsTestsSuite`, which can prove deletion about `renameMainAppResource` in `baseDriverContainer` does not affect the process about re-write primary resource.

Closes #36044 from dcoliversun/SPARK-38770.

Authored-by: Qian.Sun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>

Author: dcoliversun

resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverCommandFeatureStep.scala

@@ -120,12 +120,6 @@ private[spark] class DriverCommandFeatureStep(conf: KubernetesDriverConf)
   }
 
   private def baseDriverContainer(pod: SparkPod, resource: String): ContainerBuilder = {
-    // re-write primary resource, app jar is also added to spark.jars by default in SparkSubmit
-    val resolvedResource = if (conf.mainAppResource.isInstanceOf[JavaMainAppResource]) {
-      KubernetesUtils.renameMainAppResource(resource, Option(conf.sparkConf), false)
-    } else {
-      resource
-    }
     var proxyUserArgs = Seq[String]()
     if (!conf.proxyUser.isEmpty) {
       proxyUserArgs = proxyUserArgs :+ "--proxy-user"
@@ -136,7 +130,7 @@ private[spark] class DriverCommandFeatureStep(conf: KubernetesDriverConf)
       .addToArgs(proxyUserArgs: _*)
       .addToArgs("--properties-file", SPARK_CONF_PATH)
       .addToArgs("--class", conf.mainClass)
-      .addToArgs(resolvedResource)
+      .addToArgs(resource)
       .addToArgs(conf.appArgs: _*)
   }
 }