Skip to content

Commit 26d9b6b

Browse files
litao-buptssesrowen
authored andcommitted
[SPARK-8657] [YARN] Fail to upload resource to viewfs
Fail to upload resource to viewfs in spark-1.4 JIRA Link: https://issues.apache.org/jira/browse/SPARK-8657 Author: Tao Li <[email protected]> Closes #7125 from litao-buptsse/SPARK-8657-for-master and squashes the following commits: 65b13f4 [Tao Li] [SPARK-8657] [YARN] Fail to upload resource to viewfs
1 parent f61c989 commit 26d9b6b

File tree

1 file changed

+2
-1
lines changed
  • yarn/src/main/scala/org/apache/spark/deploy/yarn

1 file changed

+2
-1
lines changed

yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -321,8 +321,9 @@ private[spark] class Client(
321321
val linkname = targetDir.map(_ + "/").getOrElse("") +
322322
destName.orElse(Option(localURI.getFragment())).getOrElse(localPath.getName())
323323
val destPath = copyFileToRemote(dst, localPath, replication)
324+
val destFs = FileSystem.get(destPath.toUri(), hadoopConf)
324325
distCacheMgr.addResource(
325-
fs, hadoopConf, destPath, localResources, resType, linkname, statCache,
326+
destFs, hadoopConf, destPath, localResources, resType, linkname, statCache,
326327
appMasterOnly = appMasterOnly)
327328
(false, linkname)
328329
} else {

0 commit comments

Comments
 (0)