MAPREDUCE-7307. Potential thread leak in LocatedFileStatusFetcher #2469

dengzhhu653 · 2020-11-18T04:32:11Z

NOTICE

Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

steveloughran

good catch. production code ready to go, just some test tuning to work on

...p-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java

...preduce-client-core/src/test/java/org/apache/hadoop/mapred/TestLocatedFileStatusFetcher.java

steveloughran · 2020-11-21T14:41:44Z

checkstyle

./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestLocatedFileStatusFetcher.java:50:  private static CountDownLatch LATCH = new CountDownLatch(1);:33: Name 'LATCH' must match pattern '^[a-z][a-zA-Z0-9]*$'. [StaticVariableName]
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestLocatedFileStatusFetcher.java:71:    final LocatedFileStatusFetcher fetcher = new LocatedFileStatusFetcher(conf, dirs, true,: Line is longer than 80 characters (found 91). [LineLength]
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestLocatedFileStatusFetcher.java:108:        // Try to sleep some time to let LocatedFileStatusFetcher#getFileStatuses be: Line is longer than 80 characters (found 84). [LineLength]

steveloughran

+1 from me, merging

…2469) Contributed by Zhihua Deng. Change-Id: Iee62539d02bd8f8a928171d8258e640487050a05

steveloughran · 2020-11-23T16:36:27Z

merged to trunk, backported to branch-3.3; tested the new test case before pushing up.

FWIW, now that trunk has a shaded guava and branch-3.3 isn't, we all need to do that extra diligence on backporting...

…sFetcher. (apache#2469) Contributed by Zhihua Deng. Conflicts: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java Change-Id: I34ca51512c2b2296dfcab834b1afa113602cdae2

Potential thread leak in LocatedFileStatusFetcher

0ee010a