Skip to content

Conversation

@sandeepvinayak
Copy link
Contributor

@sandeepvinayak sandeepvinayak commented Feb 16, 2021

Currently, when we deque we do not reset the current wal path, and it blows up while opening the next log, the current path still points to previous wal which is not correct.

private boolean openNextLog() throws IOException {
  Path nextPath = logQueue.peek();
  if (nextPath != null) {
    openReader(nextPath); <=== if we blow up here
    if (reader != null) {
      return true;
    }
  } else {
    // no more files in queue, this could happen for recovered queue, or for a wal group of a sync
    // replication peer which has already been transited to DA or S.
    setCurrentPath(null);
  }
  return false;
} 

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 20s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 3m 51s master passed
+1 💚 compile 3m 11s master passed
+1 💚 checkstyle 1m 4s master passed
+1 💚 spotbugs 2m 3s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 29s the patch passed
+1 💚 compile 3m 9s the patch passed
+1 💚 javac 3m 9s the patch passed
+1 💚 checkstyle 1m 2s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 17m 13s Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚 spotbugs 2m 7s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 15s The patch does not generate ASF License warnings.
46m 17s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2959
JIRA Issue HBASE-25541
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname Linux b7d96ffb323d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / b6649a8
Default Java AdoptOpenJDK-1.8.0_232-b09
Max. process+thread count 95 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/console
versions git=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@bharathv bharathv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nit to add some detail to the commit message to set the context.

LOG.debug("EOF, closing {}", currentPath);
closeReader();
logQueue.remove();
setCurrentPath(null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix, can you add some detail to the commit message, I think it's subtle.

Looks like the issue is if openNextLog() throws, we want currentPath to be null rather than stale.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have good context here, but is it possible to add a small test? That could help anyone get to know this bit better, it's fine though if it is too complicated to add it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got to know from @sandeepvinayak that writing test is complicated. We can skip it.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 3m 47s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 51s master passed
+1 💚 compile 0m 57s master passed
+1 💚 shadedjars 6m 38s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 38s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 23s the patch passed
+1 💚 compile 0m 58s the patch passed
+1 💚 javac 0m 58s the patch passed
+1 💚 shadedjars 6m 38s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 36s the patch passed
_ Other Tests _
+1 💚 unit 142m 24s hbase-server in the patch passed.
172m 13s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #2959
JIRA Issue HBASE-25541
Optional Tests javac javadoc unit shadedjars compile
uname Linux 160676e5f9dd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / b6649a8
Default Java AdoptOpenJDK-1.8.0_232-b09
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/testReport/
Max. process+thread count 4700 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 12s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 54s master passed
+1 💚 compile 1m 15s master passed
+1 💚 shadedjars 7m 29s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 43s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 39s the patch passed
+1 💚 compile 1m 14s the patch passed
+1 💚 javac 1m 14s the patch passed
+1 💚 shadedjars 7m 28s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 43s the patch passed
_ Other Tests _
+1 💚 unit 199m 17s hbase-server in the patch passed.
230m 46s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2959
JIRA Issue HBASE-25541
Optional Tests javac javadoc unit shadedjars compile
uname Linux a7f3885b30f4 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / b6649a8
Default Java AdoptOpenJDK-11.0.6+10
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/testReport/
Max. process+thread count 3265 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2959/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@virajjasani virajjasani merged commit 1beda0d into apache:master Feb 18, 2021
virajjasani pushed a commit that referenced this pull request Feb 18, 2021
virajjasani pushed a commit that referenced this pull request Feb 18, 2021
virajjasani pushed a commit that referenced this pull request Feb 18, 2021
virajjasani pushed a commit that referenced this pull request Feb 18, 2021
virajjasani pushed a commit that referenced this pull request Feb 18, 2021
ddupg pushed a commit to ddupg/hbase that referenced this pull request Feb 19, 2021
wchevreuil pushed a commit to wchevreuil/hbase that referenced this pull request Feb 24, 2021
wchevreuil pushed a commit to wchevreuil/hbase that referenced this pull request May 24, 2021
apache#2959)

Signed-off-by: Bharath Vissapragada <[email protected]>
(cherry picked from commit ed9f8e6)
Change-Id: I6d68df2f91a53fc8eae1a75bc63d847e0da2d63b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants