HBASE-28321 RpcConnectionRegistry is broken when security is enabled … #5688
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Almost done, still need to add some tests to test more corner cases, and also enable the UTs for blocking rpc client. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
…and we use different principal for master and region server
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
bbeaudreault
approved these changes
Feb 23, 2024
There was a problem hiding this comment.
@Apache9 Sorry for the delay here. Note that I am out of office next week, so won't be able to look back until the following week.
Overall this looks good. I had a few comments/questions. I don't have hbase kerberos in my environment, so I won't be able to test this for the 2.6.0 release. I will rely on someone else in the RC vote process. I can test it without kerberos though.
| // preamble call, so we should fallback to randomly select a principal to use | ||
| // TODO: find a way to reconnect without failing all the pending calls, for now, when we | ||
| // reach here, shutdown should have already been scheduled | ||
| return; |
There was a problem hiding this comment.
Where does this fallback to random happen? I had the same question when reading the blocking client impl.
There was a problem hiding this comment.
I've sent a discuss email to the dev list, there is no response so I just choose the easiest way, just do not implement the fallback logic. In our current architecture it really not easy to implement the logic here, as said in the comment, when we reach here, the connection should have already been closed because server will close the connection immediately when preamble header mismatches. And at client side, all the calls are associated with a connection, so closing a connection means failing these calls.
https://lists.apache.org/thread/81p7f5f5lrrqh1v51mvct43spx8l0rhc
| // Call[id=32153218,methodName=Get] | ||
| return new ToStringBuilder(this, ToStringStyle.SHORT_PREFIX_STYLE).append("id", id) | ||
| .append("methodName", md.getName()).toString(); | ||
| .append("methodName", md != null ? md.getName() : "").toString(); |
There was a problem hiding this comment.
might be weird to see Call[id=23123123,methodName=] in null case. Is there anything more useful we can put?
There was a problem hiding this comment.
The log has been like this for a long time so just want to keep it align, as this is not a problem which want to resolve here. Can open another issue for optimizing this message. For logging in this PR, I outout the response type so we could know the preamble call type.
| // fail all pending calls | ||
| ch.pipeline().fireUserEventTriggered(BufferCallEvent.fail(e)); | ||
| shutdown0(); | ||
| rpcClient.failedServers.addToFailedServers(remoteId.getAddress(), e); |
There was a problem hiding this comment.
this is new right? what's the reason. it's probably good, but just checking
There was a problem hiding this comment.
We have this line right after the failInit call when connecting fail, it is at line 334 in the old code. I moved it here since we will call failInit at other places, I think for all those cases we should add the server to failedServers.
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java
Show resolved
Hide resolved
Apache9
added a commit
that referenced
this pull request
Feb 25, 2024
…and we use different principal for master and region server (#5688) Signed-off-by: Bryan Beaudreault <[email protected]> (cherry picked from commit c4a02f7)
Apache9
added a commit
to Apache9/hbase
that referenced
this pull request
Feb 25, 2024
…and we use different principal for master and region server (apache#5688) Signed-off-by: Bryan Beaudreault <[email protected]> (cherry picked from commit c4a02f7)
Apache9
added a commit
that referenced
this pull request
Feb 28, 2024
…and we use different principal for master and region server (#5688) (#5707) Signed-off-by: Bryan Beaudreault <[email protected]> (cherry picked from commit c4a02f7)
Apache9
added a commit
that referenced
this pull request
Feb 28, 2024
…and we use different principal for master and region server (#5688) (#5707) Signed-off-by: Bryan Beaudreault <[email protected]> (cherry picked from commit c4a02f7)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…and we use different principal for master and region server