HBASE-26340 - fix RegionSizeCalculator getLEngth to bytes instead of … #3872
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…converting from MB
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
petersomogyi
approved these changes
Nov 24, 2021
Apache9
reviewed
Nov 24, 2021
| totalStaticBloomSizeKB += (int) (store.getTotalStaticBloomSize() / 1024); | ||
| } | ||
| //HBASE-26340 Fix false "0" size under 1MB | ||
| if(storefileSizeMB < 1 && nonEmptyStoreExist) { |
There was a problem hiding this comment.
Do we need to apply this trick to other fields as well? Just introduce a round method
long round(long size, long unit) {
// normally return size / unit, if size is smaller than unit but greater than 0, return 1
}
And all the size should be calculated by this method
There was a problem hiding this comment.
storefileSizeMB is integer, it is either 0 or 1 here (well, or any other positive integer, but staying at the less then 1MB example), and we go through all the storelist above, if we do the rounding there, we keep adding 1MB every time there is like 10KB of data, that would result in a false size as well. We need "nonEmptyStoreExist" to tell us if there were storefile anywhere, or else we'll just see 0 here and cant round up to 1 bc it was either really empty or size didn't reach the 1MB
There was a problem hiding this comment.
I could use a new variable "long storefileSizeMBTemp" and create a function "int round(long size, int unit)" but it just seems kind of a special case to create a function for it.
There was a problem hiding this comment.
At region server side we should know the accurate size in bytes, so when calculating we could just cumulate the size in bytes, and at last we convert them to size in KB or size in MB or what ever unit. WDYT?
There was a problem hiding this comment.
That works, sure, at least I think we are thinking the same when I said above we could use long to track "storefileSize(MB)(Temp)" - no need for Temp though just remove MB - but at the end we set this as MB just like before. I'll do the change today.
…ean check Change-Id: Idce876f4b95b0f28e3e84a08b367f77bc76fb268
Apache9
approved these changes
Dec 8, 2021
There was a problem hiding this comment.
It is OK for fixing the problem in HBASE-26340, so +1 on merging.
But I think we'd better have a follow on issue to change all other size calculating using the same pattern, such as the storeUncompressedSizeMB, memstoreSizeMB, etc.
Thanks~
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
The pre commit is still running? Anyway, let's wait for the newest result... |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
Apache9
pushed a commit
that referenced
this pull request
Dec 18, 2021
Signed-off-by: Peter Somogyi <[email protected] Signed-off-by: Duo Zhang <[email protected]>
Apache9
pushed a commit
that referenced
this pull request
Dec 18, 2021
Signed-off-by: Peter Somogyi <[email protected] Signed-off-by: Duo Zhang <[email protected]>
Apache9
pushed a commit
that referenced
this pull request
Dec 18, 2021
Signed-off-by: Peter Somogyi <[email protected] Signed-off-by: Duo Zhang <[email protected]>
vinayakphegde
pushed a commit
to vinayakphegde/hbase
that referenced
this pull request
Apr 4, 2024
Signed-off-by: Peter Somogyi <[email protected] Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 8fa8344) Change-Id: I380b2e9acd4385a2a1228e1115f686c63410ef6e
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If we have store files under 1MB, list_regions will report region size of 0. This fix will use 1MB of value if there are store files available for a region. It makes more sense than using 0MB if store file size are less than 1MB.