Skip to content

HBASE-25590 Bulkload replication HFileRefs cannot be cleared in some … #2969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 26, 2021
Merged

HBASE-25590 Bulkload replication HFileRefs cannot be cleared in some … #2969

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 26 additions & 3 deletions hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.yetus.audience.InterfaceAudience;

import org.apache.hbase.thirdparty.org.apache.commons.collections4.CollectionUtils;

/**
* A configuration for the replication peer cluster.
*/
Expand Down Expand Up @@ -301,6 +303,19 @@ public String toString() {
* @return true if the table need replicate to the peer cluster
*/
public boolean needToReplicate(TableName table) {
return needToReplicate(table, null);
}

/**
* Decide whether the passed family of the table need replicate to the peer cluster according to
* this peer config.
* @param table name of the table
* @param family family name
* @return true if (the family of) the table need replicate to the peer cluster.
* If passed family is null, return true if any CFs of the table need replicate;
* If passed family is not null, return true if the passed family need replicate.
*/
public boolean needToReplicate(TableName table, byte[] family) {
String namespace = table.getNamespaceAsString();
if (replicateAllUserTables) {
// replicate all user tables, but filter by exclude namespaces and table-cfs config
Expand All @@ -312,9 +327,12 @@ public boolean needToReplicate(TableName table) {
return true;
}
Collection<String> cfs = excludeTableCFsMap.get(table);
// if cfs is null or empty then we can make sure that we do not need to replicate this table,
// If cfs is null or empty then we can make sure that we do not need to replicate this table,
// otherwise, we may still need to replicate the table but filter out some families.
return cfs != null && !cfs.isEmpty();
return cfs != null && !cfs.isEmpty()
// If exclude-table-cfs contains passed family then we make sure that we do not need to
// replicate this family.
&& (family == null || !cfs.contains(Bytes.toString(family)));
} else {
// Not replicate all user tables, so filter by namespaces and table-cfs config
if (namespaces == null && tableCFsMap == null) {
Expand All @@ -325,7 +343,12 @@ public boolean needToReplicate(TableName table) {
if (namespaces != null && namespaces.contains(namespace)) {
return true;
}
return tableCFsMap != null && tableCFsMap.containsKey(table);
// If table-cfs contains this table then we can make sure that we need replicate some CFs of
// this table. Further we need all CFs if tableCFsMap.get(table) is null or empty.
return tableCFsMap != null && tableCFsMap.containsKey(table)
&& (family == null || CollectionUtils.isEmpty(tableCFsMap.get(table))
// If table-cfs must contain passed family then we need to replicate this family.
|| tableCFsMap.get(table).contains(Bytes.toString(family)));
}
}
}
Loading