HBASE-26196 Support configuration override for remote cluster of HFileOutputFormat locality sensitive

[bitterfox]

https://issues.apache.org/jira/browse/HBASE-26196

Usecase:

        TableName tableName = TableName.valueOf("table");
        Configuration conf = job.getConfiguration();

        // assume cluster B is secure
        Configuration clusterB = HBaseConfiguration.createClusterConf(
                conf,  "hbase-b-1.example.com,hbase-b-2.example.com,hbase-b-3.example.com:2181:/hbase-b");
        clusterB.set(User.HBASE_SECURITY_CONF_KEY, "kerberos");
        // configure others too like MASTER_KRB_PRINCIPAL, REGIONSERVER_KRB_PRINCIPAL and so on

        // hack. TableMapReduceUtil.initCredentialsForCluster create UserProvider using job's configuration
        // Thus if User.HBASE_SECURITY_CONF_KEY is simple, token isn't obtained
        conf.set(User.HBASE_SECURITY_CONF_KEY, "kerberos");

        TableMapReduceUtil.initCredentialsForCluster(job, clusterB);

        // assume cluster A isn't secure
        conf.set(HConstants.ZOOKEEPER_QUORUM, "hbase-a-1.example.com,hbase-a-2.example.com,hbase-a-3.example.com");
        conf.setInt(HConstants.ZOOKEEPER_CLIENT_PORT, 2181);
        conf.set(HConstants.ZOOKEEPER_ZNODE_PARENT, "/hbase-a");
        conf.set(User.HBASE_SECURITY_CONF_KEY, "simple");

        TableMapReduceUtil.initTableMapperJob(tableName, scanner, Mapper.class, ImmutableBytesWritable.class, Put.class, job);

        try (Connection con = ConnectionFactory.createConnection(clusterB);
             Table table = con.getTable(tableName);
             RegionLocator regionLocator = con.getRegionLocator(tableName)) {
            HFileOutputFormat2.configureIncrementalLoad(job, table, regionLocator);
            conf.set(HFileOutputFormat2.REMOTE_CLUSTER_CONF_PREFIX + User.HBASE_SECURITY_CONF_KEY, "kerberos");
        }


        return job.waitForCompletion(true) ? 0 : 1;

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 26s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 2s	master passed
+1 💚	compile	0m 27s	master passed
+1 💚	shadedjars	8m 9s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 43s	the patch passed
+1 💚	compile	0m 26s	the patch passed
+1 💚	javac	0m 26s	the patch passed
+1 💚	shadedjars	8m 12s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 19s	the patch passed
		_ Other Tests _
+1 💚	unit	11m 7s	hbase-mapreduce in the patch passed.
		38m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3582
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 51fec86558d9 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 11222fc
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/testReport/
Max. process+thread count	4025 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 2s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 24s	master passed
+1 💚	compile	0m 47s	master passed
+1 💚	checkstyle	0m 20s	master passed
+1 💚	spotbugs	0m 47s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 4s	the patch passed
+1 💚	compile	0m 44s	the patch passed
-0 ⚠️	javac	0m 44s	hbase-mapreduce generated 1 new + 197 unchanged - 1 fixed = 198 total (was 198)
-0 ⚠️	checkstyle	0m 18s	hbase-mapreduce: The patch generated 1 new + 40 unchanged - 0 fixed = 41 total (was 40)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	20m 28s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	0m 55s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		42m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3582
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux c2a6b551c3f3 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 11222fc
Default Java	AdoptOpenJDK-1.8.0_282-b08
javac	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-mapreduce.txt
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-mapreduce.txt
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 5s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 4s	master passed
+1 💚	compile	0m 30s	master passed
+1 💚	shadedjars	9m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 47s	the patch passed
+1 💚	compile	0m 28s	the patch passed
+1 💚	javac	0m 28s	the patch passed
+1 💚	shadedjars	9m 11s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	the patch passed
		_ Other Tests _
+1 💚	unit	13m 56s	hbase-mapreduce in the patch passed.
		45m 51s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3582
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f0878c386829 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 11222fc
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/testReport/
Max. process+thread count	2901 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.