Skip to content

HBASE-22628 Document the custom WAL directory (hbase.wal.dir) usage #391

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 24, 2019
Merged

HBASE-22628 Document the custom WAL directory (hbase.wal.dir) usage #391

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions src/main/asciidoc/_chapters/architecture.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2491,6 +2491,26 @@ For example: If source cluster FS client configurations are copied to the destin

NOTE: `DefaultSourceFSConfigurationProvider` supports only `xml` type files. It loads source cluster FS client configuration only once, so if source cluster FS client configuration files are updated, every peer(s) cluster RS must be restarted to reload the configuration.

[[arch.custom.wal.dir]]
=== Custom WAL Directory
HBASE-17437 added support for specifying a WAL directory outside the HBase root directory or even in a different FileSystem since 1.3.3/2.0+. Some FileSystems (such as Amazon S3) don’t support append or consistent writes, in such scenario WAL directory needs to be configured in a different FileSystem to avoid loss of writes.

Following configurations were added to accomplish this:
. `hbase.wal.dir`
+
This defines where the root WAL directory is located, could be on a different FileSystem than the root directory. WAL directory can not be set to a subdirectory of the root directory. The default value of this is the root directory if unset.
+
. `hbase.rootdir.perms`
+
Configures FileSystem permissions to set on the root directory. This is '700' by default.
+
. `hbase.wal.dir.perms`
+
Configures FileSystem permissions to set on the WAL directory FileSystem. This is '700' by default.
+

NOTE: While migrating to custom WAL dir (outside the HBase root directory or a different FileSystem) existing WAL files must be copied manually to new WAL dir, otherwise it may lead to data loss/inconsistency as HMaster has no information about previous WAL directory.

[[arch.hdfs]]
== HDFS

Expand Down