-
Notifications
You must be signed in to change notification settings - Fork 9.1k
HDDS-1094. Performance test infrastructure : skip writing user data on Datanode. Contributed by Supratim Deka #1323
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
...rvice/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerFactory.java
Outdated
Show resolved
Hide resolved
...rvice/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerFactory.java
Outdated
Show resolved
Hide resolved
...rvice/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerFactory.java
Outdated
Show resolved
Hide resolved
...rvice/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerFactory.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this required because you are added a test for freon with the null ChunkManager?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is required because each individual test case in TestDataValidate sets this parameter in the RandomKeyGenerator builder.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this nice test hook @supratimdeka.
The patch looks pretty good to me. A few minor comments.
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
…n Datanode. Contributed by Supratim Deka
5a23756
to
4c68469
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
…n Datanode. Contributed by Supratim Deka (apache#1323)
…n Datanode. Contributed by Supratim Deka (apache#1323)
https://issues.apache.org/jira/browse/HDDS-1094
Added an alternate ChunkManager implementation which drops all chunk writes without writing to disk. Chunk Reads are cooked up zero-filled buffers.
The goal of this infrastructure is to enable high-throughput tests and stress the pipeline (including the Ozone metadata components) without using faster storage devices like flash drives.
Added an extension to TestDataValidate (with the RandomKeyGenerator) to test the alternate ChunkManager.