Security: reduce memory usage of DnRoleMapper #34250
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The `DnRoleMapper` class is used to map distinguished names of groups and users to role names. This mapper builds in an internal map that maps from a `com.unboundid.ldap.sdk.DN` to a `Set<String>`. In cases where a lot of distinct DNs are mapped to roles, this can consume quite a bit of memory. The majority of the memory is consumed by the DN object. For example, a 94 character DN that has 9 relative DNs (RDN) will retain 4KB of memory, whereas the String itself consumes less than 250 bytes. In order to reduce memory usage, we can map from a normalized DN string to a List of roles. The normalized string is actually how the DN class determines equality with another DN and we can drop the overhead of needing to keep all of the other objects in memory. Additionally the use of a List provides memory savings as each HashSet is backed by a HashMap, which consumes a great deal more memory than an appropriately sized ArrayList. The uniqueness we get from a Set is maintained by first building a set when parsing the file and then converting to a list upon completion. Closes elastic#34237
jaymode
added
>enhancement
v7.0.0
:Security/Authentication
Collaborator
|
Pinging @elastic/es-security |
bizybot
approved these changes
Oct 3, 2018
Contributor
bizybot
left a comment
bizybot
left a comment
There was a problem hiding this comment.
Good find, LGTM, Thank you.
jaymode
added a commit
that referenced
this pull request
Oct 3, 2018
The `DnRoleMapper` class is used to map distinguished names of groups and users to role names. This mapper builds in an internal map that maps from a `com.unboundid.ldap.sdk.DN` to a `Set<String>`. In cases where a lot of distinct DNs are mapped to roles, this can consume quite a bit of memory. The majority of the memory is consumed by the DN object. For example, a 94 character DN that has 9 relative DNs (RDN) will retain 4KB of memory, whereas the String itself consumes less than 250 bytes. In order to reduce memory usage, we can map from a normalized DN string to a List of roles. The normalized string is actually how the DN class determines equality with another DN and we can drop the overhead of needing to keep all of the other objects in memory. Additionally the use of a List provides memory savings as each HashSet is backed by a HashMap, which consumes a great deal more memory than an appropriately sized ArrayList. The uniqueness we get from a Set is maintained by first building a set when parsing the file and then converting to a list upon completion. Closes #34237
kcm
pushed a commit
that referenced
this pull request
Oct 30, 2018
The `DnRoleMapper` class is used to map distinguished names of groups and users to role names. This mapper builds in an internal map that maps from a `com.unboundid.ldap.sdk.DN` to a `Set<String>`. In cases where a lot of distinct DNs are mapped to roles, this can consume quite a bit of memory. The majority of the memory is consumed by the DN object. For example, a 94 character DN that has 9 relative DNs (RDN) will retain 4KB of memory, whereas the String itself consumes less than 250 bytes. In order to reduce memory usage, we can map from a normalized DN string to a List of roles. The normalized string is actually how the DN class determines equality with another DN and we can drop the overhead of needing to keep all of the other objects in memory. Additionally the use of a List provides memory savings as each HashSet is backed by a HashMap, which consumes a great deal more memory than an appropriately sized ArrayList. The uniqueness we get from a Set is maintained by first building a set when parsing the file and then converting to a list upon completion. Closes #34237
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
DnRoleMapperclass is used to map distinguished names of groupsand users to role names. This mapper builds in an internal map that
maps from a
com.unboundid.ldap.sdk.DNto aSet<String>. In caseswhere a lot of distinct DNs are mapped to roles, this can consume quite
a bit of memory. The majority of the memory is consumed by the DN
object. For example, a 94 character DN that has 9 relative DNs (RDN)
will retain 4KB of memory, whereas the String itself consumes less than
250 bytes.
In order to reduce memory usage, we can map from a normalized DN string
to a List of roles. The normalized string is actually how the DN class
determines equality with another DN and we can drop the overhead of
needing to keep all of the other objects in memory. Additionally the
use of a List provides memory savings as each HashSet is backed by a
HashMap, which consumes a great deal more memory than an appropriately
sized ArrayList. The uniqueness we get from a Set is maintained by
first building a set when parsing the file and then converting to a
list upon completion.
Closes #34237