Add SAML IdP plugin for internal use #54046

tvernum · 2020-03-24T01:21:26Z

This change merges the "feature-internal-idp" branch into Elasticsearch.

This introduces a small identity-provider plugin as a child of the x-pack module.
This allows ES to act as a SAML IdP, for users who are authenticated against the
Elasticsearch cluster.

This feature is intended for internal use within Elastic Cloud environments
and is not supported for any other use case. It falls under an enterprise license tier.

The IdP is disabled by default.

This introduces a new sub-plugin to X-Pack that implements the backend for a basic IdP built on top of the X-Pack security primitives. In this commit there is a mostly empty plugin implementation and a class the implements the minimum necessary to generate a SAML 2.0 authentication "Response".

Adds a new "SamlObjectSigner" class that can sign messages such as authn Response objects

Necessary rest handler and transport action for consuming an SP EntityID and producing a SAML response for the authenticating user. TransportSamlInitiateSingleSignOnAction needs to be adjusted to take advantage of the secondary authentication handling logic once this is available.

Adds a new UserPrivilegeResolver class that determines whether a user has access to a service, and with what groups, based on an application resource model that represents the service.

Compilation of the feature-internal-idp branch was broken due to conflicting merges.

This adds low level classes for reading & writing documents that model a SAML Service Provider in the IdP. (These classes are not currently used)

Triggers the installation of the index template automatically due to cluster state changes, or before writing a SP doc

ApacheXMLSecurityInitializer (a dependency of OpenSAML) attempts to set the "org.apache.xml.security.ignoreLineBreaks" system property if it doesn't exist

- Expose an API that consumes (possibly signed) AuthnRequests as defined by the HTTP-Redirect binding. - Process AuthnRequests, validate and parse them into a minimum set of information to be used for subsequent API calls to get a SAML Response

Create the SAML responses for the principal that we should. The credentials of this user need to have been passed in the es-secondary-authorization header.

This change adds an API for generating SAML metadata for the configured Identity Provider. All parameters that can contribute to the metadata, from signing keys to Organization and contact details are all configured in node settings. The caller of the API needs to define the SP that will consume this metadata as the values included can potentially be different, depending on the registered SP configuration and the IDP policy.

This change adds a new PUT /_idp/saml/sp/{entity_id} endpoint that registers (or updates) a Service Provider in the IDP.

This prepares for reading the real Service Provider data from the saml index. We intend to support a large number of ServiceProviders within a single cluster, so it is not reasible to keep them all cached in memory. Since the read from the underyling index will be asynchronous, all methods that make use of a service provider also need to be asychronous.

This change adds a `SamlServiceProviderResolver` that can build a `SamlServiceProvider` object for a given entity-id, based on the document model stored in `SamlServiceProviderDocument`. This resolver includes a cache that is aware of document versioning (seqNo, primaryTerm) and avoids JSON parsing and object construction if the document has not changed since last use.

The SamlIdentityProvider / CloudIdp classes had evolved gradually from their initial commit to the point where they no longer were quite the right fit for the project. This change merges the class and interface, and migrates some of the instantiation logic that had been split between the Plugin and the class into a separate Builder object. We also move the Contact and Organisation information from the metadata builder into the IdentityProvider class so that the class is self-contained.

Updates SamlServiceProviderDocument to use the new read/writeOptionalVLong methods from #53145

The Identity Provider has a feature to release a standard set of attributes to a Service Provider. One of those attributes is intended to represent the groups or roles to which the user belongs. From an Identity Provider point of view, there is no concrete definition of what these attributes represent or should be used for - the Service Provider may treat them as "groups" (a named collection of users) or "roles" (a named representation of the user's access rights). Terminology across IdPs varies, but we have decided that within the ES IdP, we will refer to them as "roles". This change replaces any use of the term "groups" with "roles".

This change hooks the SamlIdentityProvider class up to the SamlServiceProviderResolver so that we return real registered service providers during SSO handling

elasticmachine · 2020-03-24T01:21:28Z

Pinging @elastic/es-security (:Security/Security)

jasontedor · 2020-03-24T03:10:04Z

test/framework/src/main/java/org/elasticsearch/test/SerializationTestUtils.java

+/**
+ * Utility methods for assisting with testing {@link Writeable} object.
+ */
+public class SerializationTestUtils {


This is a drive-by comment. I can't find any uses of this? And also, I think that AbstractWireSerializingTestCase provides roughly the same functionality?

It's used in

idp.action.DeleteSamlServiceProviderRequestTests

idp.action.PutSamlServiceProviderRequestTests

idp.saml.sp.SamlServiceProviderDocumentTests

idp.saml.support.SamlAuthenticationStateTests

You're right, I can switch those to use the existing methods.
I was sure we had something like that, but I just couldn't find it, so I figured I must have remembered incorrectly.

Strange, the IDE showed me there were not any uses. Either way, thanks for switching over.

This change fixes a radomised failure in TransportSamlInitiateSingleSignOnRequestTests Under some seeds (e.g. 856F19C80D0A73B6) the roles array would be empty, which would mean that the generated SAML response would not contain a roles attribute.

This change replaces the SerializationTestUtils class with methods from ESTestCase

jkakavas

(after we merge #54053 )

Needed since RestorableContextClassLoader was moved to core (from security)

This change merges the "feature-internal-idp" branch into Elasticsearch. This introduces a small identity-provider plugin as a child of the x-pack module. This allows ES to act as a SAML IdP, for users who are authenticated against the Elasticsearch cluster. This feature is intended for internal use within Elastic Cloud environments and is not supported for any other use case. It falls under an enterprise license tier. The IdP is disabled by default. Co-authored-by: Ioannis Kakavas <[email protected]>

This change merges the "feature-internal-idp" branch into Elasticsearch. This introduces a small identity-provider plugin as a child of the x-pack module. This allows ES to act as a SAML IdP, for users who are authenticated against the Elasticsearch cluster. This feature is intended for internal use within Elastic Cloud environments and is not supported for any other use case. It falls under an enterprise license tier. The IdP is disabled by default. Co-authored-by: Ioannis Kakavas <[email protected]> Co-authored-by: Tim Vernum <[email protected]>

…ulate authing User (#61354) The API key document currently doesn't include the user's full_name or email attributes, and as a result, when those attributes return `null` when hitting `GET`ing `/_security/_authenticate`, and in the SAML response from the [IdP Plugin](#54046). This changeset adds those fields to the document and extracts them to fill in the User when authenticating. They're effectively going to be a snapshot of the User from when the key was created, but this is in line with roles and metadata as well. Signed-off-by: lloydmeta <[email protected]>

…hem to populate authing User (elastic#61354) The API key document currently doesn't include the user's full_name or email attributes, and as a result, when those attributes return `null` when hitting `GET`ing `/_security/_authenticate`, and in the SAML response from the [IdP Plugin](elastic#54046). This changeset adds those fields to the document and extracts them to fill in the User when authenticating. They're effectively going to be a snapshot of the User from when the key was created, but this is in line with roles and metadata as well. Signed-off-by: lloydmeta <[email protected]>

…se them to populate authing User (elastic#61354) The API key document currently doesn't include the user's full_name or email attributes, and as a result, when those attributes return `null` when hitting `GET`ing `/_security/_authenticate`, and in the SAML response from the [IdP Plugin](elastic#54046). This changeset adds those fields to the document and extracts them to fill in the User when authenticating. They're effectively going to be a snapshot of the User from when the key was created, but this is in line with roles and metadata as well. Signed-off-by: lloydmeta <[email protected]>

…hem to populate authing User (#61354) (#61403) The API key document currently doesn't include the user's full_name or email attributes, and as a result, when those attributes return `null` when hitting `GET`ing `/_security/_authenticate`, and in the SAML response from the [IdP Plugin](#54046). This changeset adds those fields to the document and extracts them to fill in the User when authenticating. They're effectively going to be a snapshot of the User from when the key was created, but this is in line with roles and metadata as well. Signed-off-by: lloydmeta <[email protected]>

…se them to populate authing User (#61354) (#61404) The API key document currently doesn't include the user's full_name or email attributes, and as a result, when those attributes return `null` when hitting `GET`ing `/_security/_authenticate`, and in the SAML response from the [IdP Plugin](#54046). This changeset adds those fields to the document and extracts them to fill in the User when authenticating. They're effectively going to be a snapshot of the User from when the key was created, but this is in line with roles and metadata as well. Signed-off-by: lloydmeta <[email protected]>

tvernum and others added 30 commits January 30, 2020 22:00

Merge remote-tracking branch 'origin/master' into feature-internal-idp

aa9e541

Add IDP Configuration settings and related tests (#51682)

207dce8

Merge remote-tracking branch 'origin/master' into feature-internal-idp

789b4e4

Merge branch 'master' into feature-internal-idp

aac2e43

Add signing support to SAML IdP (#51517)

bf2dee1

Adds a new "SamlObjectSigner" class that can sign messages such as authn Response objects

Merge branch 'master' into feature-internal-idp

eb88cdd

Merge branch 'master' into feature-internal-idp

726b587

Resolve application privileges to SP access (#52250)

749c3c4

Adds a new UserPrivilegeResolver class that determines whether a user has access to a service, and with what groups, based on an application resource model that represents the service.

Merge branch 'master' into feature-internal-idp

97eed15

Fix compile / merge issue (#52643)

c84a9c0

Compilation of the feature-internal-idp branch was broken due to conflicting merges.

Write SAML SP data to a dedicated index (#52401)

760b39a

This adds low level classes for reading & writing documents that model a SAML Service Provider in the IdP. (These classes are not currently used)

Automatically install SAML SP index template (#52706)

6151bb8

Triggers the installation of the index template automatically due to cluster state changes, or before writing a SP doc

Update IdP plugin security policy (#52740)

8188616

ApacheXMLSecurityInitializer (a dependency of OpenSAML) attempts to set the "org.apache.xml.security.ignoreLineBreaks" system property if it doesn't exist

Handle incoming AuthnRequests (#52018)

eaa2e4f

- Expose an API that consumes (possibly signed) AuthnRequests as defined by the HTTP-Redirect binding. - Process AuthnRequests, validate and parse them into a minimum set of information to be used for subsequent API calls to get a SAML Response

Create SAML Responses for secondary authN (#52805)

3b22e1e

Create the SAML responses for the principal that we should. The credentials of this user need to have been passed in the es-secondary-authorization header.

Merge branch 'master' into feature-internal-idp

bfe6d43

Merge branch 'master' into feature-internal-idp

443b805

Add REST endpoint to register a new SAML SP (#52936)

0cd3ba9

This change adds a new PUT /_idp/saml/sp/{entity_id} endpoint that registers (or updates) a Service Provider in the IDP.

Merge branch 'master' into feature-internal-idp

34c1419

Use optionalVLong (#53149)

498bb7f

Updates SamlServiceProviderDocument to use the new read/writeOptionalVLong methods from #53145

Merge remote-tracking branch 'origin/master' into feature-internal-idp

4e38c4e

Resolve real (indexed) SPs from IdentityProvider (#53211)

1874fc6

This change hooks the SamlIdentityProvider class up to the SamlServiceProviderResolver so that we return real registered service providers during SSO handling

Merge branch 'master' into feature-internal-idp

82b9e07

tvernum added the v7.7.0 label Mar 24, 2020

tvernum requested a review from jkakavas March 24, 2020 01:21

jasontedor reviewed Mar 24, 2020

View reviewed changes

tvernum added 2 commits March 24, 2020 17:48

Fix test to always have roles (#54048)

070698b

This change fixes a radomised failure in TransportSamlInitiateSingleSignOnRequestTests Under some seeds (e.g. 856F19C80D0A73B6) the roles array would be empty, which would mean that the generated SAML response would not contain a roles attribute.

Remove SerializationTestUtils (#54052)

a403ae2

This change replaces the SerializationTestUtils class with methods from ESTestCase

jkakavas approved these changes Mar 24, 2020

View reviewed changes

Wrap RestorableContextClassLoader with doPrivileged (#54053)

ab53868

Needed since RestorableContextClassLoader was moved to core (from security)

jkakavas merged commit c1d8341 into master Mar 24, 2020

jkakavas mentioned this pull request Mar 24, 2020

Add SAML IdP plugin for internal use (#54046) #54124

Merged

colings86 deleted the feature-internal-idp branch May 27, 2020 07:39

lloydmeta mentioned this pull request Aug 20, 2020

[API keys] Add full_name and email to API key doc and use them to populate authing User #61354

Merged

lloydmeta mentioned this pull request Aug 21, 2020

[Backport to 7.x][API keys] Add full_name and email to API key doc and use them to populate authing User #61403

Merged

lloydmeta mentioned this pull request Aug 21, 2020

[Backport 7.9][API keys] Add full_name and email to API key doc and use them to populate authing User (#61354) #61404

Merged

jakelandis added v8.0.0-alpha1 and removed v8.0.0 labels Jul 26, 2021

tvernum mentioned this pull request Sep 21, 2021

Allow Elasticsearch to be an Identity Provider #48962

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add SAML IdP plugin for internal use #54046

Add SAML IdP plugin for internal use #54046

Uh oh!

tvernum commented Mar 24, 2020

Uh oh!

elasticmachine commented Mar 24, 2020

Uh oh!

jasontedor Mar 24, 2020

Uh oh!

tvernum Mar 24, 2020

Uh oh!

jasontedor Mar 24, 2020

Uh oh!

jkakavas left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Add SAML IdP plugin for internal use #54046

Add SAML IdP plugin for internal use #54046

Uh oh!

Conversation

tvernum commented Mar 24, 2020

Uh oh!

elasticmachine commented Mar 24, 2020

Uh oh!

jasontedor Mar 24, 2020

Choose a reason for hiding this comment

Uh oh!

tvernum Mar 24, 2020

Choose a reason for hiding this comment

Uh oh!

jasontedor Mar 24, 2020

Choose a reason for hiding this comment

Uh oh!

jkakavas left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants