diff --git a/x-pack/docs/en/security/authentication/active-directory-realm.asciidoc b/x-pack/docs/en/security/authentication/active-directory-realm.asciidoc new file mode 100644 index 0000000000000..c0461f4f33885 --- /dev/null +++ b/x-pack/docs/en/security/authentication/active-directory-realm.asciidoc @@ -0,0 +1,78 @@ +[role="xpack"] +[[active-directory-realm]] +=== Active Directory user authentication + +You can configure {security} to communicate with Active Directory to authenticate +users. To integrate with Active Directory, you configure an `active_directory` +realm and map Active Directory users and groups to {security} roles in the +<>. + +See {ref}/configuring-ad-realm.html[Configuring an Active Directory Realm]. + +{security} uses LDAP to communicate with Active Directory, so `active_directory` +realms are similar to <>. Like LDAP directories, +Active Directory stores users and groups hierarchically. The directory's +hierarchy is built from containers such as the _organizational unit_ (`ou`), +_organization_ (`o`), and _domain controller_ (`dc`). + +The path to an entry is a _Distinguished Name_ (DN) that uniquely identifies a +user or group. User and group names typically have attributes such as a +_common name_ (`cn`) or _unique ID_ (`uid`). A DN is specified as a string, for +example `"cn=admin,dc=example,dc=com"` (white spaces are ignored). + +{security} only supports Active Directory security groups. You cannot map +distribution groups to roles. + +NOTE: When you use Active Directory for authentication, the username entered by + the user is expected to match the `sAMAccountName` or `userPrincipalName`, + not the common name. + +The Active Directory realm authenticates users using an LDAP bind request. After +authenticating the user, the realm then searches to find the user's entry in +Active Directory. Once the user has been found, the Active Directory realm then +retrieves the user's group memberships from the `tokenGroups` attribute on the +user's entry in Active Directory. + +[[ad-load-balancing]] +==== Load balancing and failover +The `load_balance.type` setting can be used at the realm level to configure how +{security} should interact with multiple Active Directory servers. Two modes of +operation are supported: failover and load balancing. + +See {ref}/security-settings.html#load-balancing[Load Balancing and Failover Settings]. + +[[ad-settings]] +==== Active Directory realm settings + +See {ref}/security-settings.html#ref-ad-settings[Active Directory Realm Settings]. + +[[mapping-roles-ad]] +==== Mapping Active Directory users and groups to roles + +See {ref}/configuring-ad-realm.html[Configuring an Active Directory realm]. + +[[ad-user-metadata]] +==== User metadata in Active Directory realms +When a user is authenticated via an Active Directory realm, the following +properties are populated in the user's _metadata_: + +|======================= +| Field | Description +| `ldap_dn` | The distinguished name of the user. +| `ldap_groups` | The distinguished name of each of the groups that were + resolved for the user (regardless of whether those + groups were mapped to a role). +|======================= + +This metadata is returned in the +{ref}/security-api-authenticate.html[authenticate API] and can be used with +<> in roles. + +Additional metadata can be extracted from the Active Directory server by configuring +the `metadata` setting on the Active Directory realm. + +[[active-directory-ssl]] +==== Setting up SSL between Elasticsearch and Active Directory + +See +{ref}/configuring-tls.html#tls-active-directory[Encrypting communications between {es} and Active Directory]. diff --git a/x-pack/docs/en/security/authentication/built-in-users.asciidoc b/x-pack/docs/en/security/authentication/built-in-users.asciidoc new file mode 100644 index 0000000000000..9b98f685bd85d --- /dev/null +++ b/x-pack/docs/en/security/authentication/built-in-users.asciidoc @@ -0,0 +1,157 @@ +[role="xpack"] +[[built-in-users]] +=== Built-in users + +{security} provides built-in user credentials to help you get up and running. +These users have a fixed set of privileges and cannot be authenticated until their +passwords have been set. The `elastic` user can be used to +<>. + +`elastic`:: A built-in _superuser_. See <>. +`kibana`:: The user Kibana uses to connect and communicate with Elasticsearch. +`logstash_system`:: The user Logstash uses when storing monitoring information in Elasticsearch. +`beats_system`:: The user the Beats use when storing monitoring information in Elasticsearch. + + +[float] +[[built-in-user-explanation]] +==== How the built-in users work +These built-in users are stored within a special `.security` index managed by +{security}. +This means that, if the password is changed, or a user is disabled, then that +change is automatically reflected on each node in the cluster. It also means +that if your `.security` index is deleted, or restored from a snapshot, then +any changes you have applied will be lost. + +Although they share the same API, the built-in users are separate and distinct +from users managed by the <>. Disabling the native +realm will not have any effect on the built-in users. The built-in users can +be disabled individually, using the +{ref}/security-api-disable-user.html[disable users API]. + +[float] +[[bootstrap-elastic-passwords]] +==== The Elastic bootstrap password + +When you install {es}, if the `elastic` user does not already have a password, +it uses a default bootstrap password. The bootstrap password is a transient +password that enables you to run the tools that set all the built-in user passwords. + +By default, the bootstrap password is derived from a randomized `keystore.seed` +setting, which is added to the keystore during installation. You do not need +to know or change this bootstrap password. If you have defined a +`bootstrap.password` setting in the keystore, however, that value is used instead. +For more information about interacting with the keystore, see +{ref}/secure-settings.html[Secure Settings]. + +NOTE: After you <>, +in particular for the `elastic` user, there is no further use for the bootstrap +password. + +[float] +[[set-built-in-user-passwords]] +==== Setting built-in user passwords + +You must set the passwords for all built-in users. + +The +elasticsearch-setup-passwords+ tool is the simplest method to set the +built-in users' passwords for the first time. It uses the `elastic` user's +bootstrap password to run user management API requests. For example, you can run +the command in an "interactive" mode, which prompts you to enter new passwords +for the `elastic`, `kibana`, `logstash_system`, and `beats_system` users: + +[source,shell] +-------------------------------------------------- +bin/elasticsearch-setup-passwords interactive +-------------------------------------------------- + +For more information about the command options, see +{ref}/setup-passwords.html[elasticsearch-setup-passwords]. + +IMPORTANT: After you set a password for the `elastic` user, the bootstrap +password is no longer valid; you cannot run the `elasticsearch-setup-passwords` +command a second time. + +Alternatively, you can set the initial passwords for the built-in users by using +the *Management > Users* page in {kib} or the +{ref}/security-api-change-password.html[Change Password API]. These methods are +more complex. You must supply the `elastic` user and its bootstrap password to +log into {kib} or run the API. This requirement means that you cannot use the +default bootstrap password that is derived from the `keystore.seed` setting. +Instead, you must explicitly set a `bootstrap.password` setting in the keystore +before you start {es}. For example, the following command prompts you to enter a +new bootstrap password: + +[source,shell] +---------------------------------------------------- +bin/elasticsearch-keystore add "bootstrap.password" +---------------------------------------------------- + +You can then start {es} and {kib} and use the `elastic` user and bootstrap +password to log into {kib} and change the passwords. Alternatively, you can +submit Change Password API requests for each built-in user. These methods are +better suited for changing your passwords after the initial setup is complete, +since at that point the bootstrap password is no longer required. + +[float] +[[add-built-in-user-passwords]] +==== Adding Built-in User Passwords To {kib}, Logstash, and Beats + +After the `kibana` user password is set, you need to update the {kib} server +with the new password by setting `elasticsearch.password` in the `kibana.yml` +configuration file: + +[source,yaml] +----------------------------------------------- +elasticsearch.password: kibanapassword +----------------------------------------------- + +The `logstash_system` user is used internally within Logstash when +monitoring is enabled for Logstash. + +To enable this feature in Logstash, you need to update the Logstash +configuration with the new password by setting `xpack.monitoring.elasticsearch.password` in +the `logstash.yml` configuration file: + +[source,yaml] +---------------------------------------------------------- +xpack.monitoring.elasticsearch.password: logstashpassword +---------------------------------------------------------- + +If you have upgraded from an older version of Elasticsearch, +the `logstash_system` user may have defaulted to _disabled_ for security reasons. +Once the password has been changed, you can enable the user via the following API call: + +[source,js] +--------------------------------------------------------------------- +PUT _xpack/security/user/logstash_system/_enable +--------------------------------------------------------------------- +// CONSOLE + +The `beats_system` user is used internally within Beats when monitoring is +enabled for Beats. + +To enable this feature in Beats, you need to update the configuration for each +of your beats to reference the correct username and password. For example: + +[source,yaml] +---------------------------------------------------------- +xpack.monitoring.elasticsearch.username: beats_system +xpack.monitoring.elasticsearch.password: beatspassword +---------------------------------------------------------- + +If you have upgraded from an older version of {es}, then you may not have set a +password for the `beats_system` user. If this is the case, then you should use +the *Management > Users* page in {kib} or the +{ref}/security-api-change-password.html[Change Password API] to set a password +for this user. + +[float] +[[disabling-default-password]] +==== Disabling default password functionality +[IMPORTANT] +============================================================================= +This setting is deprecated. The elastic user no longer has a default password. +The password must be set before the user can be used. +See <>. +============================================================================= diff --git a/x-pack/docs/en/security/authentication/file-realm.asciidoc b/x-pack/docs/en/security/authentication/file-realm.asciidoc new file mode 100644 index 0000000000000..1161778bb801c --- /dev/null +++ b/x-pack/docs/en/security/authentication/file-realm.asciidoc @@ -0,0 +1,27 @@ +[role="xpack"] +[[file-realm]] +=== File-based user authentication + +You can manage and authenticate users with the built-in `file` realm. +With the `file` realm, users are defined in local files on each node in the cluster. + +IMPORTANT: As the administrator of the cluster, it is your responsibility to + ensure the same users are defined on every node in the cluster. + {security} does not deliver any mechanism to guarantee this. + +The `file` realm is primarily supported to serve as a fallback/recovery realm. It +is mostly useful in situations where all users locked themselves out of the system +(no one remembers their username/password). In this type of scenarios, the `file` +realm is your only way out - you can define a new `admin` user in the `file` realm +and use it to log in and reset the credentials of all other users. + +IMPORTANT: When you configure realms in `elasticsearch.yml`, only the +realms you specify are used for authentication. To use the +`file` realm as a fallback, you must include it in the realm chain. + +To define users, {security} provides the {ref}/users-command.html[users] +command-line tool. This tool enables you to add and remove users, assign user +roles, and manage user passwords. + +For more information, see +{ref}/configuring-file-realm.html[Configuring a file realm]. diff --git a/x-pack/docs/en/security/authentication/index.asciidoc b/x-pack/docs/en/security/authentication/index.asciidoc new file mode 100644 index 0000000000000..120241e990c0e --- /dev/null +++ b/x-pack/docs/en/security/authentication/index.asciidoc @@ -0,0 +1,20 @@ + +include::overview.asciidoc[] +include::built-in-users.asciidoc[] +include::internal-users.asciidoc[] +include::realms.asciidoc[] +include::active-directory-realm.asciidoc[] +include::file-realm.asciidoc[] +include::ldap-realm.asciidoc[] +include::native-realm.asciidoc[] +include::pki-realm.asciidoc[] +include::saml-realm.asciidoc[] +include::kerberos-realm.asciidoc[] + +include::{xes-repo-dir}/security/authentication/custom-realm.asciidoc[] + +include::{xes-repo-dir}/security/authentication/anonymous-access.asciidoc[] + +include::{xes-repo-dir}/security/authentication/user-cache.asciidoc[] + +include::{xes-repo-dir}/security/authentication/saml-guide.asciidoc[] diff --git a/x-pack/docs/en/security/authentication/internal-users.asciidoc b/x-pack/docs/en/security/authentication/internal-users.asciidoc new file mode 100644 index 0000000000000..77571a53a56f3 --- /dev/null +++ b/x-pack/docs/en/security/authentication/internal-users.asciidoc @@ -0,0 +1,13 @@ +[role="xpack"] +[[internal-users]] +=== Internal users + +{security} has three _internal_ users (`_system`, `_xpack`, and `_xpack_security`) +that are responsible for the operations that take place inside an {es} cluster. + +These users are only used by requests that originate from within the cluster. +For this reason, they cannot be used to authenticate against the API and there +is no password to manage or reset. + +From time-to-time you may find a reference to one of these users inside your +logs, including <>. diff --git a/x-pack/docs/en/security/authentication/kerberos-realm.asciidoc b/x-pack/docs/en/security/authentication/kerberos-realm.asciidoc new file mode 100644 index 0000000000000..956f7f9e48f85 --- /dev/null +++ b/x-pack/docs/en/security/authentication/kerberos-realm.asciidoc @@ -0,0 +1,63 @@ +[role="xpack"] +[[kerberos-realm]] +=== Kerberos authentication + +You can configure {security} to support Kerberos V5 authentication, +an industry standard protocol to authenticate users in {es}. + +NOTE: You cannot use the Kerberos realm to authenticate users in {kib} +and on the transport network layer. + +To authenticate users with Kerberos, you need to +{ref}/configuring-kerberos-realm.html[configure a Kerberos realm] and +<>. +For more information on realm settings, see +{ref}/security-settings.html#ref-kerberos-settings[Kerberos realm settings]. + +[[kerberos-terms]] +==== Key concepts + +There are a few terms and concepts that you'll encounter when you're setting up +Kerberos realms: + +_kdc_:: +Key Distribution Center. A service that issues Kerberos tickets. + +_principal_:: +A Kerberos principal is a unique identity to which Kerberos can assign +tickets. It can be used to identify a user or a service provided by a +server. ++ +-- +Kerberos V5 principal names are of format `primary/instance@REALM`, where +`primary` is a user name. + +`instance` is an optional string that qualifies the primary and is separated +by a slash(`/`) from the primary. For a user, usually it is not used; for +service hosts, it is the fully qualified domain name of the host. + +`REALM` is the Kerberos realm. Usually it is is the domain name in upper case. +An example of a typical user principal is `user@ES.DOMAIN.LOCAL`. An example of +a typical service principal is `HTTP/es.domain.local@ES.DOMAIN.LOCAL`. +-- + +_realm_:: +Realms define the administrative boundary within which the authentication server +has authority to authenticate users and services. + +_keytab_:: +A file that stores pairs of principals and encryption keys. + +IMPORTANT: Anyone with read permissions to this file can use the +credentials in the network to access other services so it is important +to protect it with proper file permissions. + +_krb5.conf_:: +A file that contains Kerberos configuration information such as the default realm +name, the location of Key distribution centers (KDC), realms information, +mappings from domain names to Kerberos realms, and default configurations for +realm session key encryption types. + +_ticket granting ticket (TGT)_:: +A TGT is an authentication ticket generated by the Kerberos authentication +server. It contains an encrypted authenticator. \ No newline at end of file diff --git a/x-pack/docs/en/security/authentication/ldap-realm.asciidoc b/x-pack/docs/en/security/authentication/ldap-realm.asciidoc new file mode 100644 index 0000000000000..355cc012df033 --- /dev/null +++ b/x-pack/docs/en/security/authentication/ldap-realm.asciidoc @@ -0,0 +1,86 @@ +[role="xpack"] +[[ldap-realm]] +=== LDAP user authentication + +You can configure {security} to communicate with a Lightweight Directory Access +Protocol (LDAP) server to authenticate users. To integrate with LDAP, you +configure an `ldap` realm and map LDAP groups to user roles in the +<>. + +LDAP stores users and groups hierarchically, similar to the way folders are +grouped in a file system. An LDAP directory's hierarchy is built from containers +such as the _organizational unit_ (`ou`), _organization_ (`o`), and +_domain controller_ (`dc`). + +The path to an entry is a _Distinguished Name_ (DN) that uniquely identifies a +user or group. User and group names typically have attributes such as a +_common name_ (`cn`) or _unique ID_ (`uid`). A DN is specified as a string, +for example `"cn=admin,dc=example,dc=com"` (white spaces are ignored). + +The `ldap` realm supports two modes of operation, a user search mode +and a mode with specific templates for user DNs. + +[[ldap-user-search]] +==== User search mode and user DN templates mode + +See {ref}/configuring-ldap-realm.html[Configuring an LDAP Realm]. + +[[ldap-load-balancing]] +==== Load balancing and failover +The `load_balance.type` setting can be used at the realm level to configure how +{security} should interact with multiple LDAP servers. {security} supports both +failover and load balancing modes of operation. + +See {ref}/security-settings.html#load-balancing[Load Balancing and Failover Settings]. + +[[ldap-settings]] +==== LDAP realm settings + +See {ref}/security-settings.html#ref-ldap-settings[LDAP Realm Settings]. + +[[mapping-roles-ldap]] +==== Mapping LDAP groups to roles + +An integral part of a realm authentication process is to resolve the roles +associated with the authenticated user. Roles define the privileges a user has +in the cluster. + +Since with the `ldap` realm the users are managed externally in the LDAP server, +the expectation is that their roles are managed there as well. If fact, LDAP +supports the notion of groups, which often represent user roles for different +systems in the organization. + +The `ldap` realm enables you to map LDAP users to roles via their LDAP +groups, or other metadata. This role mapping can be configured via the +{ref}/security-api-put-role-mapping.html[add role mapping API] or by using a file stored +on each node. When a user authenticates with LDAP, the privileges +for that user are the union of all privileges defined by the roles to which +the user is mapped. For more information, see +{ref}/configuring-ldap-realm.html[Configuring an LDAP Realm]. + +[[ldap-user-metadata]] +==== User metadata in LDAP realms +When a user is authenticated via an LDAP realm, the following properties are +populated in the user's _metadata_: + +|======================= +| Field | Description +| `ldap_dn` | The distinguished name of the user. +| `ldap_groups` | The distinguished name of each of the groups that were + resolved for the user (regardless of whether those + groups were mapped to a role). +|======================= + +This metadata is returned in the +{ref}/security-api-authenticate.html[authenticate API], and can be used with +<> in roles. + +Additional fields can be included in the user's metadata by configuring +the `metadata` setting on the LDAP realm. This metadata is available for use +with the <> or in +<>. + +[[ldap-ssl]] +==== Setting up SSL Between Elasticsearch and LDAP + +See {ref}/configuring-tls.html#tls-ldap[Encrypting Communications Between {es} and LDAP]. diff --git a/x-pack/docs/en/security/authentication/native-realm.asciidoc b/x-pack/docs/en/security/authentication/native-realm.asciidoc new file mode 100644 index 0000000000000..738d1f255666c --- /dev/null +++ b/x-pack/docs/en/security/authentication/native-realm.asciidoc @@ -0,0 +1,31 @@ +[role="xpack"] +[[native-realm]] +=== Native user authentication + +The easiest way to manage and authenticate users is with the internal `native` +realm. You can use the REST APIs or Kibana to add and remove users, assign user roles, and +manage user passwords. + +[[native-realm-configuration]] +[float] +==== Configuring a native realm + +See {ref}/configuring-native-realm.html[Configuring a native realm]. + +[[native-settings]] +==== Native realm settings + +See {ref}/security-settings.html#ref-native-settings[Native realm settings]. + +[[managing-native-users]] +==== Managing native users + +{security} enables you to easily manage users in {kib} on the +*Management / Security / Users* page. + +Alternatively, you can manage users through the `user` API. For more +information and examples, see {ref}/security-api.html#security-user-apis[user management APIs]. + +[[migrating-from-file]] +NOTE: To migrate file-based users to the `native` realm, use the +{ref}/migrate-tool.html[migrate tool]. diff --git a/x-pack/docs/en/security/authentication/overview.asciidoc b/x-pack/docs/en/security/authentication/overview.asciidoc new file mode 100644 index 0000000000000..0d6a58d99d7a2 --- /dev/null +++ b/x-pack/docs/en/security/authentication/overview.asciidoc @@ -0,0 +1,25 @@ +[role="xpack"] +[[setting-up-authentication]] +== User authentication + +Authentication identifies an individual. To gain access to restricted resources, +a user must prove their identity, via passwords, credentials, or some other +means (typically referred to as authentication tokens). + +The {stack} authenticates users by identifying the users behind the requests +that hit the cluster and verifying that they are who they claim to be. The +authentication process is handled by one or more authentication services called +<>. + +You can use the native support for managing and authenticating users, or +integrate with external user management systems such as LDAP and Active +Directory. + +{security} provides built-in realms such as `native`,`ldap`, `active_directory`, +`pki`, `file`, and `saml`. If none of the built-in realms meet your needs, you +can also build your own custom realm and plug it into the {stack}. + +When {security} is enabled, depending on the realms you've configured, you must +attach your user credentials to the requests sent to {es}. For example, when +using realms that support usernames and passwords you can simply attach +{wikipedia}/Basic_access_authentication[basic auth] header to the requests. diff --git a/x-pack/docs/en/security/authentication/pki-realm.asciidoc b/x-pack/docs/en/security/authentication/pki-realm.asciidoc new file mode 100644 index 0000000000000..6ce9b0e0770a4 --- /dev/null +++ b/x-pack/docs/en/security/authentication/pki-realm.asciidoc @@ -0,0 +1,21 @@ +[role="xpack"] +[[pki-realm]] +=== PKI user authentication + +You can configure {security} to use Public Key Infrastructure (PKI) certificates +to authenticate users in {es}. This requires clients to present X.509 +certificates. + +NOTE: You cannot use PKI certificates to authenticate users in {kib}. + +To use PKI in {es}, you configure a PKI realm, enable client authentication on +the desired network layers (transport or http), and map the Distinguished Names +(DNs) from the user certificates to {security} roles in the +<>. + +See {ref}/configuring-pki-realm.html[Configuring a PKI realm]. + +[[pki-settings]] +==== PKI realm settings + +See {ref}/security-settings.html#ref-pki-settings[PKI realm settings]. diff --git a/x-pack/docs/en/security/authentication/realms.asciidoc b/x-pack/docs/en/security/authentication/realms.asciidoc new file mode 100644 index 0000000000000..f49f24e7fa060 --- /dev/null +++ b/x-pack/docs/en/security/authentication/realms.asciidoc @@ -0,0 +1,128 @@ +[role="xpack"] +[[realms]] +=== Realms + +Authentication in {security} is handled by one or more authentication services +called _realms_. A _realm_ is used to resolve and authenticate users based on +authentication tokens. {security} provides the following built-in realms: + +_native_:: +An internal realm where users are stored in a dedicated {es} index. +This realm supports an authentication token in the form of username and password, +and is available by default when no realms are explicitly configured. The users +are managed via the {ref}/security-api.html#security-user-apis[user management APIs]. +See <>. + +_ldap_:: +A realm that uses an external LDAP server to authenticate the +users. This realm supports an authentication token in the form of username and +password, and requires explicit configuration in order to be used. See +<>. + +_active_directory_:: +A realm that uses an external Active Directory Server to authenticate the +users. With this realm, users are authenticated by usernames and passwords. +See <>. + +_pki_:: +A realm that authenticates users using Public Key Infrastructure (PKI). This +realm works in conjunction with SSL/TLS and identifies the users through the +Distinguished Name (DN) of the client's X.509 certificates. See <>. + +_file_:: +An internal realm where users are defined in files stored on each node in the +{es} cluster. This realm supports an authentication token in the form +of username and password and is always available. See <>. + +_saml_:: +A realm that facilitates authentication using the SAML 2.0 Web SSO protocol. +This realm is designed to support authentication through {kib} and is not +intended for use in the REST API. See <>. + +_kerberos_:: +A realm that authenticates a user using Kerberos authentication. Users are +authenticated on the basis of Kerberos tickets. See <>. + +{security} also supports custom realms. If you need to integrate with another +authentication system, you can build a custom realm plugin. For more information, +see <>. + +Realms live within a _realm chain_. It is essentially a prioritized list of +configured realms (typically of various types). The order of the list determines +the order in which the realms will be consulted. You should make sure each +configured realm has a distinct `order` setting. In the event that two or more +realms have the same `order`, they will be processed in `name` order. +During the authentication process, {security} will consult and try to +authenticate the request one realm at a time. +Once one of the realms successfully authenticates the request, the authentication +is considered to be successful and the authenticated user will be associated +with the request (which will then proceed to the authorization phase). If a realm +cannot authenticate the request, the next in line realm in the chain will be +consulted. If all realms in the chain could not authenticate the request, the +authentication is then considered to be unsuccessful and an authentication error +will be returned (as HTTP status code `401`). + +NOTE: Some systems (e.g. Active Directory) have a temporary lock-out period after + several successive failed login attempts. If the same username exists in + multiple realms, unintentional account lockouts are possible. For more + information, please see <>. + +The default realm chain contains the `native` and `file` realms. To explicitly, +configure a realm chain, you specify the chain in `elasticsearch.yml`. When you +configure a realm chain, only the realms you specify are used for authentication. +To use the `native` and `file` realms, you must include them in the chain. + +The following snippet configures a realm chain that includes the `file` and +`native` realms, as well as two LDAP realms and an Active Directory realm. + +[source,yaml] +---------------------------------------- +xpack.security.authc: + realms: + + file: + type: file + order: 0 + + native: + type: native + order: 1 + + ldap1: + type: ldap + order: 2 + enabled: false + url: 'url_to_ldap1' + ... + + ldap2: + type: ldap + order: 3 + url: 'url_to_ldap2' + ... + + ad1: + type: active_directory + order: 4 + url: 'url_to_ad' +---------------------------------------- + +As can be seen above, each realm has a unique name that identifies it and each +realm type dictates its own set of required and optional settings. That said, +there are +{ref}/security-settings.html#ref-realm-settings[settings that are common to all realms]. + +Realm types can roughly be classified in two categories: + +Internal:: Realms that are internal to Elasticsearch and don't require any + communication with external parties. They are fully managed by + {security}. There can only be a maximum of one configured realm + per internal realm type. {security} provides two internal realm + types: `native` and `file`. + +External:: Realms that require interaction with parties/components external to + {es}, typically, with enterprise grade identity management + systems. Unlike internal realms, there can be as many external realms + as one would like - each with its own unique name and configuration. + {security} provides the following external realm types: `ldap`, + `active_directory`, `saml`, `kerberos` and `pki`. diff --git a/x-pack/docs/en/security/authentication/saml-guide.asciidoc b/x-pack/docs/en/security/authentication/saml-guide.asciidoc index d55230569fa7f..079243f2a7391 100644 --- a/x-pack/docs/en/security/authentication/saml-guide.asciidoc +++ b/x-pack/docs/en/security/authentication/saml-guide.asciidoc @@ -855,6 +855,7 @@ It is possible to have one or more {kib} instances that use SAML, while other instances use basic authentication against another realm type (e.g. <> or <>). +[[saml-troubleshooting]] === Troubleshooting SAML Realm Configuration The SAML 2.0 specification offers a lot of options and flexibility for the implementers diff --git a/x-pack/docs/en/security/authentication/saml-realm.asciidoc b/x-pack/docs/en/security/authentication/saml-realm.asciidoc new file mode 100644 index 0000000000000..a55ae270a19a1 --- /dev/null +++ b/x-pack/docs/en/security/authentication/saml-realm.asciidoc @@ -0,0 +1,41 @@ +[role="xpack"] +[[saml-realm]] +=== SAML authentication +{security} supports user authentication using SAML Single Sign On. +{security} provides this support using the Web Browser SSO profile of the SAML +2.0 protocol. + +This protocol is specifically designed to support authentication via an +interactive web browser, so it does not operate as a standard authentication +realm. Instead, {security} provides features in {kib} and {es} that work +together to enable interactive SAML sessions. + +This means that the SAML realm is not suitable for use by standard REST clients. +If you configure a SAML realm for use in {kib}, you should also configure +another realm, such as the <> in your authentication +chain. + +In order to simplify the process of configuring SAML authentication within the +Elastic Stack, there is a step-by-step guide to +<>. + +The remainder of this document will describe {es} specific configuration options +for SAML realms. + +[[saml-settings]] +==== SAML realm settings + +See {ref}/security-settings.html#ref-saml-settings[SAML Realm Settings]. + +==== SAML realm signing settings + +See {ref}/security-settings.html#ref-saml-signing-settings[SAML Realm Signing Settings]. + +==== SAML realm encryption settings + +See {ref}/security-settings.html#ref-saml-encryption-settings[SAML Realm Encryption Settings]. + +==== SAML realm SSL settings + +See {ref}/security-settings.html#ref-saml-ssl-settings[SAML Realm SSL Settings]. + diff --git a/x-pack/docs/en/security/authorization/built-in-roles.asciidoc b/x-pack/docs/en/security/authorization/built-in-roles.asciidoc new file mode 100644 index 0000000000000..f0150541e1a5e --- /dev/null +++ b/x-pack/docs/en/security/authorization/built-in-roles.asciidoc @@ -0,0 +1,115 @@ +[role="xpack"] +[[built-in-roles]] +=== Built-in roles + +{security} applies a default role to all users, including +<>. The default role enables users to access +the authenticate endpoint, change their own passwords, and get information about +themselves. + +{security} also provides a set of built-in roles you can explicitly assign +to users. These roles have a fixed set of privileges and cannot be updated. + +[[built-in-roles-ingest-user]] `ingest_admin` :: +Grants access to manage *all* index templates and *all* ingest pipeline configurations. ++ +NOTE: This role does *not* provide the ability to create indices; those privileges +must be defined in a separate role. + +[[built-in-roles-kibana-dashboard]] `kibana_dashboard_only_user` :: +Grants access to the {kib} Dashboard and read-only permissions to Kibana. +This role does not have access to editing tools in {kib}. For more +information, see +{kibana-ref}/xpack-dashboard-only-mode.html[{kib} Dashboard Only Mode]. + +[[built-in-roles-kibana-system]] `kibana_system` :: +Grants access necessary for the {kib} system user to read from and write to the +{kib} indices, manage index templates, and check the availability of the {es} cluster. +This role grants read access to the `.monitoring-*` indices and read and write access +to the `.reporting-*` indices. For more information, see +{kibana-ref}/using-kibana-with-security.html[Configuring Security in {kib}]. ++ +NOTE: This role should not be assigned to users as the granted permissions may +change between releases. + +[[built-in-roles-kibana-user]] `kibana_user`:: +Grants the privileges required for users of {kib}. This role grants access to Kibana +and grants monitoring privileges for the cluster. + +[[built-in-roles-logstash-admin]] `logstash_admin` :: +Grants access to the `.logstash*` indices for managing configurations. + +[[built-in-roles-logstash-system]] `logstash_system` :: +Grants access necessary for the Logstash system user to send system-level data +(such as monitoring) to {es}. For more information, see +{logstash-ref}/ls-security.html[Configuring Security in Logstash]. ++ +NOTE: This role should not be assigned to users as the granted permissions may +change between releases. ++ +NOTE: This role does not provide access to the logstash indices and is not +suitable for use within a Logstash pipeline. + +[[built-in-roles-beats-system]] `beats_system` :: +Grants access necessary for the Beats system user to send system-level data +(such as monitoring) to {es}. ++ +NOTE: This role should not be assigned to users as the granted permissions may +change between releases. ++ +NOTE: This role does not provide access to the beats indices and is not +suitable for writing beats output to {es}. + +[[built-in-roles-ml-admin]] `machine_learning_admin`:: +Grants `manage_ml` cluster privileges and read access to the `.ml-*` indices. + +[[built-in-roles-ml-user]] `machine_learning_user`:: +Grants the minimum privileges required to view {ml} configuration, +status, and results. This role grants `monitor_ml` cluster privileges and +read access to the `.ml-notifications` and `.ml-anomalies*` indices, +which store {ml} results. + +[[built-in-roles-monitoring-user]] `monitoring_user`:: +Grants the minimum privileges required for any user of {monitoring} other than those +required to use {kib}. This role grants access to the monitoring indices and grants +privileges necessary for reading basic cluster information. Monitoring users should +also be assigned the `kibana_user` role. + +[[built-in-roles-remote-monitoring-agent]] `remote_monitoring_agent`:: +Grants the minimum privileges required for a remote monitoring agent to write data +into this cluster. + +[[built-in-roles-reporting-user]] `reporting_user`:: +Grants the specific privileges required for users of {reporting} other than those +required to use {kib}. This role grants access to the reporting indices; each +user has access to only their own reports. Reporting users should also be +assigned the `kibana_user` role and a role that grants them access to the data +that will be used to generate reports. + +[[built-in-roles-superuser]] `superuser`:: +Grants full access to the cluster, including all indices and data. A user with +the `superuser` role can also manage users and roles and +<> any other user in the system. Due to the +permissive nature of this role, take extra care when assigning it to a user. + +[[built-in-roles-transport-client]] `transport_client`:: +Grants the privileges required to access the cluster through the Java Transport +Client. The Java Transport Client fetches information about the nodes in the +cluster using the _Node Liveness API_ and the _Cluster State API_ (when +sniffing is enabled). Assign your users this role if they use the +Transport Client. ++ +NOTE: Using the Transport Client effectively means the users are granted access +to the cluster state. This means users can view the metadata over all indices, +index templates, mappings, node and basically everything about the cluster. +However, this role does not grant permission to view the data in all indices. + +[[built-in-roles-watcher-admin]] `watcher_admin`:: ++ +Grants read access to the `.watches` index, read access to the watch history and +the triggered watches index and allows to execute all watcher actions. + +[[built-in-roles-watcher-user]] `watcher_user`:: ++ +Grants read access to the `.watches` index, the get watch action and the watcher +stats. diff --git a/x-pack/docs/en/security/authorization/document-level-security.asciidoc b/x-pack/docs/en/security/authorization/document-level-security.asciidoc new file mode 100644 index 0000000000000..badbf15ed6159 --- /dev/null +++ b/x-pack/docs/en/security/authorization/document-level-security.asciidoc @@ -0,0 +1,60 @@ +[role="xpack"] +[[document-level-security]] +=== Document level security + +Document level security restricts the documents that users have read access to. +In particular, it restricts which documents can be accessed from document-based +read APIs. + +To enable document level security, you use a query to specify the documents that +each role can access. The document query is associated with a particular index +or index pattern and operates in conjunction with the privileges specified for +the indices. + +The following role definition grants read access only to documents that +belong to the `click` category within all the `events-*` indices: + +[source,js] +-------------------------------------------------- +POST /_xpack/security/role/click_role +{ + "indices": [ + { + "names": [ "events-*" ], + "privileges": [ "read" ], + "query": "{\"match\": {\"category\": \"click\"}}" + } + ] +} +-------------------------------------------------- +// CONSOLE + +NOTE: Omitting the `query` entry entirely disables document level security for + the respective indices permission entry. + +The specified `query` expects the same format as if it was defined in the +search request and supports the full {es} {ref}/query-dsl.html[Query DSL]. + +For example, the following role grants read access only to the documents whose +`department_id` equals `12`: + +[source,js] +-------------------------------------------------- +POST /_xpack/security/role/dept_role +{ + "indices" : [ + { + "names" : [ "*" ], + "privileges" : [ "read" ], + "query" : { + "term" : { "department_id" : 12 } + } + } + ] +} +-------------------------------------------------- +// CONSOLE + +NOTE: `query` also accepts queries written as string values. + +For more information, see <>. \ No newline at end of file diff --git a/x-pack/docs/en/security/authorization/field-level-security.asciidoc b/x-pack/docs/en/security/authorization/field-level-security.asciidoc new file mode 100644 index 0000000000000..1a8bc2fe2e0c8 --- /dev/null +++ b/x-pack/docs/en/security/authorization/field-level-security.asciidoc @@ -0,0 +1,230 @@ +[role="xpack"] +[[field-level-security]] +=== Field level security + +Field level security restricts the fields that users have read access to. +In particular, it restricts which fields can be accessed from document-based +read APIs. + +To enable field level security, specify the fields that each role can access +as part of the indices permissions in a role definition. Field level security is +thus bound to a well-defined set of indices (and potentially a set of +<>). + +The following role definition grants read access only to the `category`, +`@timestamp`, and `message` fields in all the `events-*` indices. + +[source,js] +-------------------------------------------------- +POST /_xpack/security/role/test_role1 +{ + "indices": [ + { + "names": [ "events-*" ], + "privileges": [ "read" ], + "field_security" : { + "grant" : [ "category", "@timestamp", "message" ] + } + } + ] +} +-------------------------------------------------- +// CONSOLE + +Access to the following meta fields is always allowed: `_id`, +`_type`, `_parent`, `_routing`, `_timestamp`, `_ttl`, `_size` and `_index`. If +you specify an empty list of fields, only these meta fields are accessible. + +NOTE: Omitting the fields entry entirely disables field level security. + +You can also specify field expressions. For example, the following +example grants read access to all fields that start with an `event_` prefix: + +[source,js] +-------------------------------------------------- +POST /_xpack/security/role/test_role2 +{ + "indices" : [ + { + "names" : [ "*" ], + "privileges" : [ "read" ], + "field_security" : { + "grant" : [ "event_*" ] + } + } + ] +} +-------------------------------------------------- +// CONSOLE + +Use the dot notations to refer to nested fields in more complex documents. For +example, assuming the following document: + +[source,js] +-------------------------------------------------- +{ + "customer": { + "handle": "Jim", + "email": "jim@mycompany.com", + "phone": "555-555-5555" + } +} +-------------------------------------------------- +// NOTCONSOLE + +The following role definition enables only read access to the customer `handle` +field: + +[source,js] +-------------------------------------------------- +POST /_xpack/security/role/test_role3 +{ + "indices" : [ + { + "names" : [ "*" ], + "privileges" : [ "read" ], + "field_security" : { + "grant" : [ "customer.handle" ] + } + } + ] +} +-------------------------------------------------- +// CONSOLE + +This is where wildcard support shines. For example, use `customer.*` to enable +only read access to the `customer` data: + +[source,js] +-------------------------------------------------- +POST /_xpack/security/role/test_role4 +{ + "indices" : [ + { + "names" : [ "*" ], + "privileges" : [ "read" ], + "field_security" : { + "grant" : [ "customer.*" ] + } + } + ] +} +-------------------------------------------------- +// CONSOLE + +You can deny permission to access fields with the following syntax: + +[source,js] +-------------------------------------------------- +POST /_xpack/security/role/test_role5 +{ + "indices" : [ + { + "names" : [ "*" ], + "privileges" : [ "read" ], + "field_security" : { + "grant" : [ "*"], + "except": [ "customer.handle" ] + } + } + ] +} +-------------------------------------------------- +// CONSOLE + +The following rules apply: + +* The absence of `field_security` in a role is equivalent to * access. +* If permission has been granted explicitly to some fields, you can specify +denied fields. The denied fields must be a subset of the fields to which +permissions were granted. +* Defining denied and granted fields implies access to all granted fields except +those which match the pattern in the denied fields. + +For example: + +[source,js] +-------------------------------------------------- +POST /_xpack/security/role/test_role6 +{ + "indices" : [ + { + "names" : [ "*" ], + "privileges" : [ "read" ], + "field_security" : { + "except": [ "customer.handle" ], + "grant" : [ "customer.*" ] + } + } + ] +} +-------------------------------------------------- +// CONSOLE + +In the above example, users can read all fields with the prefix "customer." +except for "customer.handle". + +An empty array for `grant` (for example, `"grant" : []`) means that access has +not been granted to any fields. + +When a user has several roles that specify field level permissions, the +resulting field level permissions per index are the union of the individual role +permissions. For example, if these two roles are merged: + +[source,js] +-------------------------------------------------- +POST /_xpack/security/role/test_role7 +{ + "indices" : [ + { + "names" : [ "*" ], + "privileges" : [ "read" ], + "field_security" : { + "grant": [ "a.*" ], + "except" : [ "a.b*" ] + } + } + ] +} + +POST /_xpack/security/role/test_role8 +{ + "indices" : [ + { + "names" : [ "*" ], + "privileges" : [ "read" ], + "field_security" : { + "grant": [ "a.b*" ], + "except" : [ "a.b.c*" ] + } + } + ] +} +-------------------------------------------------- +// CONSOLE + +The resulting permission is equal to: + +[source,js] +-------------------------------------------------- +{ + // role 1 + role 2 + ... + "indices" : [ + { + "names" : [ "*" ], + "privileges" : [ "read" ], + "field_security" : { + "grant": [ "a.*" ], + "except" : [ "a.b.c*" ] + } + } + ] +} +-------------------------------------------------- +// NOTCONSOLE + +NOTE: Field-level security should not be set on {ref}/alias.html[`alias`] fields. To secure a +concrete field, its field name must be used directly. + +For more information, see <>. \ No newline at end of file diff --git a/x-pack/docs/en/security/authorization/images/authorization.png b/x-pack/docs/en/security/authorization/images/authorization.png new file mode 100644 index 0000000000000..1d692f2e3a9e2 Binary files /dev/null and b/x-pack/docs/en/security/authorization/images/authorization.png differ diff --git a/x-pack/docs/en/security/authorization/index.asciidoc b/x-pack/docs/en/security/authorization/index.asciidoc new file mode 100644 index 0000000000000..c8216278c6b59 --- /dev/null +++ b/x-pack/docs/en/security/authorization/index.asciidoc @@ -0,0 +1,22 @@ + +include::overview.asciidoc[] + +include::built-in-roles.asciidoc[] + +include::{xes-repo-dir}/security/authorization/managing-roles.asciidoc[] + +include::privileges.asciidoc[] + +include::document-level-security.asciidoc[] + +include::field-level-security.asciidoc[] + +include::{xes-repo-dir}/security/authorization/alias-privileges.asciidoc[] + +include::{xes-repo-dir}/security/authorization/mapping-roles.asciidoc[] + +include::{xes-repo-dir}/security/authorization/field-and-document-access-control.asciidoc[] + +include::{xes-repo-dir}/security/authorization/run-as-privilege.asciidoc[] + +include::{xes-repo-dir}/security/authorization/custom-roles-provider.asciidoc[] diff --git a/x-pack/docs/en/security/authorization/overview.asciidoc b/x-pack/docs/en/security/authorization/overview.asciidoc new file mode 100644 index 0000000000000..85cf43416bfbb --- /dev/null +++ b/x-pack/docs/en/security/authorization/overview.asciidoc @@ -0,0 +1,77 @@ +[role="xpack"] +[[authorization]] +== User authorization + +{security} introduces the concept of _authorization_ to the {stack}. +Authorization is the process of determining whether the user behind an incoming +request is allowed to execute the request. + +This process takes place after the user is successfully identified and +<>. + +[[roles]] +[float] +=== Role-based access control + +{security} provides a role-based access control (RBAC) mechanism, which enables +you to authorize users by assigning privileges to roles and assigning roles to +users or groups. + +image::security/authorization/images/authorization.png[This image illustrates role-based access control] + +The authorization process revolves around the following constructs: + +_Secured Resource_:: +A resource to which access is restricted. Indices, aliases, documents, fields, +users, and the {es} cluster itself are all examples of secured objects. + +_Privilege_:: +A named group of one or more actions that a user may execute against a +secured resource. Each secured resource has its own sets of available privileges. +For example, `read` is an index privilege that represents all actions that enable +reading the indexed/stored data. For a complete list of available privileges +see <>. + +_Permissions_:: +A set of one or more privileges against a secured resource. Permissions can +easily be described in words, here are few examples: + * `read` privilege on the `products` index + * `manage` privilege on the cluster + * `run_as` privilege on `john` user + * `read` privilege on documents that match query X + * `read` privilege on `credit_card` field + +_Role_:: +A named set of permissions + +_User_:: +The authenticated user. + +_Group_:: +One or more groups to which a user belongs. Groups are not supported in some +realms, such as native, file, or PKI realms. + +A role has a unique name and identifies a set of permissions that translate to +privileges on resources. You can associate a user or group with an arbitrary +number of roles. When you map roles to groups, the roles of a user in that group +are the combination of the roles assigned to that group and the roles assigned +to that user. Likewise, the total set of permissions that a user has is defined +by the union of the permissions in all its roles. + +The method for assigning roles to users varies depending on which realms you use +to authenticate users. For more information, see <>. + +[[attributes]] +[float] +=== Attribute-based access control + +{security} also provides an attribute-based access control (ABAC) mechanism, +which enables you to use attributes to restrict access to documents in search +queries and aggregations. For example, you can assign attributes to users and +documents, then implement an access policy in a role definition. Users with that +role can read a specific document only if they have all the required attributes. + +For more information, see +https://www.elastic.co/blog/attribute-based-access-control-with-xpack[Document-level attribute-based access control with X-Pack 6.1]. + +//TO-DO: Add task related to configuring ABAC. \ No newline at end of file diff --git a/x-pack/docs/en/security/authorization/privileges.asciidoc b/x-pack/docs/en/security/authorization/privileges.asciidoc new file mode 100644 index 0000000000000..f54666744bc1c --- /dev/null +++ b/x-pack/docs/en/security/authorization/privileges.asciidoc @@ -0,0 +1,165 @@ +[role="xpack"] +[[security-privileges]] +=== Security privileges + +This section lists the privileges that you can assign to a role. + +[[privileges-list-cluster]] +==== Cluster privileges + +[horizontal] +`all`:: +All cluster administration operations, like snapshotting, node shutdown/restart, +settings update, rerouting, or managing users and roles. + +`manage`:: +Builds on `monitor` and adds cluster operations that change values in the cluster. +This includes snapshotting, updating settings, and rerouting. It also includes +obtaining snapshot and restore status. This privilege does not include the +ability to manage security. + +`manage_index_templates`:: +All operations on index templates. + +`manage_ingest_pipelines`:: +All operations on ingest node pipelines. + +`manage_ml`:: +All {ml} operations, such as creating and deleting {dfeeds}, jobs, and model +snapshots. ++ +-- +NOTE: {dfeeds-cap} that were created prior to version 6.2 or created when {security} +was disabled run as a system user with elevated privileges, including permission +to read all indices. Newer {dfeeds} run with the security roles of the user who created +or updated them. + +-- + +`manage_pipeline`:: +All operations on ingest pipelines. + +`manage_rollup`:: +All rollup operations, including creating, starting, stopping and deleting +rollup jobs. + +`manage_saml`:: +Enables the use of internal {es} APIs to initiate and manage SAML authentication +on behalf of other users. + +`manage_security`:: +All security related operations such as CRUD operations on users and roles and +cache clearing. + +`manage_watcher`:: +All watcher operations, such as putting watches, executing, activate or acknowledging. ++ +-- +NOTE: Watches that were created prior to version 6.1 or created when {security} +was disabled run as a system user with elevated privileges, including permission +to read and write all indices. Newer watches run with the security roles of the user +who created or updated them. + +-- + +`monitor`:: +All cluster read-only operations, like cluster health and state, hot threads, +node info, node and cluster stats, and pending cluster tasks. + +`monitor_ml`:: +All read only {ml} operations, such as getting information about {dfeeds}, jobs, +model snapshots, or results. + +`monitor_rollup`:: +All read only rollup operations, such as viewing the list of historical and +currently running rollup jobs and their capabilities. + +`monitor_watcher`:: +All read only watcher operations, such as getting a watch and watcher stats. + +`transport_client`:: +All privileges necessary for a transport client to connect. Required by the remote +cluster to enable <>. + +[[privileges-list-indices]] +==== Indices privileges + +[horizontal] +`all`:: +Any action on an index + +`create`:: +Privilege to index documents. Also grants access to the update mapping +action. ++ +-- +NOTE: This privilege does not restrict the index operation to the creation +of documents but instead restricts API use to the index API. The index API allows a user +to overwrite a previously indexed document. + +-- + +`create_index`:: +Privilege to create an index. A create index request may contain aliases to be +added to the index once created. In that case the request requires the `manage` +privilege as well, on both the index and the aliases names. + +`delete`:: +Privilege to delete documents. + +`delete_index`:: +Privilege to delete an index. + +`index`:: +Privilege to index and update documents. Also grants access to the update +mapping action. + +`manage`:: +All `monitor` privileges plus index administration (aliases, analyze, cache clear, +close, delete, exists, flush, mapping, open, force merge, refresh, settings, +search shards, templates, validate). + +`monitor`:: +All actions that are required for monitoring (recovery, segments info, index +stats and status). + +`read`:: +Read only access to actions (count, explain, get, mget, get indexed scripts, +more like this, multi percolate/search/termvector, percolate, scroll, +clear_scroll, search, suggest, tv). + +`read_cross_cluster`:: +Read only access to the search action from a <>. + +`view_index_metadata`:: +Read-only access to index metadata (aliases, aliases exists, get index, exists, field mappings, +mappings, search shards, type exists, validate, warmers, settings). This +privilege is primarily available for use by {kib} users. + +`write`:: +Privilege to perform all write operations to documents, which includes the +permission to index, update, and delete documents as well as performing bulk +operations. Also grants access to the update mapping action. + +==== Run as privilege + +The `run_as` permission enables an authenticated user to submit requests on +behalf of another user. The value can be a user name or a comma-separated list +of user names. (You can also specify users as an array of strings or a YAML +sequence.) For more information, see +<>. + +[[application-privileges]] +==== Application privileges + +Application privileges are managed within {es} and can be retrieved with the +{ref}/security-api-has-privileges.html[has privileges API] and the +{ref}/security-api-get-privileges.html[get application privileges API]. They do +not, however, grant access to any actions or resources within {es}. Their +purpose is to enable applications to represent and store their own privilege +models within {es} roles. + +To create application privileges, use the +{ref}/security-api-put-privileges.html[add application privileges API]. You can +then associate these application privileges with roles, as described in +<>. diff --git a/x-pack/docs/en/security/get-started-builtin-users.asciidoc b/x-pack/docs/en/security/get-started-builtin-users.asciidoc new file mode 100644 index 0000000000000..ad61abd6b9d7b --- /dev/null +++ b/x-pack/docs/en/security/get-started-builtin-users.asciidoc @@ -0,0 +1,26 @@ +There are built-in users that you can use for specific administrative purposes: +`elastic`, `kibana`, `logstash_system`, and `beats_system`. + +Before you can use them, you must set their passwords: + +. Restart {es}. For example, if you installed {es} with a `.tar.gz` package, run +the following command from the {es} directory: ++ +-- +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./bin/elasticsearch +---------------------------------------------------------------------- + +See {ref}/starting-elasticsearch.html[Starting {es}]. +-- + +. Set the built-in users' passwords. Run the following command from the {es} +directory: ++ +-- +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./bin/elasticsearch-setup-passwords interactive +---------------------------------------------------------------------- +-- diff --git a/x-pack/docs/en/security/get-started-enable-security.asciidoc b/x-pack/docs/en/security/get-started-enable-security.asciidoc new file mode 100644 index 0000000000000..7a09701b18ae1 --- /dev/null +++ b/x-pack/docs/en/security/get-started-enable-security.asciidoc @@ -0,0 +1,39 @@ +When you use the trial license, {security} is disabled by default. To enable it: + +. Stop {kib}. The method for starting and stopping {kib} varies depending on +how you installed it. For example, if you installed {kib} from an archive +distribution (`.tar.gz` or `.zip`), stop it by entering `Ctrl-C` on the command +line. See {kibana-ref}/start-stop.html[Starting and stopping {kib}]. + +. Stop {es}. For example, if you installed {es} from an archive distribution, +enter `Ctrl-C` on the command line. See +{ref}/stopping-elasticsearch.html[Stopping {es}]. + +. Add the `xpack.security.enabled` setting to the +`ES_PATH_CONF/elasticsearch.yml` file. ++ +-- +TIP: The `ES_PATH_CONF` environment variable contains the path for the {es} +configuration files. If you installed {es} using archive distributions (`zip` or +`tar.gz`), it defaults to `ES_HOME/config`. If you used package distributions +(Debian or RPM), it defaults to `/etc/elasticsearch`. For more information, see +{ref}/settings.html[Configuring {es}]. + +For example, add the following setting: + +[source,yaml] +---- +xpack.security.enabled: true +---- + +TIP: If you have a basic or trial license, the default value for this setting is +`false`. If you have a gold or higher license, the default value is `true`. +Therefore, it is a good idea to explicitly add this setting to avoid confusion +about whether {security} is enabled. + +-- + +When you enable {security}, basic authentication is enabled by default. To +communicate with the cluster, you must specify a username and password. +Unless you <>, all requests that don't +include a user name and password are rejected. diff --git a/x-pack/docs/en/security/get-started-kibana-users.asciidoc b/x-pack/docs/en/security/get-started-kibana-users.asciidoc new file mode 100644 index 0000000000000..f37615558b75d --- /dev/null +++ b/x-pack/docs/en/security/get-started-kibana-users.asciidoc @@ -0,0 +1,60 @@ +When {security} is enabled, users must log in to {kib} with a valid user ID and +password. + +{kib} also performs some tasks under the covers that require use of the +built-in `kibana` user. + +. Configure {kib} to use the built-in `kibana` user and the password that you +created: + +** If you don't mind having passwords visible in your configuration file, +uncomment and update the following settings in the `kibana.yml` file in your +{kib} directory: ++ +-- +TIP: If you installed {kib} using archive distributions (`zip` or +`tar.gz`), the `kibana.yml` configuration file is in `KIBANA_HOME/config`. If +you used package distributions (Debian or RPM), it's in `/etc/kibana`. For more +information, see {kibana-ref}/settings.html[Configuring {kib}]. + +For example, add the following settings: + +[source,yaml] +---- +elasticsearch.username: "kibana" +elasticsearch.password: "your_password" +---- + +Specify the password that you set with the `elasticsearch-setup-passwords` +command then save your changes to the file. +-- + +** If you prefer not to put your user ID and password in the `kibana.yml` file, +store them in a keystore instead. Run the following commands to create the {kib} +keystore and add the secure settings: ++ +-- +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./bin/kibana-keystore create +./bin/kibana-keystore add elasticsearch.username +./bin/kibana-keystore add elasticsearch.password +---------------------------------------------------------------------- + +When prompted, specify the `kibana` built-in user and its password for these +setting values. The settings are automatically applied when you start {kib}. +To learn more, see {kibana-ref}/secure-settings.html[Secure settings]. +-- + +. Restart {kib}. For example, if you installed +{kib} with a `.tar.gz` package, run the following command from the {kib} +directory: ++ +-- +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./bin/kibana +---------------------------------------------------------------------- + +See {kibana-ref}/start-stop.html[Starting and stopping {kib}]. +-- \ No newline at end of file diff --git a/x-pack/docs/en/security/get-started-security.asciidoc b/x-pack/docs/en/security/get-started-security.asciidoc new file mode 100644 index 0000000000000..a23af294bdf1b --- /dev/null +++ b/x-pack/docs/en/security/get-started-security.asciidoc @@ -0,0 +1,357 @@ +[role="xpack"] +[[security-getting-started]] +== Getting started with security + +In this tutorial, you learn how to secure a cluster by configuring users and +roles in {es}, {kib}, {ls}, and {metricbeat}. + +[float] +[[get-started-security-prerequisites]] +=== Before you begin + +. Install and configure {es}, {kib}, {ls}, and {metricbeat} as described in +{stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}]. + +. Stop {ls}. The method for starting and stopping {ls} varies depending on whether +you are running it from the command line or running it as a service. For example, +if you are running {ls} from the command line, you can stop it by entering +`Ctrl-C`. See {logstash-ref}/shutdown.html[Shutting down {ls}]. + +. Stop {metricbeat}. For example, enter `Ctrl-C` on the command line where it is +running. + +. Launch the {kib} web interface by pointing your browser to port 5601. For +example, http://127.0.0.1:5601[http://127.0.0.1:5601]. + +[role="xpack"] +[[get-started-license]] +=== Install a trial license + +include::get-started-trial.asciidoc[] + +[role="xpack"] +[[get-started-enable-security]] +=== Enable security in {es} + +include::get-started-enable-security.asciidoc[] + +NOTE: This tutorial involves a single node cluster, but if you had multiple +nodes, you would enable {security} on every node in the cluster and configure +Transport Layer Security (TLS) for internode-communication, which is beyond the +scope of this tutorial. + +[role="xpack"] +[[get-started-built-in-users]] +=== Create passwords for built-in users + +include::get-started-builtin-users.asciidoc[] + +You need these built-in users in subsequent steps, so choose passwords that you +can remember! + +NOTE: This tutorial does not use the built-in `logstash_system` and +`beats_system` users, which are typically associated with monitoring. For more +information, see +{logstash-ref}/ls-security.html#ls-monitoring-user[Configuring credentials for {ls} monitoring] +and {metricbeat-ref}/monitoring.html[Monitoring {metricbeat}]. + +[role="xpack"] +[[get-started-kibana-user]] +=== Add the built-in user to {kib} + +include::get-started-kibana-users.asciidoc[] + +[role="xpack"] +[[get-started-authentication]] +=== Configure authentication + +Now that you've set up the built-in users, you need to decide how you want to +manage all the other users. + +The {stack} _authenticates_ users to ensure that they are valid. The +authentication process is handled by _realms_. You can use one or more built-in +realms, such as the native, file, LDAP, PKI, Active Directory, or SAML realms. +Alternatively, you can create your own custom realms. In this tutorial, we'll +use a native realm. + +In general, you configure realms by adding `xpack.security.authc.realms` +settings in the `elasticsearch.yml` file. However, the native realm is available +by default when no other realms are configured. Therefore, you don't need to do +any extra configuration steps in this tutorial. You can jump straight to +creating users! + +If you want to learn more about authentication and realms, see +<>. + +[role="xpack"] +[[get-started-users]] +=== Create users + +Let's create two users in the native realm. + +. Log in to {kib} with the `elastic` built-in user. + +. Go to the *Management / Security / Users* page: ++ +-- +[role="screenshot"] +image::security/images/management-builtin-users.jpg["User management screenshot in Kibana"] + +In this example, you can see all of the built-in users. +-- + +. Click *Create user* to make a new user. For example, create a user for yourself: ++ +-- +[role="screenshot"] +image::security/images/create-user.jpg["Creating a user in Kibana"] + +You'll notice that when you create a user, you can assign it a role. Don't +choose a role yet--we'll come back to that in subsequent steps. +-- + +. Click *Create user* and create a `logstash_internal` user. ++ +-- +In {stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], +you configured {ls} to listen for {metricbeat} +input and to send the events to {es}. You therefore need to create a user +that {ls} can use to communicate with {es}. For example: + +[role="screenshot"] +image::security/images/create-logstash-user.jpg["Creating a {ls} user in {kib}"] +-- + +[role="xpack"] +[[get-started-roles]] +=== Assign roles + +By default, all users can change their own passwords, get information about +themselves, and run the `authenticate` API. If you want them to do more than +that, you need to give them one or more _roles_. + +Each role defines a specific set of actions (such as read, create, or delete) +that can be performed on specific secured resources (such as indices, aliases, +documents, fields, or clusters). To help you get up and running, there are +built-in roles. + +Go to the *Management / Security / Roles* page to see them: + +[role="screenshot"] +image::security/images/management-roles.jpg["Role management screenshot in Kibana"] + +Select a role to see more information about its privileges. For example, if you +select the `kibana_user` role, you will see that it grants `manage`, `read`, +`index`, and `delete` privileges on the `.kibana*` indices. To learn more about +these privileges, see <>. + +Let's assign the `kibana_user` role to your user. Go back to the +*Management / Security / Users* page and select your user. Add the `kibana_user` +role and save the change. For example: + +[role="screenshot"] +image::security/images/assign-role.jpg["Assigning a role to a user in Kibana"] + +This user now has the minimum privileges required to use {kib}. + +If you completed all of the steps in +{stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], you should +have {metricbeat} data stored in {es}. Let's create two roles that grant +different levels of access to that data. + +Go to the *Management / Security / Roles* page and click *Create role*. + +Create a `metricbeat_reader` role that has `read` and `view_index_metadata` +privileges on the `metricbeat-*` indices: + +[role="screenshot"] +image::security/images/create-reader-role.jpg["Creating a role in Kibana"] + +Create a `metricbeat_writer` role that has `manage_index_templates` and `monitor` +cluster privileges, as well as `write`, `delete`, and `create_index` privileges +on the `metricbeat-*` indices: + +[role="screenshot"] +image::security/images/create-writer-role.jpg["Creating another role in Kibana"] + +Now go back to the *Management / Security / Users* page and assign these roles +to the appropriate users. Assign the `metricbeat_reader` role to your personal +user. Assign the `metricbeat_writer` role to the `logstash_internal` user. + +The list of users should now contain all of the built-in users as well as the +two you created. It should also show the appropriate roles for your users: + +[role="screenshot"] +image::security/images/management-users.jpg["User management screenshot in Kibana"] + +If you want to learn more about authorization and roles, see <>. + +[role="xpack"] +[[get-started-logstash-user]] +=== Add user information in {ls} + +In order for {ls} to send data successfully to {es}, you must configure its +authentication credentials in the {ls} configuration file. + +. Configure {ls} to use the `logstash_internal` user and the password that you +created: + +** If you don't mind having passwords visible in your configuration file, add +the following `user` and `password` settings in the `demo-metrics-pipeline.conf` +file in your {ls} directory: ++ +-- +[source,ruby] +---- +... + +output { + elasticsearch { + hosts => "localhost:9200" + manage_template => false + index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" + user => "logstash_internal" <1> + password => "your_password" <2> + } +} +---- +<1> Specify the `logstash_internal` user that you created earlier in this tutorial. +<2> Specify the password that you chose for this user ID. +-- + +** If you prefer not to put your user ID and password in the configuration file, +store them in a keystore instead. ++ +-- +Run the following commands to create the {ls} +keystore and add the secure settings: + +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +set +o history +export LOGSTASH_KEYSTORE_PASS=mypassword <1> +set -o history +./bin/logstash-keystore create +./bin/logstash-keystore add ES_USER +./bin/logstash-keystore add ES_PWD +---------------------------------------------------------------------- +<1> You can optionally protect access to the {ls} keystore by storing a password +in an environment variable called `LOGSTASH_KEYSTORE_PASS`. For more information, +see {logstash-ref}/keystore.html#keystore-password[Keystore password]. + +When prompted, specify the `logstash_internal` user and its password for the +`ES_USER` and `ES_PWD` values. + +NOTE: The {ls} keystore differs from the {kib} keystore. Whereas the {kib} +keystore enables you to store `kibana.yml` settings by name, the {ls} keystore +enables you to create arbitrary names that you can reference in the {ls} +configuration. To learn more, see +{logstash-ref}/keystore.html[Secrets keystore for secure settings]. + +You can now use these `ES_USER` and `ES_PWD` keys in your configuration +file. For example, add the `user` and `password` settings in the +`demo-metrics-pipeline.conf` file as follows: + +[source,ruby] +---- +... + +output { + elasticsearch { + hosts => "localhost:9200" + manage_template => false + index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" + user => "${ES_USER}" + password => "${ES_PWD}" + } +} +---- +-- + +. Start {ls} by using the appropriate method for your environment. ++ +-- +For example, to +run {ls} from a command line, go to the {ls} directory and enter the following +command: + +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./bin/logstash -f demo-metrics-pipeline.conf +---------------------------------------------------------------------- + +To start {ls} as a service, see +{logstash-ref}/running-logstash.html[Running {ls} as a service on Debian or RPM]. +-- + +. If you were connecting directly from {metricbeat} to {es}, you would need +to configure authentication credentials for the {es} output in the {metricbeat} +configuration file. In +{stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], +however, you configured +{metricbeat} to send the data to {ls} for additional parsing, so no extra +settings are required in {metricbeat}. For more information, see +{metricbeat-ref}/securing-metricbeat.html[Securing {metricbeat}]. + +. Start {metricbeat} by using the appropriate method for your environment. ++ +-- +For example, on macOS, run the following command from the {metricbeat} directory: + +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./metricbeat -e +---------------------------------------------------------------------- + +For more methods, see {metricbeat-ref}/metricbeat-starting.html[Starting {metricbeat}]. +-- + +Wait a few minutes for new data to be sent from {metricbeat} to {ls} and {es}. + +[role="xpack"] +[[get-started-verify-users]] +=== View system metrics in {kib} + +Log in to {kib} with the user ID that has `metricbeat_reader` and `kibana_user` +roles (for example, `jdoe`). + +These roles enable the user to see the system metrics in {kib} (for example, on +the *Discover* page or in the +http://localhost:5601/app/kibana#/dashboard/Metricbeat-system-overview[{metricbeat} system overview dashboard]). + +[float] +[[gs-security-nextsteps]] +=== What's next? + +Congratulations! You've successfully set up authentication and authorization by +using the native realm. You learned how to create user IDs and roles that +prevent unauthorized access to the {stack}. + +Next, you'll want to try other features that are unlocked by your trial license, +such as {ml}. See <>. + +Later, when you're ready to increase the number of nodes in your cluster or set +up an production environment, you'll want to encrypt communications across the +{stack}. To learn how, read <>. + +For more detailed information about securing the {stack}, see: + +* {ref}/configuring-security.html[Configuring security in {es}]. Encrypt +inter-node communications, set passwords for the built-in users, and manage your +users and roles. + +* {kibana-ref}/using-kibana-with-security.html[Configuring security in {kib}]. +Set the authentication credentials in {kib} and encrypt communications between +the browser and the {kib} server. + +* {logstash-ref}/ls-security.html[Configuring security in Logstash]. Set the +authentication credentials for Logstash and encrypt communications between +Logstash and {es}. + +* <>. Configure authentication +credentials and encrypt connections to {es}. + +* <>. + +* {hadoop-ref}/security.html[Configuring {es} for Apache Hadoop to use secured transport]. + diff --git a/x-pack/docs/en/security/get-started-trial.asciidoc b/x-pack/docs/en/security/get-started-trial.asciidoc new file mode 100644 index 0000000000000..b2b9c9ad2abf7 --- /dev/null +++ b/x-pack/docs/en/security/get-started-trial.asciidoc @@ -0,0 +1,21 @@ +By default, when you install {stack} products, they apply basic licenses with no +expiration dates. To view your license in {kib}, go to **Management** and click +**License Management**. + +[role="screenshot"] +image::images/management-license.png["The License Management page in {kib}"] + +For more information about Elastic license levels, see +https://www.elastic.co/subscriptions. + +You can start a 30-day trial to try out all of the platinum features, including +{security} and {ml}. Click **Start trial** on the **License Management** page in +{kib}. + +IMPORTANT: If your cluster has already activated a trial license for the current +major version, you cannot start a new trial. For example, if you have already +activated a trial for v6.0, you cannot start a new trial until v7.0. + +At the end of the trial period, the platinum features operate in a +<>. You can revert to a basic license, extend +the trial, or purchase a subscription. diff --git a/x-pack/docs/en/security/how-security-works.asciidoc b/x-pack/docs/en/security/how-security-works.asciidoc new file mode 100644 index 0000000000000..a0f35b598e642 --- /dev/null +++ b/x-pack/docs/en/security/how-security-works.asciidoc @@ -0,0 +1,49 @@ +[role="xpack"] +[[how-security-works]] +== How security works + +An Elasticsearch cluster is typically made out of many moving parts. There are +the Elasticsearch nodes that form the cluster, and often Logstash instances, +Kibana instances, Beats agents an clients, all communicating with the it. +It should not come as a surprise that securing such clusters has many facets and +layers. + +{security} provides the means to secure the Elastic cluster on several levels: + + * <> + * <> + * Node/client authentication and channel encryption + * Auditing + +[float] +=== Node/client authentication and channel encryption + +{security} supports configuring SSL/TLS for securing the communication channels +to, from and within the cluster. This support accounts for: + + * Encryption of data transmitted over the wires + * Certificate based node authentication - preventing unauthorized nodes/clients + from establishing a connection with the cluster. + +For more information, see <>. + +{security} also enables you to <> which can +be seen as a light mechanism for node/client authentication. With IP Filtering +you can restrict the nodes and clients that can connect to the cluster based +on their IP addresses. The IP filters configuration provides whitelisting +and blacklisting of IPs, subnets and DNS domains. + + +[float] +=== Auditing +When dealing with any secure system, it is critical to have a audit trail +mechanism set in place. Audit trails log various activities/events that occur in +the system, enabling you to analyze and back track past events when things go +wrong (e.g. security breach). + +{security} provides such audit trail functionality for all nodes in the cluster. +You can configure the audit level which accounts for the type of events that are +logged. These events include failed authentication attempts, user access denied, +node connection denied, and more. + +For more information on auditing see <>. diff --git a/x-pack/docs/en/security/images/assign-role.jpg b/x-pack/docs/en/security/images/assign-role.jpg new file mode 100644 index 0000000000000..c5eefaf2a39df Binary files /dev/null and b/x-pack/docs/en/security/images/assign-role.jpg differ diff --git a/x-pack/docs/en/security/images/create-logstash-user.jpg b/x-pack/docs/en/security/images/create-logstash-user.jpg new file mode 100644 index 0000000000000..c41f9b239835f Binary files /dev/null and b/x-pack/docs/en/security/images/create-logstash-user.jpg differ diff --git a/x-pack/docs/en/security/images/create-reader-role.jpg b/x-pack/docs/en/security/images/create-reader-role.jpg new file mode 100644 index 0000000000000..a0e5ac884c577 Binary files /dev/null and b/x-pack/docs/en/security/images/create-reader-role.jpg differ diff --git a/x-pack/docs/en/security/images/create-user.jpg b/x-pack/docs/en/security/images/create-user.jpg new file mode 100644 index 0000000000000..39498a52cbb50 Binary files /dev/null and b/x-pack/docs/en/security/images/create-user.jpg differ diff --git a/x-pack/docs/en/security/images/create-writer-role.jpg b/x-pack/docs/en/security/images/create-writer-role.jpg new file mode 100644 index 0000000000000..e49d7a5c0752e Binary files /dev/null and b/x-pack/docs/en/security/images/create-writer-role.jpg differ diff --git a/x-pack/docs/en/security/images/management-builtin-users.jpg b/x-pack/docs/en/security/images/management-builtin-users.jpg new file mode 100644 index 0000000000000..d4134ce025528 Binary files /dev/null and b/x-pack/docs/en/security/images/management-builtin-users.jpg differ diff --git a/x-pack/docs/en/security/images/management-license.png b/x-pack/docs/en/security/images/management-license.png new file mode 100644 index 0000000000000..8a4c1db6a9f72 Binary files /dev/null and b/x-pack/docs/en/security/images/management-license.png differ diff --git a/x-pack/docs/en/security/images/management-roles.jpg b/x-pack/docs/en/security/images/management-roles.jpg new file mode 100644 index 0000000000000..eb3fe6006ec1e Binary files /dev/null and b/x-pack/docs/en/security/images/management-roles.jpg differ diff --git a/x-pack/docs/en/security/images/management-users.jpg b/x-pack/docs/en/security/images/management-users.jpg new file mode 100644 index 0000000000000..342d75e248d43 Binary files /dev/null and b/x-pack/docs/en/security/images/management-users.jpg differ diff --git a/x-pack/docs/en/security/index.asciidoc b/x-pack/docs/en/security/index.asciidoc new file mode 100644 index 0000000000000..6ee1b2c37598e --- /dev/null +++ b/x-pack/docs/en/security/index.asciidoc @@ -0,0 +1,114 @@ +[role="xpack"] +[[elasticsearch-security]] += Securing the {stack} + +[partintro] +-- +{security} enables you to easily secure a cluster. With {security}, +you can password-protect your data as well as implement more advanced security +measures such as encrypting communications, role-based access control, +IP filtering, and auditing. This guide describes how to configure the security +features you need, and interact with your secured cluster. + +Security protects Elasticsearch clusters by: + +* <> + with password protection, role-based access control, and IP filtering. +* <> + with message authentication and SSL/TLS encryption. +* <> + so you know who's doing what to your cluster and the data it stores. + +[float] +[[preventing-unauthorized-access]] +=== Preventing Unauthorized Access + +To prevent unauthorized access to your Elasticsearch cluster, you must have a +way to _authenticate_ users. This simply means that you need a way to validate +that a user is who they claim to be. For example, you have to make sure only +the person named _Kelsey Andorra_ can sign in as the user `kandorra`. {security} +provides a standalone authentication mechanism that enables you to +quickly password-protect your cluster. If you're already using <>, +<>, or <> to manage +users in your organization, {security} is able to integrate with those +systems to perform user authentication. + +In many cases, simply authenticating users isn't enough. You also need a way to +control what data users have access to and what tasks they can perform. {security} +enables you to _authorize_ users by assigning access _privileges_ to _roles_, +and assigning those roles to users. For example, this +<> mechanism (a.k.a RBAC) enables +you to specify that the user `kandorra` can only perform read operations on the +`events` index and can't do anything at all with other indices. + +{security} also supports <>. You can +whitelist and blacklist specific IP addresses or subnets to control network-level +access to a server. + +[float] +[[preserving-data-integrity]] +=== Preserving Data Integrity + +A critical part of security is keeping confidential data confidential. +Elasticsearch has built-in protections against accidental data loss and +corruption. However, there's nothing to stop deliberate tampering or data +interception. {security} preserves the integrity of your data by +<> to and from nodes. +For even greater protection, you can increase the <> and +<>. + + +[float] +[[maintaining-audit-trail]] +=== Maintaining an Audit Trail + +Keeping a system secure takes vigilance. By using {security} to maintain +an audit trail, you can easily see who is accessing your cluster and what they're +doing. By analyzing access patterns and failed attempts to access your cluster, +you can gain insights into attempted attacks and data breaches. Keeping an +auditable log of the activity in your cluster can also help diagnose operational +issues. + +[float] +=== Where to Go Next + +* <> + steps through how to install and start using Security for basic authentication. + +* <> + provides more information about how Security supports user authentication, + authorization, and encryption. + +* <> + shows you how to interact with an Elasticsearch cluster protected by + {security}. + +* <> + provides detailed information about the access privileges you can grant to + users, the settings you can configure for Security in `elasticsearch.yml`, + and the files where Security configuration information is stored. + +[float] +=== Have Comments, Questions, or Feedback? + +Head over to our {security-forum}[Security Discussion Forum] +to share your experience, questions, and suggestions. +-- + +include::get-started-security.asciidoc[] + +include::how-security-works.asciidoc[] + +include::authentication/index.asciidoc[] + +include::authorization/index.asciidoc[] + +include::{xes-repo-dir}/security/auditing/index.asciidoc[] + +include::{xes-repo-dir}/security/securing-communications.asciidoc[] + +include::{xes-repo-dir}/security/using-ip-filtering.asciidoc[] + +include::{xes-repo-dir}/security/tribe-clients-integrations.asciidoc[] + +include::{xes-repo-dir}/security/reference.asciidoc[] diff --git a/x-pack/docs/en/security/limitations.asciidoc b/x-pack/docs/en/security/limitations.asciidoc new file mode 100644 index 0000000000000..4597969156675 --- /dev/null +++ b/x-pack/docs/en/security/limitations.asciidoc @@ -0,0 +1,89 @@ +[role="xpack"] +[[security-limitations]] +== Security limitations + +[float] +=== Plugins + +Elasticsearch's plugin infrastructure is extremely flexible in terms of what can +be extended. While it opens up Elasticsearch to a wide variety of (often custom) +additional functionality, when it comes to security, this high extensibility level +comes at a cost. We have no control over the third-party plugins' code (open +source or not) and therefore we cannot guarantee their compliance with {security}. +For this reason, third-party plugins are not officially supported on clusters +with {security} enabled. + +[float] +=== Changes in index wildcard behavior + +Elasticsearch clusters with {security} enabled apply the `/_all` wildcard, and +all other wildcards, to the indices that the current user has privileges for, not +the set of all indices on the cluster. +While creating or retrieving aliases by providing wildcard expressions for alias names, if there are no existing authorized aliases +that match the wildcard expression provided an IndexNotFoundException is returned. + +[float] +=== Multi document APIs + +Multi get and multi term vectors API throw IndexNotFoundException when trying to access non existing indices that the user is +not authorized for. By doing that they leak information regarding the fact that the index doesn't exist, while the user is not +authorized to know anything about those indices. + +[float] +=== Filtered index aliases + +Aliases containing filters are not a secure way to restrict access to individual +documents, due to the limitations described in <>. +{security} provides a secure way to restrict access to documents through the +<> feature. + +[float] +=== Field and document level security limitations + +When a user's role enables document or field level security for an index: + +* The user cannot perform write operations: +** The update API isn't supported. +** Update requests included in bulk requests aren't supported. +* The request cache is disabled for search requests. + +When a user's role enables document level security for an index: + +* Document level security isn't applied for APIs that aren't document based. + An example is the field stats API. +* Document level security doesn't affect global index statistics that relevancy + scoring uses. So this means that scores are computed without taking the role + query into account. Note that documents not matching with the role query are + never returned. +* The `has_child` and `has_parent` queries aren't supported as query in the + role definition. The `has_child` and `has_parent` queries can be used in the + search API with document level security enabled. +* Any query that makes remote calls to fetch data to query by isn't supported. + The following queries aren't supported: +** The `terms` query with terms lookup isn't supported. +** The `geo_shape` query with indexed shapes isn't supported. +** The `percolate` query isn't supported. +* If suggesters are specified and document level security is enabled then + the specified suggesters are ignored. +* A search request cannot be profiled if document level security is enabled. + +[float] +[[alias-limitations]] +=== Index and field names can be leaked when using aliases + +Calling certain Elasticsearch APIs on an alias can potentially leak information +about indices that the user isn't authorized to access. For example, when you get +the mappings for an alias with the `_mapping` API, the response includes the +index name and mappings for each index that the alias applies to. + +Until this limitation is addressed, avoid index and field names that contain +confidential or sensitive information. + +[float] +=== LDAP realm + +The <> does not currently support the discovery of nested +LDAP Groups. For example, if a user is a member of `group_1` and `group_1` is a +member of `group_2`, only `group_1` will be discovered. However, the +<> *does* support transitive +group membership. diff --git a/x-pack/docs/en/security/troubleshooting.asciidoc b/x-pack/docs/en/security/troubleshooting.asciidoc new file mode 100644 index 0000000000000..1ed44c3067e5e --- /dev/null +++ b/x-pack/docs/en/security/troubleshooting.asciidoc @@ -0,0 +1,791 @@ +[role="xpack"] +[[security-troubleshooting]] +== Troubleshooting security +++++ +Security +++++ + +Use the information in this section to troubleshoot common problems and find +answers for frequently asked questions. + +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> + + +To get help, see <>. + +[[security-auth-failure-upgrade]] +=== Can't log in after upgrading to {version} + +*Symptoms:* + +* Native realm credentials don't work after $version upgrade or snapshot restore. +* Error message: "Security index is not on the current version - the native +realm will not be operational until the upgrade API is run on the security index" + +*Resolution:* + +You must upgrade the `.security` index to the 6.x format. For +instructions, see +{stack-ref}/upgrading-elastic-stack.html#upgrade-internal-indices[Upgrading +internal indices]. + +There are two situations where it's necessary to manually upgrade the +`.security` index: + +* After upgrading directly to {version} from 5.5 or earlier. +* After restoring a snapshot from 5.5 or earlier that contains a `.security` +index in the old format to a 6.0 cluster. + +[[security-trb-settings]] +=== Some settings are not returned via the nodes settings API + +*Symptoms:* + +* When you use the {ref}/cluster-nodes-info.html[nodes info API] to retrieve +settings for a node, some information is missing. + +*Resolution:* + +This is intentional. Some of the settings are considered to be highly +sensitive: all `ssl` settings, ldap `bind_dn`, and `bind_password`. +For this reason, we filter these settings and do not expose them via +the nodes info API rest endpoint. You can also define additional +sensitive settings that should be hidden using the +`xpack.security.hide_settings` setting. For example, this snippet +hides the `url` settings of the `ldap1` realm and all settings of the +`ad1` realm. + +[source, yaml] +------------------------------------------ +xpack.security.hide_settings: xpack.security.authc.realms.ldap1.url, +xpack.security.authc.realms.ad1.* +------------------------------------------ + +[[security-trb-roles]] +=== Authorization exceptions + +*Symptoms:* + +* I configured the appropriate roles and the users, but I still get an +authorization exception. +* I can authenticate to LDAP, but I still get an authorization exception. + + +*Resolution:* + +. Verify that the role names associated with the users match the roles defined +in the `roles.yml` file. You can use the `elasticsearch-users` tool to list all +the users. Any unknown roles are marked with `*`. ++ +-- +[source, shell] +------------------------------------------ +bin/elasticsearch-users list +rdeniro : admin +alpacino : power_user +jacknich : monitoring,unknown_role* <1> +------------------------------------------ +<1> `unknown_role` was not found in `roles.yml` + +For more information about this command, see the +{ref}/users-command.html[`elasticsearch-users` command]. +-- + +. If you are authenticating to LDAP, a number of configuration options can cause +this error. ++ +-- +|====================== +|_group identification_ | + +Groups are located by either an LDAP search or by the "memberOf" attribute on +the user. Also, If subtree search is turned off, it will search only one +level deep. See the <> for all the options. +There are many options here and sticking to the defaults will not work for all +scenarios. + +| _group to role mapping_| + +Either the `role_mapping.yml` file or the location for this file could be +misconfigured. See <> for more. + +|_role definition_| + +The role definition might be missing or invalid. + +|====================== + +To help track down these possibilities, add the following lines to the end of +the `log4j2.properties` configuration file in the `ES_PATH_CONF`: + +[source,properties] +---------------- +logger.authc.name = org.elasticsearch.xpack.security.authc +logger.authc.level = DEBUG +---------------- + +A successful authentication should produce debug statements that list groups and +role mappings. +-- + +[[security-trb-extraargs]] +=== Users command fails due to extra arguments + +*Symptoms:* + +* The `elasticsearch-users` command fails with the following message: +`ERROR: extra arguments [...] were provided`. + +*Resolution:* + +This error occurs when the `elasticsearch-users` tool is parsing the input and +finds unexpected arguments. This can happen when there are special characters +used in some of the arguments. For example, on Windows systems the `,` character +is considered a parameter separator; in other words `-r role1,role2` is +translated to `-r role1 role2` and the `elasticsearch-users` tool only +recognizes `role1` as an expected parameter. The solution here is to quote the +parameter: `-r "role1,role2"`. + +For more information about this command, see +{ref}/users-command.html[`elasticsearch-users` command]. + +[[trouble-shoot-active-directory]] +=== Users are frequently locked out of Active Directory + +*Symptoms:* + +* Certain users are being frequently locked out of Active Directory. + +*Resolution:* + +Check your realm configuration; realms are checked serially, one after another. +If your Active Directory realm is being checked before other realms and there +are usernames that appear in both Active Directory and another realm, a valid +login for one realm might be causing failed login attempts in another realm. + +For example, if `UserA` exists in both Active Directory and a file realm, and +the Active Directory realm is checked first and file is checked second, an +attempt to authenticate as `UserA` in the file realm would first attempt to +authenticate against Active Directory and fail, before successfully +authenticating against the `file` realm. Because authentication is verified on +each request, the Active Directory realm would be checked - and fail - on each +request for `UserA` in the `file` realm. In this case, while the authentication +request completed successfully, the account on Active Directory would have +received several failed login attempts, and that account might become +temporarily locked out. Plan the order of your realms accordingly. + +Also note that it is not typically necessary to define multiple Active Directory +realms to handle domain controller failures. When using Microsoft DNS, the DNS +entry for the domain should always point to an available domain controller. + + +[[trb-security-maccurl]] +=== Certificate verification fails for curl on Mac + +*Symptoms:* + +* `curl` on the Mac returns a certificate verification error even when the +`--cacert` option is used. + + +*Resolution:* + +Apple's integration of `curl` with their keychain technology disables the +`--cacert` option. +See http://curl.haxx.se/mail/archive-2013-10/0036.html for more information. + +You can use another tool, such as `wget`, to test certificates. Alternately, you +can add the certificate for the signing certificate authority MacOS system +keychain, using a procedure similar to the one detailed at the +http://support.apple.com/kb/PH14003[Apple knowledge base]. Be sure to add the +signing CA's certificate and not the server's certificate. + + +[[trb-security-sslhandshake]] +=== SSLHandshakeException causes connections to fail + +*Symptoms:* + +* A `SSLHandshakeException` causes a connection to a node to fail and indicates +that there is a configuration issue. Some of the common exceptions are shown +below with tips on how to resolve these issues. + + +*Resolution:* + +`java.security.cert.CertificateException: No name matching node01.example.com found`:: ++ +-- +Indicates that a client connection was made to `node01.example.com` but the +certificate returned did not contain the name `node01.example.com`. In most +cases, the issue can be resolved by ensuring the name is specified during +certificate creation. For more information, see <>. Another scenario is +when the environment does not wish to use DNS names in certificates at all. In +this scenario, all settings in `elasticsearch.yml` should only use IP addresses +including the `network.publish_host` setting. +-- + +`java.security.cert.CertificateException: No subject alternative names present`:: ++ +-- +Indicates that a client connection was made to an IP address but the returned +certificate did not contain any `SubjectAlternativeName` entries. IP addresses +are only used for hostname verification if they are specified as a +`SubjectAlternativeName` during certificate creation. If the intent was to use +IP addresses for hostname verification, then the certificate will need to be +regenerated with the appropriate IP address. See <>. +-- + +`javax.net.ssl.SSLHandshakeException: null cert chain` and `javax.net.ssl.SSLException: Received fatal alert: bad_certificate`:: ++ +-- +The `SSLHandshakeException` indicates that a self-signed certificate was +returned by the client that is not trusted as it cannot be found in the +`truststore` or `keystore`. This `SSLException` is seen on the client side of +the connection. +-- + +`sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target` and `javax.net.ssl.SSLException: Received fatal alert: certificate_unknown`:: ++ +-- +This `SunCertPathBuilderException` indicates that a certificate was returned +during the handshake that is not trusted. This message is seen on the client +side of the connection. The `SSLException` is seen on the server side of the +connection. The CA certificate that signed the returned certificate was not +found in the `keystore` or `truststore` and needs to be added to trust this +certificate. +-- + +[[trb-security-ssl]] +=== Common SSL/TLS exceptions + +*Symptoms:* + +* You might see some exceptions related to SSL/TLS in your logs. Some of the +common exceptions are shown below with tips on how to resolve these issues. + + + + +*Resolution:* + +`WARN: received plaintext http traffic on a https channel, closing connection`:: ++ +-- +Indicates that there was an incoming plaintext http request. This typically +occurs when an external applications attempts to make an unencrypted call to the +REST interface. Please ensure that all applications are using `https` when +calling the REST interface with SSL enabled. +-- + +`org.elasticsearch.common.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:`:: ++ +-- +Indicates that there was incoming plaintext traffic on an SSL connection. This +typically occurs when a node is not configured to use encrypted communication +and tries to connect to nodes that are using encrypted communication. Please +verify that all nodes are using the same setting for +`xpack.security.transport.ssl.enabled`. + +For more information about this setting, see +{ref}/security-settings.html[Security Settings in {es}]. +-- + +`java.io.StreamCorruptedException: invalid internal transport message format, got`:: ++ +-- +Indicates an issue with data received on the transport interface in an unknown +format. This can happen when a node with encrypted communication enabled +connects to a node that has encrypted communication disabled. Please verify that +all nodes are using the same setting for `xpack.security.transport.ssl.enabled`. + +For more information about this setting, see +{ref}/security-settings.html[Security Settings in {es}]. +-- + +`java.lang.IllegalArgumentException: empty text`:: ++ +-- +This exception is typically seen when a `https` request is made to a node that +is not using `https`. If `https` is desired, please ensure the following setting +is in `elasticsearch.yml`: + +[source,yaml] +---------------- +xpack.security.http.ssl.enabled: true +---------------- + +For more information about this setting, see +{ref}/security-settings.html[Security Settings in {es}]. +-- + +`ERROR: unsupported ciphers [...] were requested but cannot be used in this JVM`:: ++ +-- +This error occurs when a SSL/TLS cipher suite is specified that cannot supported +by the JVM that {es} is running in. Security tries to use the specified cipher +suites that are supported by this JVM. This error can occur when using the +Security defaults as some distributions of OpenJDK do not enable the PKCS11 +provider by default. In this case, we recommend consulting your JVM +documentation for details on how to enable the PKCS11 provider. + +Another common source of this error is requesting cipher suites that use +encrypting with a key length greater than 128 bits when running on an Oracle JDK. +In this case, you must install the +<>. +-- + +[[trb-security-kerberos]] +=== Common Kerberos exceptions + +*Symptoms:* + +* User authentication fails due to either GSS negotiation failure +or a service login failure (either on the server or in the {es} http client). +Some of the common exceptions are listed below with some tips to help resolve +them. + +*Resolution:* + +`Failure unspecified at GSS-API level (Mechanism level: Checksum failed)`:: ++ +-- + +When you see this error message on the HTTP client side, then it may be +related to an incorrect password. + +When you see this error message in the {es} server logs, then it may be +related to the {es} service keytab. The keytab file is present but it failed +to log in as the user. Please check the keytab expiry. Also check whether the +keytab contain up-to-date credentials; if not, replace them. + +You can use tools like `klist` or `ktab` to list principals inside +the keytab and validate them. You can use `kinit` to see if you can acquire +initial tickets using the keytab. Please check the tools and their documentation +in your Kerberos environment. + +Kerberos depends on proper hostname resolution, so please check your DNS infrastructure. +Incorrect DNS setup, DNS SRV records or configuration for KDC servers in `krb5.conf` +can cause problems with hostname resolution. + +-- + +`Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34))`:: + +`Failure unspecified at GSS-API level (Mechanism level: Clock skew too great (37))`:: ++ +-- + +To prevent replay attacks, Kerberos V5 sets a maximum tolerance for computer +clock synchronization and it is typically 5 minutes. Please check whether +the time on the machines within the domain is in sync. + +-- + +As Kerberos logs are often cryptic in nature and many things can go wrong +as it depends on external services like DNS and NTP. You might +have to enable additional debug logs to determine the root cause of the issue. + +{es} uses a JAAS (Java Authentication and Authorization Service) Kerberos login +module to provide Kerberos support. To enable debug logs on {es} for the login +module use following Kerberos realm setting: +[source,yaml] +---------------- +xpack.security.authc.realms..krb.debug: true +---------------- + +For detailed information, see {ref}/security-settings.html#ref-kerberos-settings[Kerberos realm settings]. + +Sometimes you may need to go deeper to understand the problem during SPNEGO +GSS context negotiation or look at the Kerberos message exchange. To enable +Kerberos/SPNEGO debug logging on JVM, add following JVM system properties: + +`-Dsun.security.krb5.debug=true` + +`-Dsun.security.spnego.debug=true` + +For more information about JVM system properties, see {ref}/jvm-options.html[configuring JVM options]. + +[[trb-security-saml]] +=== Common SAML issues + +Some of the common SAML problems are shown below with tips on how to resolve +these issues. + +. *Symptoms:* ++ +-- +Authentication in {kib} fails and the following error is printed in the {es} +logs: + +.... +Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, +assertionConsumerServiceURL=https://my.kibana.url/api/security/v1/saml}] +.... + +*Resolution:* + +{es}, {kib} and your Identity Provider need all have the same view on what the +Assertion Consumer Service URL of the SAML Service Provider is. + +.. {es} discovers this via the `sp.acs` setting in your {es} SAML realm configuration +.. {kib} constructs this value using the `server.host` and `server.port` in +`kibana.yml`. For instance: ++ +[source, shell] +----------------------------------------------- +server.host: kibanaserver.org +server.port: 3456 +----------------------------------------------- ++ +These settings would mean that {kib} would construct the Assertion Consumer +Service URL as `https://kibanaserver.org:3456/api/security/v1/saml`. However, +if for example, {kib} is behind a reverse proxy and you have configured the +following `xpack.security.public.*` settings: ++ +[source, shell] +----------------------------------------------- +xpack.security.public: + protocol: https + hostname: kibana.proxy.com + port: 8080 +----------------------------------------------- ++ +These settings would instruct {kib} to construct the Assertion Consumer Service +URL as `https://kibana.proxy.com:8080/api/security/v1/saml` + +.. The SAML Identity Provider is either explicitly configured by the IdP +administrator or consumes the SAML metadata that are generated by {es} and as +such contain the same value for the +as the one +that is configured in the the `sp.acs` setting in the {es} SAML realm +configuration. +-- ++ +The error encountered here indicates that the Assertion Consumer Service URL +that {kib} has constructed via one of the aforementioned ways +(`https://my.kibana.url/api/security/v1/saml`) is not the one that {es} is +configured with. Note that these two URLs are compared as case-sensitive strings +and not as canonicalized URLs. ++ +Often, this can be resolved by changing the `sp.acs` URL in `elasticearch.yml` +to match the value that {kib} has constructed. Note however, that the SAML IdP +configuration needs to also be adjusted to reflect this change. ++ +Alternatively, if you think {kib} is using the wrong value for the Assertion +Consumer Service URL, you will need to change the configuration in `kibana.yml` +by adjusting either the `server.host` and `server.port` to change the URL {kib} +listens to or the `xpack.security.public.*` settings to make {kib} aware about +its correct public URL. + +. *Symptoms:* ++ +-- +Authentication in {kib} fails and the following error is printed in the +{es} logs: + +.... +Authentication to realm saml1 failed - Provided SAML response is not valid for realm +saml/saml1 (Caused by ElasticsearchSecurityException[Conditions [https://some-url-here...] +do not match required audience [https://my.kibana.url]]) +.... + +*Resolution:* + +We received a SAML response that is addressed to another SAML Service Provider. +This usually means that the configured SAML Service Provider Entity ID in +`elasticsearch.yml` (`sp.entity_id`) does not match what has been configured as +the SAML Service Provider Entity ID in the SAML Identity Provider documentation. + +To resolve this issue, ensure that both the saml realm in {es} and the IdP are +configured with the same string for the SAML Entity ID of the Service Provider. + +TIP: These strings are compared as case-sensitive strings and not as +canonicalized URLs even when the values are URL-like. Be mindful of trailing +slashes, port numbers, etc. + +-- + +. *Symptoms:* ++ +-- +Authentication in {kib} fails and the following error is printed in the +{es} logs: + +.... +Cannot find metadata for entity [your:entity.id] in [metadata.xml] +.... + +*Resolution:* + +We could not find the metadata for the SAML Entity ID `your:entity.id` in the +configured metadata file (`metadata.xml`). + +.. Ensure that the `metadata.xml` file you are using is indeed the one provided +by your SAML Identity Provider. +.. Ensure that the `metadata.xml` file contains one element +as follows: `] +for action [cluster:admin/xpack/security/saml/authenticate] +.... + +*Resolution:* + +This error indicates that {es} failed to process the incoming SAML +authentication message. Since the message can't be processed, {es} is not aware +of who the to-be authenticated user is and the `` +placeholder is used instead. To diagnose the _actual_ problem, you must check +the {es} logs for further details. +-- + +. *Symptoms:* ++ +-- +Authentication in {kib} fails and the following error is printed in the +{es} logs: + +.... +Authentication to realm my-saml-realm failed - +Provided SAML response is not valid for realm saml/my-saml-realm +(Caused by ElasticsearchSecurityException[SAML Response is not a 'success' response: +Code=urn:oasis:names:tc:SAML:2.0:status:AuthnFailed Message=null Detail=null]) +.... + +*Resolution:* + +This means that the SAML Identity Provider failed to authenticate the user and +sent a SAML Response to the Service Provider ({stack}) indicating this failure. +The `Code`, `Message` and `Detail` can convey different error identifiers and +additional information that might offer an indication about the cause of the +failure. In case `Message` and `Detail` are null, please consult the logs and +documentation of the Identity Provider in order to further diagnose the problem. +-- + +. *Symptoms:* ++ +-- +Authentication in {kib} fails and the following error is printed in the +{es} logs: + +.... +The XML Signature of this SAML message cannot be validated. Please verify that the saml +realm uses the correct SAMLmetadata file/URL for this Identity Provider +.... + +*Resolution:* + +This means that {es} failed to validate the digital signature of the SAML +message that the Identity Provider sent. {es} uses the public key of the +Identity Provider that is included in the SAML metadata, in order to validate +the signature that the IdP has created using its corresponding private key. +Failure to do so, can have a number of causes: + +.. As the error message indicates, the most common cause is that the wrong +metadata file is used and as such the public key it contains doesn't correspond +to the private key the Identity Provider uses. +.. The configuration of the Identity Provider has changed or the key has been +rotated and the metadata file that {es} is using has not been updated. +.. The SAML Response has been altered in transit and the signature cannot be +validated even though the correct key is used. + +NOTE: The private keys and public keys and self-signed X.509 certificates that +are used in SAML for digital signatures as described above have no relation to +the keys and certificates that are used for TLS either on the transport or the +http layer. A failure such as the one described above has nothing to do with +your `xpack.ssl` related configuration. + +-- + +. *Symptoms:* ++ +-- +Users are unable to login with a local username and password in {kib} because +SAML is enabled. + +*Resolution:* + +If you want your users to be able to use local credentials to authenticate to +{kib} in addition to using the SAML realm for Single Sign-On, you must enable +the `basic` `authProvider` in {kib}. The process is documented in the +<> +-- + +*Logging:* + +Very detailed trace logging can be enabled specifically for the SAML realm by +setting the following transient setting: + +[source, shell] +----------------------------------------------- +PUT /_cluster/settings +{ + "transient": { + "logger.org.elasticsearch.xpack.security.authc.saml": "trace" + } +} +----------------------------------------------- + + +Alternatively, you can add the following lines to the end of the +`log4j2.properties` configuration file in the `ES_PATH_CONF`: + +[source,properties] +---------------- +logger.saml.name = org.elasticsearch.xpack.security.authc.saml +logger.saml.level = TRACE +---------------- + +[[trb-security-internalserver]] +=== Internal Server Error in Kibana + +*Symptoms:* + +* In 5.1.1, an `UnhandledPromiseRejectionWarning` occurs and {kib} displays an +Internal Server Error. +//TBD: Is the same true for later releases? + +*Resolution:* + +If the Security plugin is enabled in {es} but disabled in {kib}, you must +still set `elasticsearch.username` and `elasticsearch.password` in `kibana.yml`. +Otherwise, {kib} cannot connect to {es}. + + +[[trb-security-setup]] +=== Setup-passwords command fails due to connection failure + +The {ref}/setup-passwords.html[elasticsearch-setup-passwords command] sets +passwords for the built-in users by sending user management API requests. If +your cluster uses SSL/TLS for the HTTP (REST) interface, the command attempts to +establish a connection with the HTTPS protocol. If the connection attempt fails, +the command fails. + +*Symptoms:* + +. {es} is running HTTPS, but the command fails to detect it and returns the +following errors: ++ +-- +[source, shell] +------------------------------------------ +Cannot connect to elasticsearch node. +java.net.SocketException: Unexpected end of file from server +... +ERROR: Failed to connect to elasticsearch at +http://127.0.0.1:9200/_xpack/security/_authenticate?pretty. +Is the URL correct and elasticsearch running? +------------------------------------------ +-- + +. SSL/TLS is configured, but trust cannot be established. The command returns +the following errors: ++ +-- +[source, shell] +------------------------------------------ +SSL connection to +https://127.0.0.1:9200/_xpack/security/_authenticate?pretty +failed: sun.security.validator.ValidatorException: +PKIX path building failed: +sun.security.provider.certpath.SunCertPathBuilderException: +unable to find valid certification path to requested target +Please check the elasticsearch SSL settings under +xpack.security.http.ssl. +... +ERROR: Failed to establish SSL connection to elasticsearch at +https://127.0.0.1:9200/_xpack/security/_authenticate?pretty. +------------------------------------------ +-- + +. The command fails because hostname verification fails, which results in the +following errors: ++ +-- +[source, shell] +------------------------------------------ +SSL connection to +https://idp.localhost.test:9200/_xpack/security/_authenticate?pretty +failed: java.security.cert.CertificateException: +No subject alternative DNS name matching +elasticsearch.example.com found. +Please check the elasticsearch SSL settings under +xpack.security.http.ssl. +... +ERROR: Failed to establish SSL connection to elasticsearch at +https://elasticsearch.example.com:9200/_xpack/security/_authenticate?pretty. +------------------------------------------ +-- + +*Resolution:* + +. If your cluster uses TLS/SSL for the HTTP interface but the +`elasticsearch-setup-passwords` command attempts to establish a non-secure +connection, use the `--url` command option to explicitly specify an HTTPS URL. +Alternatively, set the `xpack.security.http.ssl.enabled` setting to `true`. + +. If the command does not trust the {es} server, verify that you configured the +`xpack.security.http.ssl.certificate_authorities` setting or the +`xpack.security.http.ssl.truststore.path` setting. + +. If hostname verification fails, you can disable this verification by setting +`xpack.security.http.ssl.verification_mode` to `certificate`. + +For more information about these settings, see +{ref}/security-settings.html[Security Settings in {es}]. + +[[trb-security-path]] +=== Failures due to relocation of the configuration files + +*Symptoms:* + +* Active Directory or LDAP realms might stop working after upgrading to {es} 6.3 +or later releases. In 6.4 or later releases, you might see messages in the {es} +log that indicate a config file is in a deprecated location. + +*Resolution:* + +By default, in 6.2 and earlier releases, the +<> are located in the +`ES_PATH_CONF/x-pack` directory, where `ES_PATH_CONF` is an environment +variable that defines the location of the +{ref}/settings.html#config-files-location[config directory]. + +In 6.3 and later releases, the config directory no longer contains an `x-pack` +directory. The files that were stored in this folder, such as the +`log4j2.properties`, `role_mapping.yml`, `roles.yml`, `users`, and `users_roles` +files, now exist directly in the config directory. + +IMPORTANT: If you upgraded to 6.3 or later releases, your old security +configuration files still exist in an `x-pack` folder. That file path is +deprecated, however, and you should move your files out of that folder. + +In 6.3 and later releases, settings such as `files.role_mapping` default to +`ES_PATH_CONF/role_mapping.yml`. If you do not want to use the default locations, +you must update the settings appropriately. See +{ref}/security-settings.html[Security settings in {es}]. +