Deprecate the kibana reserved user; introduce kibana_system user (#54967)

* deprecate the kibana reserved user; introduce kibana_system user

* fix license and test errors

* fix IdentityProviderAuthenticationIT tests

* test deprecation logging

* First pass at SetupPasswordTool updates

* fix checkstyle

* update docs

* update number of expected users

* update test to expect deprecation header

Co-authored-by: Elastic Machine <[email protected]>

Author: legrego

client/rest-high-level/src/test/java/org/elasticsearch/client/documentation/SecurityDocumentationIT.java

@@ -200,11 +200,11 @@ public void testGetUsers() throws Exception {
             List<User> users = new ArrayList<>(3);
             users.addAll(response.getUsers());
             assertNotNull(response);
-            // 9 users are expected to be returned
+            // 10 users are expected to be returned
             // test_users (3): user1, user2, user3
-            // system_users (6): elastic, beats_system, apm_system, logstash_system, kibana, remote_monitoring_user
+            // system_users (6): elastic, beats_system, apm_system, logstash_system, kibana, kibana_system, remote_monitoring_user
             logger.info(users);
-            assertThat(users.size(), equalTo(9));
+            assertThat(users.size(), equalTo(10));
         }
 
         {

docs/reference/migration/migrate_8_0/security.asciidoc

@@ -132,3 +132,40 @@ It is now an error to enable SSL for the HTTP (Rest) server without also configu
 a certificate and key through use of the `xpack.security.http.ssl.keystore.path`
 setting or the `xpack.security.http.ssl.certificate` and
 `xpack.security.http.ssl.key` settings.
+
+
+[float]
+[[builtin-users-changes]]
+==== Changes to built-in users
+
+[float]
+===== The `kibana` user has been removed in favor of the `kibana_system` user
+
+The `kibana` user was historically used to authenticate {kib} to {es}.
+The name of this user was confusing, and was often mistakenly used to login to {kib}.
+This has been renamed to `kibana_system` in order to reduce confusion, and to better
+align with other built-in system accounts.
+
+If your `kibana.yml` used to contain:
+[source,yaml]
+--------------------------------------------------
+elasticsearch.username: kibana
+--------------------------------------------------
+
+then you should update to use the new `kibana_system` user instead:
+[source,yaml]
+--------------------------------------------------
+elasticsearch.username: kibana_system
+--------------------------------------------------
+
+
+[float]
+[[builtin-roles-changes]]
+==== Changes to built-in roles
+
+[float]
+===== The `kibana_user` role has been removed in favor of the `kibana_admin` role
+
+Users who were previously assigned the `kibana_user` role should instead be assigned
+the `kibana_admin` role. This role grants the same set of privileges as `kibana_user`, but has been
+renamed to better reflect its intended use.

x-pack/docs/en/security/auditing/output-logfile.asciidoc

@@ -10,7 +10,7 @@ are similar. For systems that are not ingesting the audit file for search or
 analytics it is strongly recommended to keep only the newer format.
 
 To turn off the deprecated output format, you can disable the logger in the
-`log4j2.properties` file: 
+`log4j2.properties` file:
 
 [source, properties]
 --------------------------------------------------
@@ -139,18 +139,18 @@ non-matching events are printed as usual.
 
 All policies are defined under the `xpack.security.audit.logfile.events.ignore_filters`
 settings namespace. For example, the following policy named _example1_ matches
-events from the _kibana_ or _admin_user_ principals **and** operating over indices of the
+events from the _kibana_system_ or _admin_user_ principals **and** operating over indices of the
 wildcard form _app-logs*_:
 
 [source,yaml]
 ----------------------------
 xpack.security.audit.logfile.events.ignore_filters:
   example1:
-    users: ["kibana", "admin_user"]
+    users: ["kibana_system", "admin_user"]
     indices: ["app-logs*"]
 ----------------------------
 
-An audit event generated by the _kibana_ user and operating over multiple indices
+An audit event generated by the _kibana_system_ user and operating over multiple indices
 , some of which do not match the indices wildcard, will not match.
 As expected, operations generated by all other users (even operating only on indices that
 match the _indices_ filter) will not match this policy either.

x-pack/docs/en/security/authentication/built-in-users.asciidoc

@@ -12,9 +12,9 @@ used to <<set-built-in-user-passwords,set all of the built-in user passwords>>.
 `logstash_system`:: The user Logstash uses when storing monitoring information in {es}.
 `beats_system`:: The user the Beats use when storing monitoring information in {es}.
 `apm_system`:: The user the APM server uses when storing monitoring information in {es}.
-`remote_monitoring_user`:: The user {metricbeat} uses when collecting and 
-storing monitoring information in {es}. It has the `remote_monitoring_agent` and 
-`remote_monitoring_collector` built-in roles. 
+`remote_monitoring_user`:: The user {metricbeat} uses when collecting and
+storing monitoring information in {es}. It has the `remote_monitoring_agent` and
+`remote_monitoring_collector` built-in roles.
 
 TIP: The built-in users serve specific purposes and are not intended for general
 use. In particular, do not use the `elastic` superuser unless full access to
@@ -65,7 +65,7 @@ The +elasticsearch-setup-passwords+ tool is the simplest method to set the
 built-in users' passwords for the first time. It uses the `elastic` user's
 bootstrap password to run user management API requests. For example, you can run
 the command in an "interactive" mode, which prompts you to enter new passwords
-for the `elastic`, `kibana`, `logstash_system`, `beats_system`, `apm_system`, 
+for the `elastic`, `kibana_system`, `logstash_system`, `beats_system`, `apm_system`,
 and `remote_monitoring_user` users:
 
 [source,shell]
@@ -107,7 +107,7 @@ since at that point the bootstrap password is no longer required.
 [[add-built-in-user-kibana]]
 ==== Adding built-in user passwords to {kib}
 
-After the `kibana` user password is set, you need to update the {kib} server
+After the `kibana_system` user password is set, you need to update the {kib} server
 with the new password by setting `elasticsearch.password` in the `kibana.yml`
 configuration file:
 
@@ -120,7 +120,7 @@ See {kibana-ref}/using-kibana-with-security.html[Configuring security in {kib}].
 
 [float]
 [[add-built-in-user-logstash]]
-==== Adding built-in user passwords to {ls} 
+==== Adding built-in user passwords to {ls}
 
 The `logstash_system` user is used internally within Logstash when
 monitoring is enabled for Logstash.
@@ -143,7 +143,7 @@ been changed, you can enable the user via the following API call:
 PUT _security/user/logstash_system/_enable
 ---------------------------------------------------------------------
 
-See {logstash-ref}/ls-security.html#ls-monitoring-user[Configuring credentials for {ls} monitoring]. 
+See {logstash-ref}/ls-security.html#ls-monitoring-user[Configuring credentials for {ls} monitoring].
 
 [float]
 [[add-built-in-user-beats]]
@@ -161,13 +161,13 @@ xpack.monitoring.elasticsearch.username: beats_system
 xpack.monitoring.elasticsearch.password: beatspassword
 ----------------------------------------------------------
 
-For example, see {metricbeat-ref}/monitoring.html[Monitoring {metricbeat}]. 
+For example, see {metricbeat-ref}/monitoring.html[Monitoring {metricbeat}].
 
-The `remote_monitoring_user` is used when {metricbeat} collects and stores 
-monitoring data for the {stack}. See <<monitoring-production>>. 
+The `remote_monitoring_user` is used when {metricbeat} collects and stores
+monitoring data for the {stack}. See <<monitoring-production>>.
 
 If you have upgraded from an older version of {es}, then you may not have set a
-password for the `beats_system` or `remote_monitoring_user` users. If this is 
+password for the `beats_system` or `remote_monitoring_user` users. If this is
 the case, then you should use the *Management > Users* page in {kib} or the
 <<security-api-change-password,change password API>> to set a password
 for these users.
@@ -178,8 +178,8 @@ for these users.
 
 The `apm_system` user is used internally within APM when monitoring is enabled.
 
-To enable this feature in APM, you need to update the 
-{apm-server-ref}/configuring-howto-apm-server.html[APM configuration file] to 
+To enable this feature in APM, you need to update the
+{apm-server-ref}/configuring-howto-apm-server.html[APM configuration file] to
 reference the correct username and password. For example:
 
 [source,yaml]
@@ -188,10 +188,10 @@ xpack.monitoring.elasticsearch.username: apm_system
 xpack.monitoring.elasticsearch.password: apmserverpassword
 ----------------------------------------------------------
 
-See {apm-server-ref}/monitoring.html[Monitoring APM Server]. 
+See {apm-server-ref}/monitoring.html[Monitoring APM Server].
 
 If you have upgraded from an older version of {es}, then you may not have set a
-password for the `apm_system` user. If this is the case, 
+password for the `apm_system` user. If this is the case,
 then you should use the *Management > Users* page in {kib} or the
 <<security-api-change-password,change password API>> to set a password
 for these users.

x-pack/docs/en/security/get-started-builtin-users.asciidoc

@@ -1,13 +1,13 @@
 // tag::create-users[]
 There are <<built-in-users,built-in users>> that you can use for specific
-administrative purposes: `apm_system`, `beats_system`, `elastic`, `kibana`,
-`logstash_system`,  and `remote_monitoring_user`. 
+administrative purposes: `apm_system`, `beats_system`, `elastic`, `kibana_system`,
+`logstash_system`,  and `remote_monitoring_user`.
 
 // end::create-users[]
 
 Before you can use them, you must set their passwords:
 
-. Restart {es}. For example, if you installed {es} with a `.tar.gz` package, run 
+. Restart {es}. For example, if you installed {es} with a `.tar.gz` package, run
 the following command from the {es} directory:
 +
 --

x-pack/docs/en/security/get-started-kibana-users.asciidoc

@@ -1,36 +1,36 @@
 When the {es} {security-features} are enabled, users must log in to {kib}
-with a valid user ID and password. 
+with a valid user ID and password.
 
-{kib} also performs some tasks under the covers that require use of the 
-built-in `kibana` user. 
+{kib} also performs some tasks under the covers that require use of the
+built-in `kibana_system` user.
 
-. Configure {kib} to use the built-in `kibana` user and the password that you 
+. Configure {kib} to use the built-in `kibana_system` user and the password that you
 created:
 
-** If you don't mind having passwords visible in your configuration file, 
-uncomment and update the following settings in the `kibana.yml` file in your 
+** If you don't mind having passwords visible in your configuration file,
+uncomment and update the following settings in the `kibana.yml` file in your
 {kib} directory:
 +
 --
-TIP: If you installed {kib} using archive distributions (`zip` or 
-`tar.gz`), the `kibana.yml` configuration file is in `KIBANA_HOME/config`. If 
-you used package distributions (Debian or RPM), it's in `/etc/kibana`. For more 
-information, see {kibana-ref}/settings.html[Configuring {kib}].  
+TIP: If you installed {kib} using archive distributions (`zip` or
+`tar.gz`), the `kibana.yml` configuration file is in `KIBANA_HOME/config`. If
+you used package distributions (Debian or RPM), it's in `/etc/kibana`. For more
+information, see {kibana-ref}/settings.html[Configuring {kib}].
 
 For example, add the following settings:
 
 [source,yaml]
 ----
-elasticsearch.username: "kibana"
+elasticsearch.username: "kibana_system"
 elasticsearch.password: "your_password"
 ----
 
-Specify the password that you set with the `elasticsearch-setup-passwords` 
-command then save your changes to the file. 
+Specify the password that you set with the `elasticsearch-setup-passwords`
+command then save your changes to the file.
 --
 
-** If you prefer not to put your user ID and password in the `kibana.yml` file, 
-store them in a keystore instead. Run the following commands to create the {kib} 
+** If you prefer not to put your user ID and password in the `kibana.yml` file,
+store them in a keystore instead. Run the following commands to create the {kib}
 keystore and add the secure settings:
 +
 --
@@ -42,14 +42,14 @@ keystore and add the secure settings:
 ./bin/kibana-keystore add elasticsearch.password
 ----------------------------------------------------------------------
 
-When prompted, specify the `kibana` built-in user and its password for these 
-setting values.  The settings are automatically applied when you start {kib}.   
+When prompted, specify the `kibana_system` built-in user and its password for these
+setting values.  The settings are automatically applied when you start {kib}.
 To learn more, see {kibana-ref}/secure-settings.html[Secure settings].
 // end::store-kibana-user[]
 --
 
-. Restart {kib}. For example, if you installed 
-{kib} with a `.tar.gz` package, run the following command from the {kib} 
+. Restart {kib}. For example, if you installed
+{kib} with a `.tar.gz` package, run the following command from the {kib}
 directory:
 +
 --
@@ -58,5 +58,5 @@ directory:
 ./bin/kibana
 ----------------------------------------------------------------------
 
-See {kibana-ref}/start-stop.html[Starting and stopping {kib}]. 
---
+See {kibana-ref}/start-stop.html[Starting and stopping {kib}].
+--

x-pack/docs/en/security/securing-communications/tutorial-tls-internode.asciidoc

@@ -1,15 +1,15 @@
 [role="xpack"]
 [testenv="trial"]
 [[encrypting-internode]]
-=== Encrypt internode communications  
+=== Encrypt internode communications
 
 Now that we've generated a certificate authority and certificates, let's update
 the cluster to use these files.
 
 IMPORTANT: When you enable {es} {security-features}, unless you have a trial
 license, you must use Transport Layer Security (TLS) to encrypt internode
 communication. By following the steps in this tutorial tutorial, you learn how
-to meet the minimum requirements to pass the 
+to meet the minimum requirements to pass the
 <<bootstrap-checks-tls,TLS bootstrap check>>.
 
 . (Optional) Name the cluster.
@@ -23,10 +23,10 @@ For example, add the <<cluster.name,cluster.name>> setting in the
 cluster.name: test-cluster
 ----
 
-TIP: The `ES_PATH_CONF` environment variable contains the path for the {es} 
-configuration files. If you installed {es} using archive distributions (`zip` or 
-`tar.gz`), it defaults to `ES_HOME/config`. If you used package distributions 
-(Debian or RPM), it defaults to `/etc/elasticsearch`. For more information, see 
+TIP: The `ES_PATH_CONF` environment variable contains the path for the {es}
+configuration files. If you installed {es} using archive distributions (`zip` or
+`tar.gz`), it defaults to `ES_HOME/config`. If you used package distributions
+(Debian or RPM), it defaults to `/etc/elasticsearch`. For more information, see
 <<settings>>.
 
 The default cluster name is `elasticsearch`. You should choose a unique name,
@@ -46,7 +46,7 @@ node.name: node-1
 
 In this tutorial, the cluster will consist of three nodes that exist on the same
 machine and share the same (loopback) IP address and hostname. Therefore, we
-must give each node a unique name. 
+must give each node a unique name.
 
 This step is also necessary if you want to use the `node.name` value to define
 the location of certificates in subsequent steps.
@@ -79,13 +79,13 @@ itself into a new cluster.
 TIP: If you are starting a cluster with multiple master-eligible nodes for the
 first time, add all of those node names to the `cluster.initial_master_nodes`
 setting.
- 
+
 See <<modules-discovery-bootstrap-cluster>> and
 <<discovery-settings>>.
 --
 
 . Enable Transport Layer Security (TLS/SSL) for transport (internode)
-communications. 
+communications.
 +
 --
 // tag::enable-tls[]
@@ -95,28 +95,28 @@ file:
 [source,yaml]
 ----
 xpack.security.enabled: true
-xpack.security.transport.ssl.enabled: true  
+xpack.security.transport.ssl.enabled: true
 xpack.security.transport.ssl.keystore.path: certs/${node.name}.p12 <1>
 xpack.security.transport.ssl.truststore.path: certs/${node.name}.p12
 ----
 <1> If the file name for your certificate does not match the `node.name` value,
-you must put the appropriate file name in the `elasticsearch.yml` file. 
+you must put the appropriate file name in the `elasticsearch.yml` file.
 // end::enable-tls[]
 
 NOTE: The PKCS#12 keystore that is output by the `elasticsearch-certutil` can be
-used as both a keystore and a truststore. If you use other tools to manage and 
+used as both a keystore and a truststore. If you use other tools to manage and
 generate your certificates, you might have different values for these settings,
 but that scenario is not covered in this tutorial.
 
-For more information, see <<get-started-enable-security>> and 
+For more information, see <<get-started-enable-security>> and
 <<transport-tls-ssl-settings>>.
 --
 
 . Store the password for the PKCS#12 file in the {es} keystore.
 +
 --
 // tag::secure-passwords[]
-For example, run the following commands: 
+For example, run the following commands:
 
 ["source","sh",subs="attributes,callouts"]
 ----------------------------------------------------------------------
@@ -146,19 +146,19 @@ command from the {es} directory:
 ----------------------------------------------------------------------
 --
 
-. Create passwords for the built-in users and configure {kib} to use them. 
+. Create passwords for the built-in users and configure {kib} to use them.
 +
 --
 NOTE: If you already configured passwords for these users in other tutorials,
 you can skip this step.
 
 include::{xes-repo-dir}/security/get-started-builtin-users.asciidoc[tag=create-users]
 
-After you setup the password for the `kibana` built-in user,
+After you setup the password for the `kibana_system` built-in user,
 <<get-started-kibana-user,configure {kib} to use it>>.
 
 For example, run the following commands to create the {kib} keystore and add the
-`kibana` built-in user and its password in secure settings:
+`kibana_system` built-in user and its password in secure settings:
 
 include::{xes-repo-dir}/security/get-started-kibana-users.asciidoc[tag=store-kibana-user]
 --
@@ -173,5 +173,5 @@ command from the {kib}  directory:
 ./bin/kibana
 ----------------------------------------------------------------------
 
-See {kibana-ref}/start-stop.html[Starting and stopping {kib}]. 
+See {kibana-ref}/start-stop.html[Starting and stopping {kib}].
 --