Missing right to write transient cluster settings in custom plugin

[olfuerniss]

Describe the feature:

I'm using ES Version 7.3.1. When I try to write transient cluster settings - in my custom plugin - it throws an exception that the [_system] user does not have the [cluster:admin/settings/update] right.

At the moment I'm patching the Java class org.elasticsearch.xpack.core.security.authz.privilege.SystemPrivilege.class found in the x-pack-core-7.3.1.jar by adding "cluster:admin/settings/*", to the ALLOWED_ACTIONS list. Doing this with every ES release is rather annoying. It would be great if you could add the missing right like you already have done with issue #33119 . Thanks!

Example code to write transient settings:

ClusterUpdateSettingsRequest settingsUpdateRequest = new ClusterUpdateSettingsRequest();
settingsUpdateRequest.transientSettings(s);

ClusterUpdateSettingsResponse clusterUpdateSettingsResponse = clientProvider.get().admin().cluster().updateSettings(settingsUpdateRequest).get();
if (clusterUpdateSettingsResponse.isAcknowledged()) {
    LOG.debug("Transient cluster setting '" + structuredSettingName + "' set to '" + value + "'");
} else {
    LOG.debug("Failed to set the transient setting '" + structuredSettingName + "' to '" + value + "'");
}

It will throw the following exception:

[2019-08-29T13:57:47,790][ERROR][d.v.b.e.p.BpcConnections ] [node-of-1][es-bpc-plugin] Failed to write the transient cluster setting 'connections'.
java.util.concurrent.ExecutionException: ElasticsearchSecurityException[action [cluster:admin/settings/update] is unauthorized for user [_system]]
	at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.getValue(BaseFuture.java:266) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:253) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:87) ~[elasticsearch-7.3.1.jar:7.3.1]
...

Updated/patched SystemPrivilege.java (ES 7.3.1)

/*
 * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
 * or more contributor license agreements. Licensed under the Elastic License;
 * you may not use this file except in compliance with the Elastic License.
 */
package org.elasticsearch.xpack.core.security.authz.privilege;

import org.elasticsearch.index.seqno.RetentionLeaseActions;
import org.elasticsearch.index.seqno.RetentionLeaseBackgroundSyncAction;
import org.elasticsearch.index.seqno.RetentionLeaseSyncAction;
import org.elasticsearch.transport.TransportActionProxy;
import org.elasticsearch.xpack.core.security.support.Automatons;

import java.util.Collections;
import java.util.function.Predicate;

public final class SystemPrivilege extends Privilege {

    public static SystemPrivilege INSTANCE = new SystemPrivilege();

    private static final Predicate<String> ALLOWED_ACTIONS = Automatons.predicate(
        "internal:*",
        "cluster:admin/settings/*",
        "indices:monitor/*", // added for monitoring
        "cluster:monitor/*",  // added for monitoring
        "cluster:admin/bootstrap/*", // for the bootstrap service
        "cluster:admin/reroute", // added for DiskThresholdDecider.DiskListener
        "indices:admin/mapping/put", // needed for recovery and shrink api
        "indices:admin/template/put", // needed for the TemplateUpgradeService
        "indices:admin/template/delete", // needed for the TemplateUpgradeService
        "indices:admin/seq_no/global_checkpoint_sync*", // needed for global checkpoint syncs
        RetentionLeaseSyncAction.ACTION_NAME + "*", // needed for retention lease syncs
        RetentionLeaseBackgroundSyncAction.ACTION_NAME + "*", // needed for background retention lease syncs
        RetentionLeaseActions.Add.ACTION_NAME + "*", // needed for CCR to add retention leases
        RetentionLeaseActions.Remove.ACTION_NAME + "*", // needed for CCR to remove retention leases
        RetentionLeaseActions.Renew.ACTION_NAME + "*", // needed for CCR to renew retention leases
        "indices:admin/settings/update" // needed for DiskThresholdMonitor.markIndicesReadOnly
    );

    private static final Predicate<String> PREDICATE = (action) -> {
        // Only allow a proxy action if the underlying action is allowed
        if (TransportActionProxy.isProxyAction(action)) {
            return ALLOWED_ACTIONS.test(TransportActionProxy.unwrapAction(action));
        } else {
            return ALLOWED_ACTIONS.test(action);
        }
    };

    private SystemPrivilege() {
        super(Collections.singleton("internal"));
    }

    @Override
    public Predicate<String> predicate() {
        return PREDICATE;
    }
}

[jasontedor]

I am not sure that we should make this change.

The reason for #34030 is that the system is taking action on behalf of the user and needs permissions to do so. If I had my druthers, we would scope those permissions even further so that the system had permission only to manage that setting and not all indices settings in general.

I would be reluctant to grant the system the privilege to manage any cluster setting.

I'll mark this as team-discuss for others to weigh in.

[elasticmachine]

Pinging @elastic/es-security

[tvernum]

I agree with @jasontedor here. The _system user isn't supposed to do this sort of thing. We have other mechanisms for performing tasks that need a higher privilege than _system. The solution isn't to add privileges to system, but to change the plugin to make use of those other mechanisms.

I think we can solve this on the Elasticsearch forum if you can talk us through what your plugin is trying to do.

[tvernum]

We discussed this internally.

The intent is that the _system user is not permitted to do these things - the system user is supposed to have a minimal set of privileges, and we don't want to give it permission to do all the sorts of things that plugins may want to do (e.g. there are plugins that read from/write to indices, but the system user should definitely not be allowed to do that).

If plugin authors need help navigating security, please open a topic on the forums and we'll help where we can.

[jessebs]

Can you point to some of the other mechanisms because I'm running into similar problems with a plugin I'm developing. The forum seems to have a few related questions that are unanswered.