[DISCUSS] Index config update

[guilload]

Currently, once an index is created, its config is immutable. We want to provide an API for updating an index config.

CLI API

Creating an index
quickwit index apply --id my-index --uri s3://my-indexes/my-index --config-file path/to/my/index/config.yaml

Updating an index config
quickwit index apply --index my-index --config-file path/to/my/index/config.yaml

Fetching an index config
quickwit index get --index my-index

REST API

TDB

Metastore API

The following API already exists for creating an index:

metastore.create_index(&self, index_metadata: IndexMetadata) -> MetastoreResult<()>;

metastore.apply_index_config(&self, index_id: &str, index_config: IndexConfig) -> MetastoreResult<()>;

Implementation

Ideally, the CLI and REST API both rely on the

Out of scope:

• Index ID update
• Index URI update
• DocMapping update

[guilload]

CLI API alternative

Creating an index (same as before)
quickwit index create --id my-index --uri s3://my-indexes/my-index --config-file path/to/my/index/config.yaml

Updating an index config
quickwit index config apply --index my-index --config-file path/to/my/index/config.yaml

Fetching an index config
quickwit index config get --index my-index

[guilload]

See #1071 for a discussion regarding source creation and source config update.

[rdettai]

Update of the retention policy and search config is addressed in #4832. Update semantics are a bit different from what was proposed here:

• the update CLI does not use --config-file path/to/my/index/config.yaml but instead allows updating the configurations that can be updated individually. The rational is that it would be kind of confusing to have an API that looks like you can reapply the entire config but actually fails on (or ignores) most changes.

Update of indexing setting will be proposed in a separate PR #4868

[rdettai]

Completed by #5078.

Even updating doc mappings is now possible thanks to #5253.