Skip to content

Support for disabling terms index off heap #825

@jainankitk

Description

@jainankitk

Is your feature request related to a problem? Please describe.
Opensearch always reads terms index from disk using mmap which can impact performance significantly, especially when the terms index size is small.

Describe the solution you'd like
There is index level setting for id field only index.force_memory_id_terms_dictionary which has limitations (only id field) and buggy (same named field across indices affected). We should have cluster level setting to keep terms indices in memory on heap. Since, index level setting anyways has wider scope, this setting should be removed or scoped at cluster level than index

Describe alternatives you've considered
I have thought about and considered below alternative:

  • Move terms index on heap by default always - This used to be the case earlier and resulted in JVM sizing issues for many customers
  • Disable off heap by default and add setting to move it off heap - This is viable option though more customers prefer to have it off heap than on heap. Hence, I prefer the proposal than this alternative

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions