[Feature] Supporting Exclusion pattern in index pattern in ISM

[Tarun-kishore]

Description

This PR implements support for exclusion patterns in ISM template index patterns using the - prefix, similar to OpenSearch's native index pattern matching. This allows users to define broad inclusion patterns while excluding specific indices from automatic ISM policy management.

Key Changes:

• Pattern Matching Logic: Added matchesIndexPatterns() function in ManagedIndexCoordinator that separates inclusion and exclusion patterns, ensuring indices are only managed if they match an inclusion pattern and don't match any exclusion pattern
• Validation: Updated validateFormat() in ISMTemplateService to accept and validate exclusion patterns (patterns prefixed with -)
• Pattern Processing: Modified findMatchingPolicy() to use the new exclusion-aware matching logic

Example Usage:

{
  "ism_template": {
    "index_patterns": ["logs-*", "-logs-test-*", "-logs-*-debug"],
    "priority": 100
  }
}

This configuration will:

• ✅ Manage: logs-production-001, logs-staging-001
• ❌ Exclude: logs-test-001, logs-production-debug

Benefits:

• Simplifies index management by allowing users to exclude specific patterns without complex naming schemes
• Maintains backward compatibility - existing templates without exclusion patterns continue to work unchanged
• Follows OpenSearch's existing pattern matching conventions (e.g., GET /logs-*,-logs-test-*/_search)

---

Testing Demonstration

Step 1: Create Policy with Exclusion Patterns

Created an ISM policy with mixed inclusion and exclusion patterns:

{
  "policy": {
    "description": "testing exclude change",
    "default_state": "noop",
    "states": [
      {
        "name": "noop",
        "actions": [],
        "transitions": []
      }
    ],
    "ism_template": {
      "index_patterns": ["test-1*", "-test-12*", "test-3*", "-test-34*"], 
      "priority": 100
    }
  }
}

Step 2: Create Test Indices

Created four test indices to verify the exclusion behavior:

• test-10 - Should be managed (matches test-1*, not excluded)
• test-12 - Should NOT be managed (matches exclusion -test-12*)
• test-30 - Should be managed (matches test-3*, not excluded)
• test-34 - Should NOT be managed (matches exclusion -test-34*)

Step 3: Verify with ISM Explain API

After coordinator processing, the ISM Explain API confirms the expected behavior:

Index	Managed	Policy Applied
test-10	✅ Yes	test_exclusion_policy
test-12	❌ No	None (excluded)
test-30	✅ Yes	test_exclusion_policy
test-34	❌ No	None (excluded)

Related Issues

Resolves #375

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.15%. Comparing base (0728413) to head (10abbf9).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1509      +/-   ##
==========================================
+ Coverage   76.08%   76.15%   +0.07%     
==========================================
  Files         375      375              
  Lines       17542    17565      +23     
  Branches     2401     2409       +8     
==========================================
+ Hits        13346    13377      +31     
+ Misses       2951     2947       -4     
+ Partials     1245     1241       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[bowenlan-amzn]

Nice!