[CI] HLRC MachineLearningIT.testDeleteExpiredData failed

[droberts195]

MachineLearningIT.testDeleteExpiredData in the HLRC tests failed in https://gradle-enterprise.elastic.co/s/umyybnb6ffm32

The error was:

method [POST], host [http://127.0.0.1:39299], URI [/.ml-state/_refresh], status line [HTTP/1.1 404 Not Found]
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [.ml-state]","resource.type":"index_or_alias","resource.id":".ml-state","index_uuid":"_na_","index":".ml-state"}],"type":"index_not_found_exception","reason":"no such index [.ml-state]","resource.type":"index_or_alias","resource.id":".ml-state","index_uuid":"_na_","index":".ml-state"},"status":404}

@przemekwitek this seems like it is probably related to the addition of ILM to the ML state index - please can you investigate?

[elasticmachine]

Pinging @elastic/ml-core (:ml)

[przemekwitek]

Sure, investigating...

[przemekwitek]

I identified the problem this test has: it indexes a document to legacy .ml-state index triggering index creation:

            // Index a randomly named unused state document
            String docId = "non_existing_job_" + randomFrom("model_state_1234567#1", "quantiles", "categorizer_state#1");
            IndexRequest indexRequest = new IndexRequest(".ml-state").id(docId);

I'll use .ml-state-000001 instead.