[ML] domainSplit painless extension not available in runtime fields

[davidkyle]

The ml plugin defines a custom painless extension function domainSplit which splits a URL into the highest registered domain and subdomains. The function can be called via script_fields in a search but if called in runtime_mappings the script errors with:
Unknown call [domainSplit] with [[org.elasticsearch.painless.node.EDot@156a1756]] arguments

To Reproduce

PUT /domain-split-test
{
  "mappings":{
    "properties": {
      "url": {
        "type": "keyword" 
      }
    }
  }
}

POST /domain-split-test/_doc
{
  "url":"www.elastic.co"
}

# OK with script fields 
GET /domain-split-test/_search
{
  "script_fields":{
    "sub":{
      "script":"return domainSplit(doc['url'].value).get(0)"
    },
    "hrd":{
      "script":"return domainSplit(doc['url'].value).get(1);"
    }
  }
}

# Error using RT mappings
GET /domain-split-test/_search
{
  "runtime_mappings":{
    "sub":{
      "type": "keyword",
      "script":"emit(domainSplit(doc['url'].value).get(0))"
    },
    "hrd":{
      "type": "keyword",
      "script":"emit(domainSplit(doc['url'].value).get(1))"
    }
  }
}

[elasticmachine]

Pinging @elastic/ml-core (Team:ML)

[javanna]

I had a look at this, and it looks like the domainSplit function is registered by the MachineLearningPainlessExtension and enhances the existing script_field context. Runtime fields have different contexts, one per data type. It is possible to enhance the existing runtime fields contexts as well in the same way. I wonder if this is needed, and if so if the function should be built-in.

[elasticmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[droberts195]

#36359 has been open for years to get the domainSplit function moved to core Elasticsearch, and it's on the list of TODO items in the meta issue #48986.

[davidkyle]

A Registered domain ingest processor has been added to the 7.13 release in #67611. The implementation is different to the one in the ml plugin as it uses org.apache.http.conn.util.PublicSuffixMatcher although both use the same public suffix list. If domainSplit'() becomes a built-in function it would make sense to use just one of the implementations throughout.

[javanna]

Thanks for the context: I have nothing against adding the function to the runtime fields contexts, but it looks like it may needed in other places too and this was raised before. I will leave it for the @elastic/es-core-infra team to discuss and decide.

[stu-elastic]

Let's move this into the core whitelists, as there is nothing core here, I'm going to remove the scripting label. Please reach out if you need help with the work.