Get all search templates/scripts

[brusic]

TL;DR I want to get all the search templates defined in the cluster metadata and I am willing to implement it myself

I found myself needing to get all the search templates defined in the cluster metadata. Noticed that there was no endpoint to get all the templates, so I though I should just add it myself. After diving into the code, I noticed why it is not currently implemented.

Templates are just scripts, so the search template endpoints end up going through the script service, which does not support getAll. The actual scripts are stored in a collection referenced only by the id. I thought I implement a hack by simply iterating through all the script and looking at the key name, but the lang context has been recently removed. I could create another data structure to keep track of the lang context, but I do not like the idea of supporting different data structures (basically multi key maps).

There has been some discussion about splitting templates from scripts (#16314) with an initial attempt by @nik9000 (#23744). I did not review the rejected PR, but perhaps it would support exactly what I need. If not, this separation would have been useful since the template service would support my (eventual) PR.

My next step would be to implement a generic getAllStoredScripts type class/method, which would support a lang param. Two issues come to mind:

1. The implementation for my use case would be rendered obsolete if Split templates from scripts? #16314 gets implemented.
2. Would highlighting all the scripts be a security concern? Due to potential overload, I would have implemented it as a cat endpoint with the actual script not being returned (I just need names).

Would it be wise to implement such an endpoint? I do not want to code something, only to have a fail to be picked up for non-technical reasons.

[rjernst]

@brusic The work on script contexts (#20426) has rendered the need for a separate template service obsolete. The separate stored template api will also soon be removed, in favor of using the stored scripts api (#24596). While designing the script contexts, we considered having context be a required parameter to storing a script. However, this was deemed too cumbersome for users. The stored script API does now allow passing a context (#25014), but this context is only used for validation of the script for that context, and is not stored with it.

Essentially, the stored scripts are allowed to be used in any context in which they will compile. I can see adding an api to retrieve a list of the stored script names (along with lang), but if you want to filter on the intended use, I think you should do so with your own naming convention.

[brusic]

I incorrectly used the term "context" to refer to the previously used "namespace". Previously, the keys were compound keys with the namespace and id. Does not matter that they are gone, decision making based on the key format is a hack and I should not have even thought about it!

In REST terms, I would love to do something like
GET /_cat/scripts?lang=mustache (although I do not like having an concrete implementation as the key name for templates) Technically what I need is /_cat/search_templates.

I was not aware of the other script issues. As always, Elasticsearch is a moving target and difficult to modify without knowing the direction. I will see how the stored scripts API pans out (subscribed to the relevant issues), although in the meantime I would need to keep search template state locally.

[rjernst]

I think before we have a cat API for scripts, we would need one that returns the scripts directly. I'm going to mark this issue as adoptme and low hanging fruit because I think it should be rather easy. I still recommend using your own naming convention in order to allow know "these scripts will be used as templates".

[elisabeth-sorrell]

@brusic Have you already started working on this?

[brusic]

Quite honestly, I have forgotten about it since it was no longer needed for the project I was working for at that time. Since 6.x is out, I might give it a shot.

[SachithS]

Hi there,
I'm first-timer to the project. Can I start working on this? Please guide me on this.

[brusic]

@SachithS I have already submitted a pull request for this issue. It is referenced above. It will require a breaking change, so I doubt it will be released before 7.0. Still awaiting on the comments to the PR.

[SachithS]

@brusic , thanks for letting me know. Will try to find a task to work on.