Standardise wildcard matching

[clintongormley]

With the new path_filter parameter (#10980) we support wildcards with * matching a single "segment" of the path and ** matching multiple segments.

Currently we use slightly different simple wildcards in a number of other places, such as when looking up field names. Index wildcards also support a leading - to exclude matches.

I'd like to propose that we standardise the syntax across all APIs that use wildcard matching, as follows:

• * matches zero or more characters up to but not including a period .
• ** matches all zero or more characters including a period
• multiple patterns can be separated by a comma ,
• any pattern can be preceded by a + to include matches (the default if not specified) or a - to exclude matches
• it should be possible to escape any of these characters: *, +, -, ,, \ with a preceding \

Relates to #11294

[nik9000]

Highlighting has an issue about excluding fields from * that this applies to. I don't have the number handy.

[clintongormley]

thanks @nik9000 - it is #9881

[abibell]

If we used Regex would that prevent us from hand crafting the parsing? How much of performance hit is that going to be? We could execute regex only when we detect the presence of regex.

[clintongormley]

You'd have to have some marker to indicate that it is a regex, otherwise foo* would be ambiguous. The obvious marker would be to wrap the pattern in /, but not all users of wildcard matching could support regexes. For instance the streaming path filtering is highly optimised because it only considers segments. If we were to change that to regexes it would have a big performance overhead.

[clintongormley]

Discussing this some more: changing the index pattern matching would be problematic, eg you'd have to escape the . in .marvel etc. Probably not worth making this change everywhere.

[clintongormley]

Won't fix