-
Notifications
You must be signed in to change notification settings - Fork 25.6k
[DOCS] Document range enrich policy #79607
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
92c5b4c
Adding docs for the range enrich policy
mjmbischoff f30a014
Fixing reference to match instead of range
mjmbischoff f2b6628
Merge branch 'elastic:master' into range_enrich_docs
mjmbischoff 4d78c8e
Addressing assertion failure, 'downgrading' to normal error by moving…
mjmbischoff 6db6165
Missing `// TEST[continued]` breaks chain leading to 3 distinct tests…
mjmbischoff af56557
Revert "Addressing assertion failure, 'downgrading' to normal error b…
mjmbischoff 471b245
Fixing TESTRESPONSE
mjmbischoff 3765972
Improving cleanup of other enrich examples
mjmbischoff 58c3773
Remove unneeded snippet comments
jrodewig 2aba898
Update create enrich policy API docs
jrodewig 30dda53
Fix typo
jrodewig 81d7117
Fix xref
jrodewig 48b19cd
include docs
jrodewig 42df797
clarify API docs
jrodewig 348c1a3
Processing review comments
mjmbischoff 935f796
Fix API docs desc
jrodewig 450c97d
Move includes to right place
jrodewig a43a9c0
Clean up API desc + example intro
jrodewig 84e997b
fix typos
jrodewig 335716f
cleanup
jrodewig File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
164 changes: 164 additions & 0 deletions
164
docs/reference/ingest/range-enrich-policy-type-ex.asciidoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,164 @@ | ||
| [role="xpack"] | ||
| [testenv="basic"] | ||
| [[range-enrich-policy-type]] | ||
| === Example: Enrich your data by matching a value to a range | ||
|
|
||
| A `range` <<enrich-policy,enrich policy>> uses a <<query-dsl-term-query,`term` | ||
| query>> to match a number, date, or IP address in incoming documents to a range | ||
| of the same type in the enrich index. Matching a range to a range is not | ||
| supported. | ||
|
|
||
| The following example creates a `range` enrich policy that adds a descriptive network name and | ||
| responsible department to incoming documents based on an IP address. It then | ||
| adds the enrich policy to a processor in an ingest pipeline. | ||
|
|
||
| Use the <<indices-create-index, create index API>> with the appropriate mappings to create a source index. | ||
|
|
||
| [source,console] | ||
| ---- | ||
| PUT /networks | ||
| { | ||
| "mappings": { | ||
| "properties": { | ||
| "range": { "type": "ip_range" }, | ||
| "name": { "type": "keyword" }, | ||
| "department": { "type": "keyword" } | ||
| } | ||
| } | ||
| } | ||
| ---- | ||
|
|
||
| The following index API request indexes a new document to that index. | ||
|
|
||
| [source,console] | ||
| ---- | ||
| PUT /networks/_doc/1?refresh=wait_for | ||
| { | ||
| "range": "10.100.0.0/16", | ||
| "name": "production", | ||
| "department": "OPS" | ||
| } | ||
| ---- | ||
| // TEST[continued] | ||
|
|
||
| Use the create enrich policy API to create an enrich policy with the | ||
| `range` policy type. This policy must include: | ||
|
|
||
| * One or more source indices | ||
| * A `match_field`, | ||
| the field from the source indices used to match incoming documents | ||
| * Enrich fields from the source indices you'd like to append to incoming | ||
| documents | ||
|
|
||
| Since we plan to enrich documents based on an IP address, the policy's | ||
| `match_field` must be an `ip_range` field. | ||
|
|
||
| [source,console] | ||
| ---- | ||
| PUT /_enrich/policy/networks-policy | ||
| { | ||
| "range": { | ||
| "indices": "networks", | ||
| "match_field": "range", | ||
| "enrich_fields": ["name", "department"] | ||
| } | ||
| } | ||
| ---- | ||
| // TEST[continued] | ||
|
|
||
| Use the <<execute-enrich-policy-api,execute enrich policy API>> to create an | ||
| enrich index for the policy. | ||
|
|
||
| [source,console] | ||
| ---- | ||
| POST /_enrich/policy/networks-policy/_execute | ||
| ---- | ||
| // TEST[continued] | ||
|
|
||
|
|
||
| Use the <<put-pipeline-api,create or update pipeline API>> to create an ingest | ||
| pipeline. In the pipeline, add an <<enrich-processor,enrich processor>> that | ||
| includes: | ||
|
|
||
| * Your enrich policy. | ||
| * The `field` of incoming documents used to match documents | ||
| from the enrich index. | ||
| * The `target_field` used to store appended enrich data for incoming documents. | ||
| This field contains the `match_field` and `enrich_fields` specified in your | ||
| enrich policy. | ||
jrodewig marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| [source,console] | ||
| ---- | ||
| PUT /_ingest/pipeline/networks_lookup | ||
| { | ||
| "processors" : [ | ||
| { | ||
| "enrich" : { | ||
| "description": "Add 'network' data based on 'ip'", | ||
| "policy_name": "networks-policy", | ||
| "field" : "ip", | ||
| "target_field": "network", | ||
| "max_matches": "10" | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ---- | ||
| // TEST[continued] | ||
|
|
||
| Use the ingest pipeline to index a document. The incoming document should | ||
| include the `field` specified in your enrich processor. | ||
|
|
||
| [source,console] | ||
| ---- | ||
| PUT /my-index-000001/_doc/my_id?pipeline=networks_lookup | ||
| { | ||
| "ip": "10.100.34.1" | ||
| } | ||
| ---- | ||
| // TEST[continued] | ||
|
|
||
| To verify the enrich processor matched and appended the appropriate field data, | ||
| use the <<docs-get,get API>> to view the indexed document. | ||
|
|
||
| [source,console] | ||
| ---- | ||
| GET /my-index-000001/_doc/my_id | ||
| ---- | ||
| // TEST[continued] | ||
|
|
||
| The API returns the following response: | ||
|
|
||
| [source,console-result] | ||
| ---- | ||
| { | ||
| "_index" : "my-index-000001", | ||
| "_id" : "my_id", | ||
| "_version" : 1, | ||
| "_seq_no" : 0, | ||
| "_primary_term" : 1, | ||
| "found" : true, | ||
| "_source" : { | ||
| "ip" : "10.100.34.1", | ||
| "network" : [ | ||
| { | ||
| "name" : "production", | ||
| "range" : "10.100.0.0/16", | ||
| "department" : "OPS" | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| ---- | ||
| // TESTRESPONSE[s/"_seq_no": \d+/"_seq_no" : $body._seq_no/ s/"_primary_term":1/"_primary_term" : $body._primary_term/] | ||
|
|
||
| //// | ||
| [source,console] | ||
| -------------------------------------------------- | ||
| DELETE /_ingest/pipeline/networks_lookup | ||
| DELETE /_enrich/policy/networks-policy | ||
| DELETE /networks | ||
| DELETE /my-index-000001 | ||
| -------------------------------------------------- | ||
| // TEST[continued] | ||
| //// | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.