-
Notifications
You must be signed in to change notification settings - Fork 508
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
adding pattern_capture token filter docs #8240
Signed-off-by: Anton Rubin <[email protected]>
- Loading branch information
1 parent
76486a4
commit 630f33d
Showing
2 changed files
with
96 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,95 @@ | ||
--- | ||
layout: default | ||
title: Pattern capture | ||
parent: Token filters | ||
nav_order: 310 | ||
--- | ||
|
||
# Pattern capture token filter | ||
|
||
The `pattern_capture` token filter a powerful filter that uses regular expressions to capture and extract parts of text according to specific patterns. This filter can be useful when you want to extract particular parts of tokens, such as email domains, hashtags, or numbers, and reuse them for further analysis or indexing. | ||
|
||
## Parameters | ||
|
||
The `pattern_capture` token filter can be configured with the following parameters: | ||
|
||
- `patterns`: An array of regular expressions used to capture parts of the text. (Array of strings, _Required_) | ||
- `preserve_original`: Retain original token in the output. Default is `true`. (Boolean, _Optional_) | ||
|
||
|
||
## Example | ||
|
||
The following example request creates a new index named `email_index` and configures an analyzer with `pattern_capture` filter to extract the local part and domain from email address: | ||
|
||
```json | ||
PUT /email_index | ||
{ | ||
"settings": { | ||
"analysis": { | ||
"filter": { | ||
"email_pattern_capture": { | ||
"type": "pattern_capture", | ||
"preserve_original": true, | ||
"patterns": [ | ||
"^([^@]+)", | ||
"@(.+)$" | ||
] | ||
} | ||
}, | ||
"analyzer": { | ||
"email_analyzer": { | ||
"tokenizer": "uax_url_email", | ||
"filter": [ | ||
"email_pattern_capture", | ||
"lowercase" | ||
] | ||
} | ||
} | ||
} | ||
} | ||
} | ||
``` | ||
{% include copy-curl.html %} | ||
|
||
## Generated tokens | ||
|
||
Use the following request to examine the tokens generated using the analyzer: | ||
|
||
```json | ||
POST /email_index/_analyze | ||
{ | ||
"text": "[email protected]", | ||
"analyzer": "email_analyzer" | ||
} | ||
``` | ||
{% include copy-curl.html %} | ||
|
||
The response contains the generated tokens: | ||
|
||
```json | ||
{ | ||
"tokens": [ | ||
{ | ||
"token": "[email protected]", | ||
"start_offset": 0, | ||
"end_offset": 20, | ||
"type": "<EMAIL>", | ||
"position": 0 | ||
}, | ||
{ | ||
"token": "john.doe", | ||
"start_offset": 0, | ||
"end_offset": 20, | ||
"type": "<EMAIL>", | ||
"position": 0 | ||
}, | ||
{ | ||
"token": "example.com", | ||
"start_offset": 0, | ||
"end_offset": 20, | ||
"type": "<EMAIL>", | ||
"position": 0 | ||
} | ||
] | ||
} | ||
``` |