Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ReplaceStringProcessor for simple string substitution that doesn't involve regex #4954

Merged
merged 3 commits into from
Sep 18, 2024

Conversation

dinujoh
Copy link
Member

@dinujoh dinujoh commented Sep 17, 2024

Add ReplaceStringProcessor for simple string substitution that doesn't involve regex

Description

Add ReplaceStringProcessor for simple string substitution that doesn't involve regex. It uses java String.replace() method that is highly optimized for simple string replacements

Issues Resolved

Resolves #4953

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Dinu John <[email protected]>
dlvenable
dlvenable previously approved these changes Sep 17, 2024
Copy link
Member

@dlvenable dlvenable left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice improvement for pipeline authors to perform new mutations.

Thank you for using EventKey.

Copy link
Collaborator

@oeyh oeyh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good!

I wonder if we should merge this into the existing substitute_string processor as a different method. Something like this:

    - substitute_string:
        entries:
          - source: "message"
            from: "ab"
            to: "ef"
            method: "simple"  # default is "regex" for existing replace using regex matcher

@JsonPropertyDescription("The string to be substituted for each match of `from`.")
private String to;

@JsonProperty("substitute_when")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call it replace_when here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch

@dlvenable
Copy link
Member

The changes look good!

I wonder if we should merge this into the existing substitute_string processor as a different method. Something like this:

    - substitute_string:
        entries:
          - source: "message"
            from: "ab"
            to: "ef"
            method: "simple"  # default is "regex" for existing replace using regex matcher

So if I follow you, maybe we don't need a new processor here? Just an option?

@dinujoh
Copy link
Member Author

dinujoh commented Sep 17, 2024

The changes look good!

I wonder if we should merge this into the existing substitute_string processor as a different method. Something like this:

    - substitute_string:
        entries:
          - source: "message"
            from: "ab"
            to: "ef"
            method: "simple"  # default is "regex" for existing replace using regex matcher

Keeping them decoupled will help with simplicity.

@oeyh
Copy link
Collaborator

oeyh commented Sep 17, 2024

So if I follow you, maybe we don't need a new processor here? Just an option?

Yes. That's the idea. It reduces duplicate code and is more extensible if we ever add other string replace methods in the future. But I'm OK with the new processor added here as well.

@dinujoh dinujoh merged commit 2919e99 into opensearch-project:main Sep 18, 2024
45 of 47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add ReplaceStringProcessor for simple string substitution that doesn't involve regex
3 participants