Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML Inference Search Processors] Always return list when using dollar symbol in input_maps #2978

Closed
wants to merge 1 commit into from

Conversation

mingshl
Copy link
Collaborator

@mingshl mingshl commented Sep 23, 2024

Description

fixing #2974

by adding a helpful method getMappedInputFromObject when reading search query/ search response _source document,

 * Retrieves the mapped input value from the object based on the provided field name.
 * when the field name start with a dollar symbol, which starts all path expressions in JSONPATH notation,
 * default to always return lists
 * otherwise, return the original format of elements retrieving from the fieldName

For example,

in search processors:

for the same item index, if configuring input_map as {"input": "$.item.text"},

the model input will be in list representation.

{
  "input": ["red shoes"]
}

for the same item index, if configuring input_map as {"input": "item.text"},

the model input will be in list representation.

{
  "input": "red shoes"
}

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@mingshl
Copy link
Collaborator Author

mingshl commented Sep 23, 2024

rerunning flaky tests

RestConnectorToolIT > testConnectorToolInFlowAgent STANDARD_ERROR
    REPRODUCE WITH: ./gradlew ':opensearch-ml-plugin:integTest' --tests "org.opensearch.ml.rest.RestConnectorToolIT.testConnectorToolInFlowAgent" -Dtests.seed=FF62341AF71E5511 -Dtests.security.manager=false -Dtests.locale=cv-Cyrl-RU -Dtests.timezone=America/Thunder_Bay -Druntime.java=21


RestConnectorToolIT > testConnectorToolInFlowAgent FAILED
REPRODUCE WITH: ./gradlew ':opensearch-ml-plugin:integTest' --tests "org.opensearch.ml.rest.RestConnectorToolIT.testConnectorToolInFlowAgent" -Dtests.seed=FF62341AF71E5511 -Dtests.security.manager=false -Dtests.locale=cv-Cyrl-RU -Dtests.timezone=America/Thunder_Bay -Druntime.java=21
    org.opensearch.client.ResponseException: method [POST], host [http://[::1]:42541], URI [/_plugins/_ml/agents/2WmEIJIBugkMbA_Nmc3O/_execute], status line [HTTP/1.1 500 Internal Server Error]
    {"status":500,"error":{"type":"OpenSearchStatusException","reason":"System Error","details":"Error communicating with remote model: bedrock-runtime.us-west-2.amazonaws.com: Name or service not known"}}
        at __randomizedtesting.SeedInfo.seed([FF62341AF71E5511:DAC9050B68D574D9]:0)
        at app//org.opensearch.client.RestClient.convertResponse(RestClient.java:501)
        at app//org.opensearch.client.RestClient.performRequest(RestClient.java:384)
        at app//org.opensearch.client.RestClient.performRequest(RestClient.java:359)
        at app//org.opensearch.ml.utils.TestHelper.makeRequest(TestHelper.java:182)
        at app//org.opensearch.ml.utils.TestHelper.makeRequest(TestHelper.java:155)
        at app//org.opensearch.ml.utils.TestHelper.makeRequest(TestHelper.java:144)
        at app//org.opensearch.ml.rest.RestBaseAgentToolsIT.executeAgent(RestBaseAgentToolsIT.java:84)
        at app//org.opensearch.ml.rest.RestConnectorToolIT.testConnectorToolInFlowAgent(RestConnectorToolIT.java:140)

RestCon

@mingshl
Copy link
Collaborator Author

mingshl commented Oct 1, 2024

after considering standardization across ml commons repo, taking another fix #2985 closing this PR

@mingshl mingshl closed this Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant