Update documentation for automatic remote model deployment #6748

Zhangxunmt · 2024-03-20T21:55:24Z

Description

Update the Model Deploy API to include the automatic deploy feature for remote models in ML-Commons.

Issues Resolved

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

ylwu-amzn · 2024-03-20T22:51:09Z

Add auto deployment to this file https://github.com/opensearch-project/documentation-website/blob/main/_ml-commons-plugin/remote-models/index.md#step-4-deploy-the-model

Step 4: Deploy the model

From 2.13, we support automatically deploy remote model by default. You can disable it by setting plugins.ml_commons.model_auto_deploy.enable as false
PUT /_cluster/settings
{
    "persistent" : {
        "plugins.ml_commons.model_auto_deploy.enable" : false 
  }
}
To deploy the registered model manually, provide its model ID from step 3 in the following request:

ylwu-amzn · 2024-03-20T22:52:53Z

@Zhangxunmt add new enable auto deploy settings to this this doc https://github.com/opensearch-project/documentation-website/blob/main/_ml-commons-plugin/cluster-settings.md

Please also check if any other setting missed

Zhangxunmt · 2024-03-21T00:06:01Z

Updated accordingly. @ylwu-amzn

ylwu-amzn · 2024-03-21T00:18:11Z

_ml-commons-plugin/api/model-apis/deploy-model.md

@@ -8,7 +8,17 @@ nav_order: 20

 # Deploy a model

-The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`.
+The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. For remote models, by default the model is deployed automatically when it's predicted the first time through the Predict API. You can disable the remote model auto deployment by setting plugins.ml_commons.model_auto_deploy.enable to false. To learn more about remote models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index).


For remote models, by default the model is deployed automatically when it's predicted the first time through the Predict API.

Suggest add version information, we only support this from 2.13

ylwu-amzn · 2024-03-21T00:18:53Z

Updated accordingly. @ylwu-amzn

Have you addressed this comment #6748 (comment)

ylwu-amzn · 2024-03-21T16:57:38Z

_ml-commons-plugin/remote-models/index.md

+}
+```
+{% include copy-curl.html %}
+


Add note that user need to manually undeploy model if they don't need this model

kolchfa-aws · 2024-03-22T21:47:52Z

_ml-commons-plugin/api/model-apis/deploy-model.md

@@ -8,7 +8,17 @@ nav_order: 20

 # Deploy a model

-The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`.
+The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. For remote models, from 2.13 the model is deployed automatically by default when it's predicted the first time through the Predict API. You can disable the remote model auto deployment by setting plugins.ml_commons.model_auto_deploy.enable to false. To learn more about remote models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index).


Suggested change

The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. For remote models, from 2.13 the model is deployed automatically by default when it's predicted the first time through the Predict API. You can disable the remote model auto deployment by setting plugins.ml_commons.model_auto_deploy.enable to false. To learn more about remote models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index).

The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`.

Starting with OpenSearch version 2.13, [externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index) are deployed automatically by default when you send a Predict API request for the first time. You can disable the remote model auto deployment by setting `plugins.ml_commons.model_auto_deploy.enable` to `false`:

kolchfa-aws

Thanks, @Zhangxunmt! A couple of suggestions.

kolchfa-aws · 2024-03-22T21:52:08Z

_ml-commons-plugin/remote-models/index.md

@@ -205,7 +205,18 @@ Take note of the returned `model_id` because you’ll need it to deploy the mode

 ## Step 4: Deploy the model

-To deploy the registered model, provide its model ID from step 3 in the following request:
+From 2.13, we support automatically deploy remote model by default so this step can be skipped. You can disable it by setting plugins.ml_commons.model_auto_deploy.enable as false. To undeploy the model, please use the undeploy API to undeploy it.


Should we remove this step altogether and just add a sentence to the previous step: Starting with OpenSearch version 2.13, externally hosted models are deployed automatically by default when you send a Predict API request for the first time.

Let's keep this step because we still support manual deploy for the sake of BWC.

Signed-off-by: Xun Zhang <[email protected]>

Zhangxunmt · 2024-03-22T22:33:03Z

@kolchfa-aws updated based on your suggestion. Can you help approve and merge this PR?

kolchfa-aws · 2024-03-22T22:35:26Z

_ml-commons-plugin/api/model-apis/deploy-model.md

-The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`.
+The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. 
+
+Starting with OpenSearch version 2.13, [externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index) are deployed automatically by default when you send a Predict API request for the first time. You can disable the remote model auto deployment by setting `plugins.ml_commons.model_auto_deploy.enable` to `false`:


Suggested change

Starting with OpenSearch version 2.13, [externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index) are deployed automatically by default when you send a Predict API request for the first time. You can disable the remote model auto deployment by setting `plugins.ml_commons.model_auto_deploy.enable` to `false`:

Starting with OpenSearch version 2.13, [externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index) are deployed automatically by default when you send a Predict API request for the first time. To disable automatic deployment for an externally hosted model, set `plugins.ml_commons.model_auto_deploy.enable` to `false`:

kolchfa-aws · 2024-03-22T22:37:02Z

_ml-commons-plugin/remote-models/index.md

@@ -205,7 +205,18 @@ Take note of the returned `model_id` because you’ll need it to deploy the mode

 ## Step 4: Deploy the model

-To deploy the registered model, provide its model ID from step 3 in the following request:
+Starting with OpenSearch version 2.13, externally hosted models are deployed automatically by default when you send a Predict API request for the first time. You can disable it by setting plugins.ml_commons.model_auto_deploy.enable as false. To undeploy the model, please use the undeploy API to undeploy it.


Suggested change

Starting with OpenSearch version 2.13, externally hosted models are deployed automatically by default when you send a Predict API request for the first time. You can disable it by setting plugins.ml_commons.model_auto_deploy.enable as false. To undeploy the model, please use the undeploy API to undeploy it.

Starting with OpenSearch version 2.13, externally hosted models are deployed automatically by default when you send a Predict API request for the first time. To disable automatic deployment for an externally hosted model, set `plugins.ml_commons.model_auto_deploy.enable` to `false`:

kolchfa-aws · 2024-03-22T22:37:50Z

_ml-commons-plugin/remote-models/index.md

+}
+```
+{% include copy-curl.html %}
+


Suggested change

To undeploy the model, use the [Undeploy API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/model-apis/undeploy-model/).

kolchfa-aws · 2024-03-22T22:39:00Z

@Zhangxunmt A couple of more suggestions, and I will move the PR to editorial review. After the editorial comments are addressed, we can merge the PR. Thanks!

Signed-off-by: Xun Zhang <[email protected]>

Zhangxunmt · 2024-03-22T22:59:05Z

@Zhangxunmt A couple of more suggestions, and I will move the PR to editorial review. After the editorial comments are addressed, we can merge the PR. Thanks!

Updated accordingly. Please go ahead with the editorial review. @kolchfa-aws

natebower

@Zhangxunmt @kolchfa-aws Just one minor comment. Thanks!

natebower · 2024-03-25T13:02:44Z

_ml-commons-plugin/api/model-apis/deploy-model.md

@@ -8,7 +8,19 @@ nav_order: 20

 # Deploy a model

-The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`.
+The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. 


"in" instead of "into"?

_ml-commons-plugin/api/model-apis/deploy-model.md

Signed-off-by: kolchfa-aws <[email protected]>

Zhangxunmt requested review from hdhalter, kolchfa-aws, Naarcha-AWS, vagimeli, AMoo-Miki, natebower, dlvenable and stephen-crawford as code owners March 20, 2024 21:55

hdhalter added 4 - Doc review PR: Doc review in progress release-notes PR: Include this PR in the automated release notes v2.13.0 labels Mar 20, 2024

Zhangxunmt force-pushed the main branch 2 times, most recently from 8c690e3 to 6ea97f6 Compare March 21, 2024 00:05

ylwu-amzn reviewed Mar 21, 2024

View reviewed changes

hdhalter changed the title ~~update documentation for automatic remote model deployment~~ Update documentation for automatic remote model deployment Mar 21, 2024

kolchfa-aws self-assigned this Mar 21, 2024

ylwu-amzn reviewed Mar 21, 2024

View reviewed changes

Zhangxunmt force-pushed the main branch from dbfb7cb to 8fed784 Compare March 21, 2024 21:15

kolchfa-aws reviewed Mar 22, 2024

View reviewed changes

kolchfa-aws approved these changes Mar 22, 2024

View reviewed changes

Zhangxunmt added 3 commits March 22, 2024 15:30

update documentation for automatic remote model deployment

df091fc

Signed-off-by: Xun Zhang <[email protected]>

add automatic deploy doc in connecting to externall hosted models

1a504f0

Signed-off-by: Xun Zhang <[email protected]>

add remind to undeploy model

1fc0f2e

Signed-off-by: Xun Zhang <[email protected]>

Zhangxunmt force-pushed the main branch from 8fed784 to 62f3957 Compare March 22, 2024 22:31

kolchfa-aws reviewed Mar 22, 2024

View reviewed changes

Zhangxunmt force-pushed the main branch from 62f3957 to 45c27b3 Compare March 22, 2024 22:56

address comments

876f410

Signed-off-by: Xun Zhang <[email protected]>

Zhangxunmt force-pushed the main branch from 45c27b3 to 876f410 Compare March 22, 2024 22:58

natebower reviewed Mar 25, 2024

View reviewed changes

kolchfa-aws reviewed Mar 25, 2024

View reviewed changes

_ml-commons-plugin/api/model-apis/deploy-model.md Outdated Show resolved Hide resolved

Update _ml-commons-plugin/api/model-apis/deploy-model.md

e178681

Signed-off-by: kolchfa-aws <[email protected]>

kolchfa-aws merged commit 88242fa into opensearch-project:main Mar 25, 2024
3 checks passed

hdhalter added 3 - Done Issue is done/complete and removed 4 - Doc review PR: Doc review in progress labels Mar 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update documentation for automatic remote model deployment #6748

Update documentation for automatic remote model deployment #6748

Zhangxunmt commented Mar 20, 2024 •

edited by hdhalter

Loading

ylwu-amzn commented Mar 20, 2024 •

edited

Loading

Step 4: Deploy the model

ylwu-amzn commented Mar 20, 2024

Zhangxunmt commented Mar 21, 2024 •

edited

Loading

ylwu-amzn Mar 21, 2024

ylwu-amzn commented Mar 21, 2024 •

edited

Loading

ylwu-amzn Mar 21, 2024 •

edited

Loading

kolchfa-aws Mar 22, 2024 •

edited

Loading

kolchfa-aws left a comment

kolchfa-aws Mar 22, 2024

Zhangxunmt Mar 22, 2024

Zhangxunmt commented Mar 22, 2024

kolchfa-aws Mar 22, 2024

kolchfa-aws Mar 22, 2024 •

edited

Loading

kolchfa-aws Mar 22, 2024

kolchfa-aws commented Mar 22, 2024

Zhangxunmt commented Mar 22, 2024

natebower left a comment

natebower Mar 25, 2024

	Starting with OpenSearch version 2.13, [externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index) are deployed automatically by default when you send a Predict API request for the first time. You can disable the remote model auto deployment by setting `plugins.ml_commons.model_auto_deploy.enable` to `false`:
	Starting with OpenSearch version 2.13, [externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index) are deployed automatically by default when you send a Predict API request for the first time. To disable automatic deployment for an externally hosted model, set `plugins.ml_commons.model_auto_deploy.enable` to `false`:

	Starting with OpenSearch version 2.13, externally hosted models are deployed automatically by default when you send a Predict API request for the first time. You can disable it by setting plugins.ml_commons.model_auto_deploy.enable as false. To undeploy the model, please use the undeploy API to undeploy it.
	Starting with OpenSearch version 2.13, externally hosted models are deployed automatically by default when you send a Predict API request for the first time. To disable automatic deployment for an externally hosted model, set `plugins.ml_commons.model_auto_deploy.enable` to `false`:



	To undeploy the model, use the [Undeploy API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/model-apis/undeploy-model/).

Update documentation for automatic remote model deployment #6748

Update documentation for automatic remote model deployment #6748

Conversation

Zhangxunmt commented Mar 20, 2024 • edited by hdhalter Loading

Description

Issues Resolved

Checklist

ylwu-amzn commented Mar 20, 2024 • edited Loading

Step 4: Deploy the model

ylwu-amzn commented Mar 20, 2024

Zhangxunmt commented Mar 21, 2024 • edited Loading

ylwu-amzn Mar 21, 2024

Choose a reason for hiding this comment

ylwu-amzn commented Mar 21, 2024 • edited Loading

ylwu-amzn Mar 21, 2024 • edited Loading

Choose a reason for hiding this comment

kolchfa-aws Mar 22, 2024 • edited Loading

Choose a reason for hiding this comment

kolchfa-aws left a comment

Choose a reason for hiding this comment

kolchfa-aws Mar 22, 2024

Choose a reason for hiding this comment

Zhangxunmt Mar 22, 2024

Choose a reason for hiding this comment

Zhangxunmt commented Mar 22, 2024

kolchfa-aws Mar 22, 2024

Choose a reason for hiding this comment

kolchfa-aws Mar 22, 2024 • edited Loading

Choose a reason for hiding this comment

kolchfa-aws Mar 22, 2024

Choose a reason for hiding this comment

kolchfa-aws commented Mar 22, 2024

Zhangxunmt commented Mar 22, 2024

natebower left a comment

Choose a reason for hiding this comment

natebower Mar 25, 2024

Choose a reason for hiding this comment

Zhangxunmt commented Mar 20, 2024 •

edited by hdhalter

Loading

ylwu-amzn commented Mar 20, 2024 •

edited

Loading

Zhangxunmt commented Mar 21, 2024 •

edited

Loading

ylwu-amzn commented Mar 21, 2024 •

edited

Loading

ylwu-amzn Mar 21, 2024 •

edited

Loading

kolchfa-aws Mar 22, 2024 •

edited

Loading

kolchfa-aws Mar 22, 2024 •

edited

Loading