Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

adding tenantId field in model group index #2862

Merged
merged 1 commit into from
Aug 29, 2024
Merged

adding tenantId field in model group index #2862

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 51 additions & 50 deletions common/src/main/java/org/opensearch/ml/common/CommonValue.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ public class CommonValue {
public static final String ML_MODEL_GROUP_INDEX = ".plugins-ml-model-group";
public static final String ML_MODEL_INDEX = ".plugins-ml-model";
public static final String ML_TASK_INDEX = ".plugins-ml-task";
public static final Integer ML_MODEL_GROUP_INDEX_SCHEMA_VERSION = 2;
public static final Integer ML_MODEL_GROUP_INDEX_SCHEMA_VERSION = 3;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self as a reminder. When we eventually get this on main we'll have to make sure there aren't any other conflicting schema updates.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

public static final Integer ML_MODEL_INDEX_SCHEMA_VERSION = 11;
public static final String ML_CONNECTOR_INDEX = ".plugins-ml-connector";
public static final Integer ML_TASK_INDEX_SCHEMA_VERSION = 2;
Expand Down Expand Up @@ -86,55 +86,56 @@ public class CommonValue {
+ " }\n"
+ " }\n";
public static final String ML_MODEL_GROUP_INDEX_MAPPING = "{\n"
+ " \"_meta\": {\n"
+ " \"schema_version\": " + ML_MODEL_GROUP_INDEX_SCHEMA_VERSION + "\n"
+ " },\n"
+ " \"properties\": {\n"
+ " \"" + MLModelGroup.MODEL_GROUP_NAME_FIELD + "\": {\n"
+ " \"type\": \"text\",\n"
+ " \"fields\": {\n"
+ " \"keyword\": {\n"
+ " \"type\": \"keyword\",\n"
+ " \"ignore_above\": 256\n"
+ " }\n"
+ " }\n"
+ " },\n"
+ " \"" + MLModelGroup.DESCRIPTION_FIELD + "\": {\n"
+ " \"type\": \"text\"\n"
+ " },\n"
+ " \"" + MLModelGroup.LATEST_VERSION_FIELD + "\": {\n"
+ " \"type\": \"integer\"\n"
+ " },\n"
+ " \"" + MLModelGroup.MODEL_GROUP_ID_FIELD + "\": {\n"
+ " \"type\": \"keyword\"\n"
+ " },\n"
+ " \"" + MLModelGroup.BACKEND_ROLES_FIELD + "\": {\n"
+ " \"type\": \"text\",\n"
+ " \"fields\": {\n"
+ " \"keyword\": {\n"
+ " \"type\": \"keyword\",\n"
+ " \"ignore_above\": 256\n"
+ " }\n"
+ " }\n"
+ " },\n"
+ " \"" + MLModelGroup.ACCESS + "\": {\n"
+ " \"type\": \"keyword\"\n"
+ " },\n"
+ " \"" + MLModelGroup.OWNER + "\": {\n"
+ " \"type\": \"nested\",\n"
+ " \"properties\": {\n"
+ " \"name\": {\"type\":\"text\", \"fields\":{\"keyword\":{\"type\":\"keyword\", \"ignore_above\":256}}},\n"
+ " \"backend_roles\": {\"type\":\"text\", \"fields\":{\"keyword\":{\"type\":\"keyword\"}}},\n"
+ " \"roles\": {\"type\":\"text\", \"fields\":{\"keyword\":{\"type\":\"keyword\"}}},\n"
+ " \"custom_attribute_names\": {\"type\":\"text\", \"fields\":{\"keyword\":{\"type\":\"keyword\"}}}\n"
+ " }\n"
+ " },\n"
+ " \"" + MLModelGroup.CREATED_TIME_FIELD + "\": {\n"
+ " \"type\": \"date\", \"format\": \"strict_date_time||epoch_millis\"},\n"
+ " \"" + MLModelGroup.LAST_UPDATED_TIME_FIELD + "\": {\n"
+ " \"type\": \"date\", \"format\": \"strict_date_time||epoch_millis\"}\n"
+ " }\n"
+ "}";
+ " \"_meta\": {\n"
+ " \"schema_version\": " + ML_MODEL_GROUP_INDEX_SCHEMA_VERSION + "\n"
+ " },\n"
+ " \"properties\": {\n"
+ " \"" + MLModelGroup.MODEL_GROUP_NAME_FIELD + "\": {\n"
+ " \"type\": \"text\",\n"
+ " \"fields\": {\n"
+ " \"keyword\": {\n"
+ " \"type\": \"keyword\",\n"
+ " \"ignore_above\": 256\n"
+ " }\n"
+ " }\n"
+ " },\n"
+ " \"" + MLModelGroup.DESCRIPTION_FIELD + "\": {\n"
+ " \"type\": \"text\"\n"
+ " },\n"
+ " \"" + MLModelGroup.LATEST_VERSION_FIELD + "\": {\n"
+ " \"type\": \"integer\"\n"
+ " },\n"
+ " \"" + MLModelGroup.MODEL_GROUP_ID_FIELD + "\": {\n"
+ " \"type\": \"keyword\"\n"
+ " },\n"
+ " \"" + MLModelGroup.BACKEND_ROLES_FIELD + "\": {\n"
+ " \"type\": \"text\",\n"
+ " \"fields\": {\n"
+ " \"keyword\": {\n"
+ " \"type\": \"keyword\",\n"
+ " \"ignore_above\": 256\n"
+ " }\n"
+ " }\n"
+ " },\n"
+ " \"" + TENANT_ID + "\" : {\"type\": \"keyword\"},\n"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works 'cause it's valid JSON, but all the other strings (see MlModelGroup.ACCESS right below) do the "pretty" JSON equivalent. Might want to be consistent.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

{
    "took": 11,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": ".plugins-ml-model-group",
                "_id": "k7BBoJEBmupm26gNRc4D",
                "_score": 1.0,
                "_source": {
                    "name": "Cohere embedding normal",
                    "latest_version": 1,
                    "description": "embedding sagemaker model",
                    "access": "public",
                    "created_time": 1724970517643,
                    "last_updated_time": 1724970517930,
                    "tenant_id": "1234567"
                }
            }
        ]
    }
}

This is mapping I got after making this change. Seems like similar?

+ " \"" + MLModelGroup.ACCESS + "\": {\n"
+ " \"type\": \"keyword\"\n"
+ " },\n"
+ " \"" + MLModelGroup.OWNER + "\": {\n"
+ " \"type\": \"nested\",\n"
+ " \"properties\": {\n"
+ " \"name\": {\"type\":\"text\", \"fields\":{\"keyword\":{\"type\":\"keyword\", \"ignore_above\":256}}},\n"
+ " \"backend_roles\": {\"type\":\"text\", \"fields\":{\"keyword\":{\"type\":\"keyword\"}}},\n"
+ " \"roles\": {\"type\":\"text\", \"fields\":{\"keyword\":{\"type\":\"keyword\"}}},\n"
+ " \"custom_attribute_names\": {\"type\":\"text\", \"fields\":{\"keyword\":{\"type\":\"keyword\"}}}\n"
+ " }\n"
+ " },\n"
+ " \"" + MLModelGroup.CREATED_TIME_FIELD + "\": {\n"
+ " \"type\": \"date\", \"format\": \"strict_date_time||epoch_millis\"},\n"
+ " \"" + MLModelGroup.LAST_UPDATED_TIME_FIELD + "\": {\n"
+ " \"type\": \"date\", \"format\": \"strict_date_time||epoch_millis\"}\n"
+ " }\n"
+ "}";

public static final String ML_CONNECTOR_INDEX_FIELDS = " \"properties\": {\n"
+ " \""
Expand Down
Loading