Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

invoke ml pipeline always failing. The link in the POST Method link gives an error { "Error": { "Code": "UnsupportedApiVersion", "Message": "The HTTP resource that matches the request URI 'https://centralus.aether.ms/api/v1.0/subscriptions/19c1eed4-f0da-4a7c-8bd9-e82e6700f2b7/resourceGroups/mmlops-RG/providers/Microsoft.MachineLearningServices/workspaces/mosml008/PipelineRuns/PipelineSubmit/3ae98478-5ffa-414a-8363-a3e3b3af42db' with API version '1.0' does not support HTTP method 'GET'.", "InnerError": null } } #418

Open
codemohsmohammad789 opened this issue Jul 12, 2023 · 1 comment

Comments

@codemohsmohammad789
Copy link

I am trying to run the train pipeline, I get an error everytime I am unable to figure it out. In the POn ST method link when I run it get an error unsupported AI verson 1.0 error. Please help I have been trying to run the invoke Ml Pipeline from a long time

Job preparation parameters
Variables:
ACI_DEPLOYMENT_NAME: $[ variablegroups.devopsforai-aml-vg.ACI_DEPLOYMENT_NAME ]
AZURE_RM_SVC_CONNECTION: $[ variablegroups.devopsforai-aml-vg.AZURE_RM_SVC_CONNECTION ]
BASE_NAME: $[ variablegroups.devopsforai-aml-vg.BASE_NAME ]
LOCATION: $[ variablegroups.devopsforai-aml-vg.LOCATION ]
RESOURCE_GROUP: $[ variablegroups.devopsforai-aml-vg.RESOURCE_GROUP ]
WORKSPACE_NAME: $[ variablegroups.devopsforai-aml-vg.WORKSPACE_NAME ]
WORKSPACE_SVC_CONNECTION: $[ variablegroups.devopsforai-aml-vg.WORKSPACE_SVC_CONNECTION ]
AMLPIPELINE_ID:
Parsing expression: <dependencies.Get_Pipeline_ID.outputs['getpipelineid.AMLPIPELINEID']>
Evaluating: dependencies['Get_Pipeline_ID']['outputs']['getpipelineid.AMLPIPELINEID']
Result: '3ae98478-5ffa-414a-8363-a3e3b3af42db'
ContinueOnError: False
TimeoutInMinutes: 0
CancelTimeoutInMinutes: 5
Expand:
MaxConcurrency: 0
MaxConcurrency: 0

Task : Invoke ML pipeline
Description : Invoke Azure Machine Learning published pipelines.
Version : 0.1.90
Author : ms-air-aiagility
Help URL : https://marketplace.visualstudio.com/items?itemName=ms-air-aiagility.vss-services-azureml&ssr=false#overview

2023-07-12T11:06:47.6918609Z POST https://centralus.aether.ms/api/v1.0/subscriptions/19c1eed4-f0da-4a7c-8bd9-e82e6700f2b7/resourceGroups/mmlops-RG/providers/Microsoft.MachineLearningServices/workspaces/mosml008/PipelineRuns/PipelineSubmit/3ae98478-5ffa-414a-8363-a3e3b3af42db
Request body: {
"AzDevOpsCallBackInfo": {
"JobId": "77aee9c5-85bc-59ae-fa90-c3f20f9c111e",
"HubName": "build",
"TaskInstanceId": "7ffef4a3-944d-544a-2764-6fcc510d84c9",
"PlanId": "6bebf0d7-465c-4c93-ad8e-49e907470d26",
"ProjectId": "0c53d160-630b-4a98-aa55-51747305200b",
"PlanUri": "https://dev.azure.com/mohsmohammad7890561/",
"Auth***": ***
},
"ExperimentName": "mlopspython",
"RunSource": "PipelineRun",
"ParameterAssignments": {
"model_name": "diabetes_regression_model.pkl"
},
"tags": {
"BuildId": "36",
"BuildUri": "https://dev.azure.com/mohsmohammad7890561/MLOPS/_build/results?buildId=36"
},
"StepTags": {
"BuildId": "36",
"BuildUri": "https://dev.azure.com/mohsmohammad7890561/MLOPS/_build/results?buildId=36"
}
}
Response Code: OK
Response: {
"Description": null,
"Status": {
"StatusCode": 0,
"StatusDetail": null,
"CreationTime": "2023-07-12T11:06:47.1409291Z",
"EndTime": null
},
"GraphId": "98f140f2-05f8-4064-9915-5de767a25c61",
"IsSubmitted": false,
"HasErrors": false,
"HasWarnings": false,
"UploadState": 0,
"ParameterAssignments": {
"model_name": "diabetes_regression_model.pkl",
"dataset_version": "latest",
"data_file_path": "none",
"caller_run_id": "none"
},
"DataPathAssignments": {},
"DataSetDefinitionValueAssignments": {},
"AssetOutputSettingsAssignments": {},
"RunHistoryExperimentName": "mlopspython",
"DisplayName": null,
"PipelineRunId": "8ed03654-914e-4c2a-b633-3090bdf94669",
"PipelineId": "3ae98478-5ffa-414a-8363-a3e3b3af42db",
"PipelineEndpointId": null,
"RunSource": "PipelineRun",
"RunType": 0,
"TotalRunSteps": 3,
"ScheduleId": null,
"RunUrl": "https://ml.azure.com/experiments/mlopspython/runs/8ed03654-914e-4c2a-b633-3090bdf94669?tid=39ac8b47-69b1-4db6-b731-b6591aee89d3&wsid=/subscriptions/19c1eed4-f0da-4a7c-8bd9-e82e6700f2b7/resourcegroups/mmlops-RG/workspaces/mosml008",
"tags": {
"BuildId": "36",
"BuildUri": "https://dev.azure.com/mohsmohammad7890561/MLOPS/_build/results?buildId=36"
},
"StepTags": {
"BuildId": "36",
"BuildUri": "https://dev.azure.com/mohsmohammad7890561/MLOPS/_build/results?buildId=36"
},
"Properties": {},
"StepProperties": {},
"CreatedBy": {
"UserObjectId": "52e7966a-2a24-4e93-883b-50f47ff658e2",
"UserTenantId": "39ac8b47-69b1-4db6-b731-b6591aee89d3",
"UserName": "3bd719d8-eced-42b7-9a65-5d43bb19f565"
},
"PreserveSubGraphs": true,
"RootPipelineRunId": "8ed03654-914e-4c2a-b633-3090bdf94669",
"EnforceRerun": null,
"ContinueRunOnFailedOptionalInput": true,
"UserAlias": null,
"DefaultCloudPriority": null,
"DefaultCompute": null,
"DefaultDatastore": null,
"IdentityConfig": null,
"PipelineTimeoutSeconds": null,
"ContinueRunOnStepFailure": false,
"EntityStatus": 0,
"Id": "8ed03654-914e-4c2a-b633-3090bdf94669",
"Etag": ""4000e293-0000-0300-0000-64ae89470000"",
"CreatedDate": "2023-07-12T11:06:47.1604515Z",
"LastModifiedDate": "2023-07-12T11:06:47.1604515Z"
}

@senar6
Copy link

senar6 commented Sep 9, 2024

Ideally this tasks fails while creating conda environment., and there is no proper error while creating as well. please help to resolve the issue

======Starting Image Build on Serverless Compute======
The run ID for the image build on serverless compute is imgbldrun_725f1f0
Additional logs for the run: https://ml.azure.com/experiments/id/prepare_image/runs/imgbldrun_725f1f0?wsid=/subscriptions/eae29237-49ec-42cc-82e7-dc1125499c88/resourcegroups/rg-sandpox-amrc/workspaces/sandbox-amrc-aml&tid=6f2b364d-3ab6-42e1-a351-e4d69eb92fe9
2024-09-05T15:26:38: Logging into Docker registry: d30f8cdd495e4db28bb266fa51695ba3.azurecr.io
2024-09-05T15:26:38: WARNING! Using --password via the CLI is insecure. Use --password-stdin.
2024-09-05T15:26:38: WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
2024-09-05T15:26:38: Configure a credential helper to remove this warning. See
2024-09-05T15:26:38: https://docs.docker.com/engine/reference/commandline/login/#credentials-store

2024-09-05T15:26:38: Login Succeeded

2024-09-05T15:26:38: Running: ['docker', 'build', '-f', 'azureml-environment-setup/Dockerfile', '.', '-t', 'd30f8cdd495e4db28bb266fa51695ba3.azurecr.io/azureml/azureml_134293382750c1bb119025659155ea48', '-t', 'd30f8cdd495e4db28bb266fa51695ba3.azurecr.io/azureml/azureml_134293382750c1bb119025659155ea48:1']
2024-09-05T15:26:38: DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
2024-09-05T15:26:38: Install the buildx component to build images with BuildKit:
2024-09-05T15:26:38: https://docs.docker.com/go/buildx/

2024-09-05T15:26:38: Sending build context to Docker daemon 71.17kB
2024-09-05T15:26:38: Step 1/17 : FROM mcr.microsoft.com/azureml/intelmpi2018.3-ubuntu16.04:20210301.v1@sha256:000d6c43f606ceaa67983790ca95c70fd741c364d8c2e3217a11d775b99741df
2024-09-05T15:26:38: mcr.microsoft.com/azureml/intelmpi2018.3-ubuntu16.04:20210301.v1@sha256:000d6c43f606ceaa67983790ca95c70fd741c364d8c2e3217a11d775b99741df: Pulling from azureml/intelmpi2018.3-ubuntu16.04
2024-09-05T15:26:38: 4007a89234b4: Pulling fs layer
2024-09-05T15:26:38: 5dfa26c6b9c9: Pulling fs layer
2024-09-05T15:26:38: 0ba7bf18aa40: Pulling fs layer
2024-09-05T15:26:38: 4c6ec688ebe3: Pulling fs layer
2024-09-05T15:26:38: 574f361512d6: Pulling fs layer
2024-09-05T15:26:38: db4d1e2d7079: Pulling fs layer
2024-09-05T15:26:38: e544ee0f522d: Pulling fs layer
2024-09-05T15:26:38: c655136086be: Pulling fs layer
2024-09-05T15:26:38: 2ec37f44090c: Pulling fs layer
2024-09-05T15:26:38: 5fba3bd4a2c4: Pulling fs layer
2024-09-05T15:26:38: 7e0ea9d0a1ab: Pulling fs layer
2024-09-05T15:26:38: da005f826951: Pulling fs layer
2024-09-05T15:26:38: 4c6ec688ebe3: Waiting
2024-09-05T15:26:38: 574f361512d6: Waiting
2024-09-05T15:26:38: db4d1e2d7079: Waiting
2024-09-05T15:26:38: e544ee0f522d: Waiting
2024-09-05T15:26:38: 7e0ea9d0a1ab: Waiting
2024-09-05T15:26:38: c655136086be: Waiting
2024-09-05T15:26:38: da005f826951: Waiting
2024-09-05T15:26:38: 2ec37f44090c: Waiting
2024-09-05T15:26:38: 5fba3bd4a2c4: Waiting
2024-09-05T15:26:38: 0ba7bf18aa40: Verifying Checksum
2024-09-05T15:26:38: 0ba7bf18aa40: Download complete
2024-09-05T15:26:38: 5dfa26c6b9c9: Download complete
2024-09-05T15:26:38: 4c6ec688ebe3: Verifying Checksum
2024-09-05T15:26:38: 4c6ec688ebe3: Download complete
2024-09-05T15:26:39: db4d1e2d7079: Verifying Checksum
2024-09-05T15:26:39: db4d1e2d7079: Download complete
2024-09-05T15:26:39: 574f361512d6: Verifying Checksum
2024-09-05T15:26:39: 574f361512d6: Download complete
2024-09-05T15:26:40: e544ee0f522d: Verifying Checksum
2024-09-05T15:26:40: e544ee0f522d: Download complete
2024-09-05T15:26:40: 4007a89234b4: Verifying Checksum
2024-09-05T15:26:40: 4007a89234b4: Download complete
2024-09-05T15:26:40: 5fba3bd4a2c4: Verifying Checksum
2024-09-05T15:26:40: 5fba3bd4a2c4: Download complete
2024-09-05T15:26:41: 7e0ea9d0a1ab: Verifying Checksum
2024-09-05T15:26:41: 7e0ea9d0a1ab: Download complete
2024-09-05T15:26:41: da005f826951: Verifying Checksum
2024-09-05T15:26:41: da005f826951: Download complete
2024-09-05T15:26:43: 2ec37f44090c: Verifying Checksum
2024-09-05T15:26:43: 2ec37f44090c: Download complete
2024-09-05T15:26:43: 4007a89234b4: Pull complete
2024-09-05T15:26:44: c655136086be: Verifying Checksum
2024-09-05T15:26:44: c655136086be: Download complete
2024-09-05T15:26:47: 5dfa26c6b9c9: Pull complete
2024-09-05T15:26:47: 0ba7bf18aa40: Pull complete
2024-09-05T15:26:47: 4c6ec688ebe3: Pull complete
2024-09-05T15:26:55: 574f361512d6: Pull complete
2024-09-05T15:26:56: db4d1e2d7079: Pull complete
2024-09-05T15:27:00: e544ee0f522d: Pull complete
2024-09-05T15:27:05: c655136086be: Pull complete
2024-09-05T15:27:11: 2ec37f44090c: Pull complete
2024-09-05T15:27:11: 5fba3bd4a2c4: Pull complete
2024-09-05T15:27:12: 7e0ea9d0a1ab: Pull complete
2024-09-05T15:27:12: da005f826951: Pull complete
2024-09-05T15:27:12: Digest: sha256:000d6c43f606ceaa67983790ca95c70fd741c364d8c2e3217a11d775b99741df
2024-09-05T15:27:12: Status: Downloaded newer image for mcr.microsoft.com/azureml/intelmpi2018.3-ubuntu16.04:20210301.v1@sha256:000d6c43f606ceaa67983790ca95c70fd741c364d8c2e3217a11d775b99741df
2024-09-05T15:27:12: ---> c942df5ba5d0
2024-09-05T15:27:12: Step 2/17 : USER root
2024-09-05T15:27:12: ---> Running in aa672599f5bc
2024-09-05T15:27:13: Removing intermediate container aa672599f5bc
2024-09-05T15:27:13: ---> 16f637ab12ee
2024-09-05T15:27:13: Step 3/17 : RUN mkdir -p $HOME/.cache
2024-09-05T15:27:13: ---> Running in 58efac0e7aec
2024-09-05T15:27:15: Removing intermediate container 58efac0e7aec
2024-09-05T15:27:15: ---> 0b9557f3ff21
2024-09-05T15:27:15: Step 4/17 : WORKDIR /
2024-09-05T15:27:15: ---> Running in 3ee153c2e13f
2024-09-05T15:27:16: Removing intermediate container 3ee153c2e13f
2024-09-05T15:27:16: ---> 856824b7accb
2024-09-05T15:27:16: Step 5/17 : COPY azureml-environment-setup/99brokenproxy /etc/apt/apt.conf.d/
2024-09-05T15:27:17: ---> be010d5a63a9
2024-09-05T15:27:17: Step 6/17 : RUN if dpkg --compare-versions conda --version | grep -oE '[^ ]+$' lt 4.4.11; then conda install conda==4.4.11; fi
2024-09-05T15:27:17: ---> Running in 8de29dbf8643
2024-09-05T15:27:20: Removing intermediate container 8de29dbf8643
2024-09-05T15:27:20: ---> 736f324cdb84
2024-09-05T15:27:20: Step 7/17 : COPY azureml-environment-setup/mutated_conda_dependencies.yml azureml-environment-setup/mutated_conda_dependencies.yml
2024-09-05T15:27:21: ---> 26d618b1d31e
2024-09-05T15:27:21: Step 8/17 : RUN ldconfig /usr/local/cuda/lib64/stubs && conda env create -p /azureml-envs/azureml_f7d0f75c0d0e79ca21c80a06521c5814 -f azureml-environment-setup/mutated_conda_dependencies.yml && rm -rf "$HOME/.cache/pip" && conda clean -aqy && CONDA_ROOT_DIR=$(conda info --root) && rm -rf "$CONDA_ROOT_DIR/pkgs" && find "$CONDA_ROOT_DIR" -type d -name pycache -exec rm -rf {} + && ldconfig
2024-09-05T15:27:21: ---> Running in 828ca357583d
2024-09-05T15:27:33: Collecting package metadata (repodata.json): ...working... done
2024-09-05T15:27:33: Solving environment: ...working... done

2024-09-05T15:27:33: Downloading and Extracting Packages
2024-09-05T15:27:34: ncurses-6.0 | 781 KB | ########## | 100%
2024-09-05T15:27:34: zlib-1.2.13 | 111 KB | ########## | 100%
2024-09-05T15:27:34: wheel-0.37.1 | 33 KB | ########## | 100%
2024-09-05T15:27:34: sqlite-3.23.1 | 808 KB | ########## | 100%
2024-09-05T15:27:35: python-3.6.2 | 23.6 MB | ########## | 100%
2024-09-05T15:27:35: xz-5.4.6 | 643 KB | ########## | 100%
2024-09-05T15:27:36: tk-8.6.14 | 3.4 MB | ########## | 100%
2024-09-05T15:27:36: readline-7.0 | 848 KB | ########## | 100%
2024-09-05T15:27:36: certifi-2021.5.30 | 139 KB | ########## | 100%
2024-09-05T15:27:36: libgcc-ng-11.2.0 | 5.3 MB | ########## | 100%
2024-09-05T15:27:37: ca-certificates-2024 | 127 KB | ########## | 100%
2024-09-05T15:27:37: libedit-3.1 | 151 KB | ########## | 100%
2024-09-05T15:27:37: libgomp-11.2.0 | 474 KB | ########## | 100%
2024-09-05T15:27:37: libstdcxx-ng-11.2.0 | 4.7 MB | ########## | 100%
2024-09-05T15:27:37: _libgcc_mutex-0.1 | 3 KB | ########## | 100%
2024-09-05T15:27:38: setuptools-58.0.4 | 788 KB | ########## | 100%
2024-09-05T15:27:38: libffi-3.2.1 | 48 KB | ########## | 100%
2024-09-05T15:27:38: openssl-1.0.2u | 2.2 MB | ########## | 100%
2024-09-05T15:27:38: pip-21.2.2 | 1.8 MB | ########## | 100%
2024-09-05T15:27:38: _openmp_mutex-5.1 | 21 KB | ########## | 100%
2024-09-05T15:27:39: Preparing transaction: ...working... done
2024-09-05T15:27:40: Verifying transaction: ...working... done
2024-09-05T15:27:41: Executing transaction: ...working... done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants