Insufficient documentation or examples for parameters in policy_assignments.update_by_id

[mlcooper]

Type of issue

Missing information

Description

My goal is to update a parameter of a policy with the policy_assignments.update_by_id method. The parameter happens to be of type Array (of strings).

My code executes cleanly, and I print the response successfully. No errors are returned. It is very unclear from the documentation about what content should go into the parameters parameter. Is it the full JSON structure of the Azure Policy and I only specify what I want to change in the parameter of the policy I am interested in? Is it just the name of the parameter in the policy and the value I want to add to its array? I have tried many different variations of the JSON in the parameters of this method, however none of them have been successful.

Why is my code not returning any errors if it fails to update the values in the array of the parameter?

No examples are given of policy_assignments.update_by_id in the examples repo maintained by Microsoft.

My code:

    credential = ClientSecretCredential(
        client_id=get_secrets()[0],
        client_secret=get_secrets()[1],
        tenant_id=os.getenv("TENANT_ID"),
    )

    policy_client = PolicyClient(
        api_version="2023-04-01",
        credential=credential,
        subscription_id=os.getenv("SUBSCRIPTION_ID"),
    )

    policy_assignment_ids = [
        {
            f"/providers/Microsoft.Management/managementGroups/{os.getenv("TENANT_ID")}/providers/Microsoft.Authorization/policyAssignments/AiModelControlBuiltInCst": "allowedModels"
        },
        {
            f"/providers/Microsoft.Management/managementGroups/{os.getenv("TENANT_ID")}/providers/Microsoft.Authorization/policyAssignments/AiModelControlBuiltIn": "allowedAssetIds"
        },
    ]

    for dict_item in policy_assignment_ids:
        for assignment, v in dict_item.items():
            k = policy_client.policy_assignments.get_by_id(
                policy_assignment_id=assignment
            )
            m = k.parameters[v]
            current_models = m.value
            diff_list = np.setdiff1d(new_model_list, current_models, assume_unique=True)
            if len(diff_list) > 0:
                for new_model in diff_list:
                    print(
                        f"Adding model '{new_model}' to policy assignment '{assignment.split("/")[8]}'."
                    )
                    # Add the new model to the policy parameter.
                    update_response = policy_client.policy_assignments.update_by_id(
                        policy_assignment_id=assignment,
                        parameters={"properties": {"parameters": {v: {"value": [new_model]}, "effect":{"value": "Deny"}}}}
                    )
                    r = update_response.parameters
                    resp = r[v]
                    print(resp)
                    print("\n")

The response:

Adding model 'mlc,1' to policy assignment 'AiModelControlBuiltInCst'.
{'additional_properties': {}, 'value': ['babbage-002,2', 'text-embedding-ada-002,2', 'davinci-002,3', 'text-embedding-3-large,1', 'text-embedding-3-small,1', 'tts,1', 'tts-hd,1', 'whisper-001,2', 'whisper,1', 'dall-e-2,2.0', 'dall-e-3,3.0', 'gpt-35-turbo-16k,0613', 'gpt-35-turbo-instruct,0914', 'gpt-4-32k,0613', 'gpt-4o,2024-08-06', 'gpt-4o-mini,2024-07-18', 'o1-mini,1', 'gpt-35-turbo,0125', 'gpt-4,turbo-2024-04-09', 'gpt-35-turbo,1106', 'gpt-35-turbo,0301', 'gpt-4o,2024-05-13', 'tts-hd,001', 'whisper,001', 'gpt-4,0613', 'gpt-4,0314', 'davinci-002,1', 'gpt-35-turbo,0613', 'gpt-4-32k,0314', 'gpt-35-turbo,0301', 'o1-mini,2024-09-12', 'tts-hd,001', 'text-davinci-002,1', 'text-davinci-003,1']}


Adding model 'mlc,1' to policy assignment 'AiModelControlBuiltIn'.
{'additional_properties': {}, 'value': ['babbage-002,2', 'text-embedding-ada-002,2', 'davinci-002,3', 'text-embedding-3-large,1', 'text-embedding-3-small,1', 'tts,1', 'tts-hd,1', 'whisper-001,2', 'whisper,1', 'dall-e-2,2.0', 'dall-e-3,3.0', 'gpt-35-turbo-16k,0613', 'gpt-35-turbo-instruct,0914', 'gpt-4-32k,0613', 'gpt-4o,2024-08-06', 'gpt-4o-mini,2024-07-18', 'o1-mini,1', 'gpt-35-turbo,0125', 'gpt-4,turbo-2024-04-09', 'gpt-35-turbo,1106', 'gpt-35-turbo,0301', 'gpt-4o,2024-05-13', 'tts-hd,001', 'whisper,001', 'gpt-4,0613', 'gpt-4,0314', 'davinci-002,1', 'gpt-35-turbo,0613', 'gpt-4-32k,0314', 'gpt-35-turbo,0301', 'o1-mini,2024-09-12', 'tts-hd,001', 'text-davinci-002,1', 'text-davinci-003,1']}

The model, 'mlc,1' is NOT successfully added to the parameter, which is an Array, in either of the policies I'm trying to update. No error is given. No logging is available.

The lack of documentation has made this a bit more difficult than I had expected.

Page URL

https://learn.microsoft.com/en-us/python/api/azure-mgmt-resource/azure.mgmt.resource.policy.v2023_04_01.operations.policyassignmentsoperations?view=azure-python

Content source URL

https://github.com/MicrosoftDocs/azure-docs-sdk-python/blob/main/docs-ref-autogen/azure-mgmt-resource/azure.mgmt.resource.policy.v2023_04_01.operations.PolicyAssignmentsOperations.yml

Document Version Independent Id

a80f2b64-ba8e-f571-9d2c-e7d0fcb9ac40

Article author

@lmazuel

Metadata

• ID: cfa07a61-9074-a160-4501-990b9ec7a5e8
• Service: azure-resource-manager

[msyyc]

Thanks for your feedback. @ChenxiJiang333 Please help to clarify @mlcooper's question about how to set parameters of policy_assignments.update_by_id

[ChenxiJiang333]

got it

[ChenxiJiang333]

Hi @mlcooper, operation update_by_id does not support edit parameters. You can directly use create_by_id to edit parameters for an existing assignment.

[mlcooper]

I believe that the fact I should have been using create_by_id should have been documented. It's not documented at all that update_by_id, which is the more intuitive name does not support updating parameters.

I found that in the parameters of create_by_id, you need to specify policyDefinitionId, which is not documented.

New Error:

new models to add: ['gpt-4o,2024-11-20', 'o1,2024-12-17']

Traceback (most recent call last):
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 1469, in _deserialize
    found_value = key_extractor(attr, attr_desc, data)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 1238, in rest_key_case_insensitive_extractor
    return attribute_key_case_insensitive_extractor(key, None, working_data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 1270, in attribute_key_case_insensitive_extractor
    return data.get(found_key)
           ^^^^^^^^
AttributeError: 'str' object has no attribute 'get'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 1644, in deserialize_data
    return self.deserialize_type[iter_type](data, data_type[1:-1])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 1688, in deserialize_dict
    return {k: self.deserialize_data(v, dict_type) for k, v in attr.items()}
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 1657, in deserialize_data
    return self._deserialize(obj_type, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 1484, in _deserialize
    raise DeserializationError(msg) from err
azure.core.exceptions.DeserializationError: Unable to deserialize to object: type

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 708, in body
    data = deserializer._deserialize(data_type, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 1480, in _deserialize
    value = self.deserialize_data(raw_value, attr_desc["type"])
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 1655, in deserialize_data
    raise DeserializationError(msg) from err
azure.core.exceptions.DeserializationError: Unable to deserialize response data. Data: {'allowedAssetIds': {'value': ['gpt-4o,2024-11-20', 'o1,2024-12-17']}, 'effect': 'Deny'}, {ParameterValuesValue}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/coopm017/sourcecode/azure/ai-models/ingestion.py", line 296, in <module>
    main()
  File "/home/coopm017/sourcecode/azure/ai-models/ingestion.py", line 278, in main
    update_response = policy_client.policy_assignments.create_by_id(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/core/tracing/decorator.py", line 105, in wrapper_use_tracer
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/v2023_04_01/operations/_operations.py", line 6861, in create_by_id
    _json = self._serialize.body(parameters, "PolicyAssignment")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/_serialization.py", line 710, in body
    raise SerializationError("Unable to build a model: " + str(err)) from err
azure.core.exceptions.SerializationError: Unable to build a model: Unable to deserialize response data. Data: {'allowedAssetIds': {'value': ['gpt-4o,2024-11-20', 'o1,2024-12-17']}, 'effect': 'Deny'}, {ParameterValuesValue}

Code used:

        policy_client = PolicyClient(
        api_version="2023-04-01",
        credential=credential,
        subscription_id=os.getenv("SUBSCRIPTION_ID"),
    )

    policy_assignment_ids = [
        {
            f"providers/Microsoft.Management/managementGroups/{os.getenv("TENANT_ID")}/providers/Microsoft.Authorization/policyAssignments/AiModelControlBuiltInCst": "allowedModels"
        },
        {
            f"providers/Microsoft.Management/managementGroups/{os.getenv("TENANT_ID")}/providers/Microsoft.Authorization/policyAssignments/AiModelControlBuiltIn": "allowedAssetIds"
        },
    ]

    for dict_item in policy_assignment_ids:
        for assignment, v in dict_item.items():
            k = policy_client.policy_assignments.get_by_id(
                policy_assignment_id=assignment
            )
            policy_definition_id = k.policy_definition_id
            m = k.parameters[v]
            current_models = m.value
            new_models = np.setdiff1d(
                new_model_list, current_models, assume_unique=True
            ).tolist()
            if len(new_models) > 0:
                print(f"new models to add: {new_models}")
                update_response = policy_client.policy_assignments.create_by_id(
                    policy_assignment_id=assignment,
                    parameters={
                        "parameters": {v: {"value": new_models}, "effect": "Deny"},
                        "policyDefinitionId": policy_definition_id,
                    },
                )
                r = update_response.parameters
                resp = r[v]
                print(resp)
                print("\n")

I have tried many things to fix this error, but none of it has worked. Can you please tell me what the error is complaining about?

[ChenxiJiang333]

Hi @mlcooper, try update code {v: {"value": new_models}, "effect": "Deny"} to

{v: {"value": new_models, "effect": {"value": "Deny"}}}

[mlcooper]

Thank you, this helped me to fix it, and I can now update the Array I have in one of my parameters. However, when I update it, it overwrites the array instead of doing a PATCH to add items to it.

Is there a PATCH operation I can use to add items to the array instead of overwriting it?

[ChenxiJiang333]

Thank you, this helped me to fix it, and I can now update the Array I have in one of my parameters. However, when I update it, it overwrites the array instead of doing a PATCH to add items to it.

Is there a PATCH operation I can use to add items to the array instead of overwriting it?

I'm afraid you cannot do it through SDK. If you don't want to overwrite the old policy assignment, I guess to create a new one with newly added items would share the same effect.

And I'm sorry for not pointing out a mistake last time that policy effects are used in policy definitions & overrides creation (see: https://learn.microsoft.com/en-us/azure/governance/policy/concepts/effect-basics), they would not take any efforts in this code, so {v: {"value": new_models}, "effect": "Deny"} would be better to be updated to {v: {"value": new_models}}.

[mlcooper]

If I remove "effect": {"value": "Deny"},, as you suggest, I get this error:

Traceback (most recent call last):
  File "/home/coopm017/sourcecode/azure/ai-models/ingestion.py", line 249, in <module>
    main()
  File "/home/coopm017/sourcecode/azure/ai-models/ingestion.py", line 233, in main
    policy_client.policy_assignments.create_by_id(
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/core/tracing/decorator.py", line 105, in wrapper_use_tracer
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/v2023_04_01/operations/_operations.py", line 6883, in create_by_id
    raise HttpResponseError(response=response, error_format=ARMErrorFormat)
azure.core.exceptions.HttpResponseError: (MissingPolicyParameter) The policy assignment 'AiModelControlBuiltInCst' is missing the parameter(s) 'effect' as defined in the policy definition '<Redacted> Cognitive Services AI Model Control version 1.0.0'.
Code: MissingPolicyParameter

[ChenxiJiang333]

If I remove "effect": {"value": "Deny"},, as you suggest, I get this error:

Traceback (most recent call last):
  File "/home/coopm017/sourcecode/azure/ai-models/ingestion.py", line 249, in <module>
    main()
  File "/home/coopm017/sourcecode/azure/ai-models/ingestion.py", line 233, in main
    policy_client.policy_assignments.create_by_id(
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/core/tracing/decorator.py", line 105, in wrapper_use_tracer
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/coopm017/sourcecode/azure/ai-models/.venv/lib/python3.12/site-packages/azure/mgmt/resource/policy/v2023_04_01/operations/_operations.py", line 6883, in create_by_id
    raise HttpResponseError(response=response, error_format=ARMErrorFormat)
azure.core.exceptions.HttpResponseError: (MissingPolicyParameter) The policy assignment 'AiModelControlBuiltInCst' is missing the parameter(s) 'effect' as defined in the policy definition '<Redacted> Cognitive Services AI Model Control version 1.0.0'.
Code: MissingPolicyParameter

My bad, seems the effect is a parameter defined in your policy definition, not the policy effect.

[mlcooper]

Thanks for your help with this issue. My script is working now. I do feel that documentation for the API methods discussed above could be improved.

1. I was using the wrong method, update_by_id, because it made the most sense from its name, but it does not support edit parameters. I should have been using create_by_id.

2. The lack of support for PATCH updating policy parameters is also not documented.

3. I found that in the parameters of create_by_id, you need to specify policyDefinitionId, which is not documented.

[ChenxiJiang333]

Hi @mlcooper, sdk is auto generated by swagger definition, so we can not make any change on it. I have transferred your advice to the service team, please follow: Azure/azure-rest-api-specs#31956. Once swagger is fixed, python SDK will release new version to support it.