Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query works but Search-AzGraph in Az.ResourceGraph v1.0.0 does not manage to parse the output #26470

Open
o-l-a-v opened this issue Oct 24, 2024 · 5 comments
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Resource Graph Service Attention This issue is responsible by Azure service team.

Comments

@o-l-a-v
Copy link

o-l-a-v commented Oct 24, 2024

Description

Following query works in Azure Resource Graph Explorer in the Azure portal ( https://portal.azure.com/#view/HubsExtension/ArgQueryBlade ), and if running Search-AzGraph with -Debug I can see that the API returns data.

resources
| where type in~ ("Microsoft.Automation/automationAccounts","Microsoft.DataFactory/factories","Microsoft.DocumentDB/databaseAccounts","Microsoft.Logic/workflows","Microsoft.KeyVault/vaults","Microsoft.Network/bastionHosts","Microsoft.Network/networkSecurityGroups","Microsoft.Network/publicIPAddresses","Microsoft.Network/virtualNetworkGateways","Microsoft.ServiceBus/namespaces")
// Add tags from resource group
| join kind = leftouter (
  resourcecontainers
  | where type == "microsoft.resources/subscriptions/resourcegroups"
  | project rgName = name, rgSubscriptionId = subscriptionId, rgTags = tags
) on $left.resourceGroup == $right.rgName and $left.subscriptionId == $right.rgSubscriptionId
| project-away rgName, rgSubscriptionId
// Add tags from subscription
| join kind = leftouter (
  resourcecontainers
  | where type == "microsoft.resources/subscriptions"
  | project subName = name, subId = subscriptionId, subTags = tags
) on $left.subscriptionId == $right.subId
| project-away subId

But Search-AzGraph fails with error:

Search-AzGraph: Cannot process argument because the value of argument "name" is not valid. Change the value of the "name" argument and run the operation again.

Issue script & Debug output

PS > Search-AzGraph -UseTenantScope -Query 'resources
| where type in~ ("Microsoft.Automation/automationAccounts","Microsoft.DataFactory/factories","Microsoft.DocumentDB/databaseAccounts","Microsoft.Logic/workflows","Microsoft.KeyVault/vaults","Microsoft.Network/bastionHosts","Microsoft.Network/networkSecurityGroups","Microsoft.Network/publicIPAddresses","Microsoft.Network/virtualNetworkGateways","Microsoft.ServiceBus/namespaces")
// Add tags from resource group
| join kind = leftouter (
  resourcecontainers
  | where type == "microsoft.resources/subscriptions/resourcegroups"
  | project rgName = name, rgSubscriptionId = subscriptionId, rgTags = tags
) on $left.resourceGroup == $right.rgName and $left.subscriptionId == $right.rgSubscriptionId
| project-away rgName, rgSubscriptionId
// Add tags from subscription
| join kind = leftouter (
  resourcecontainers
  | where type == "microsoft.resources/subscriptions"
  | project subName = name, subId = subscriptionId, subTags = tags
) on $left.subscriptionId == $right.subId
| project-away subId'

Search-AzGraph: Cannot process argument because the value of argument "name" is not valid. Change the value of the "name" argument and run the operation again.

PS >

Environment data

  • Windows 11 24H2 x64
  • PowerShell v7.4.6

Module versions

  • Az.Accounts v3.0.4
  • Az.ResourceGraph v1.0.0

Error output

PS > Resolve-AzError -Last

Message        : Cannot process argument because the value of argument "name" is not valid. Change the value of the "name" argument and run the operation again.
StackTrace     :    at Microsoft.Azure.Commands.ResourceGraph.Utilities.ResultExtensions.ToPsObjects(Object data)
                    at Microsoft.Azure.Commands.ResourceGraph.Cmdlets.SearchAzureRmGraph.ExecuteCmdlet()
Exception      : System.ArgumentOutOfRangeException
InvocationInfo : {Search-AzGraph}
Line           : Search-AzGraph @Splat -Debug
Position       : At line:1 char:1
                 + Search-AzGraph @Splat -Debug
                 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 53

Message        : Cannot process argument because the value of argument "name" is not valid. Change the value of the "name" argument and run the operation again.
StackTrace     :    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ToPsObject(JToken jtoken)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ConvertPropertyValueForPsObject(JToken propertyValue)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ToPsObject(JToken jtoken)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ConvertPropertyValueForPsObject(JToken propertyValue)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ToPsObject(JToken jtoken)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ConvertPropertyValueForPsObject(JToken propertyValue)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ToPsObject(JToken jtoken)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ConvertPropertyValueForPsObject(JToken propertyValue)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ToPsObject(JToken jtoken)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ConvertPropertyValueForPsObject(JToken propertyValue)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ToPsObject(JToken jtoken)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ConvertPropertyValueForPsObject(JToken propertyValue)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ToPsObject(JToken jtoken)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ConvertPropertyValueForPsObject(JToken propertyValue)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ToPsObject(JToken jtoken)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ConvertPropertyValueForPsObject(JToken propertyValue)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ToPsObject(JToken jtoken)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ConvertPropertyValueForPsObject(JToken propertyValue)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.JTokenExtensions.ToPsObject(JToken jtoken)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.ResultExtensions.ToPsObjects(IList`1 rows)
                    at Microsoft.Azure.Commands.ResourceGraph.Utilities.ResultExtensions.ToPsObjects(Object data)
Exception      : System.Management.Automation.PSArgumentException
InvocationInfo : {Search-AzGraph}
Line           : Search-AzGraph @Splat -Debug
Position       : At line:1 char:1
                 + Search-AzGraph @Splat -Debug
                 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 53

PS >
@o-l-a-v o-l-a-v added bug This issue requires a change to an existing behavior in the product in order to be resolved. needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Oct 24, 2024
@microsoft-github-policy-service microsoft-github-policy-service bot added customer-reported needs-triage This is a new issue that needs to be triaged to the appropriate team. and removed needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Oct 24, 2024
@o-l-a-v
Copy link
Author

o-l-a-v commented Oct 24, 2024

Might be relevant: I started researching changing out Search-AzGraph with Invoke-AzRestMethod and found that PowerShell could not convert $Response.content from JSON to PSCustomObject with error:

PS > ConvertFrom-Json -InputObject $Response.'content'

ConvertFrom-Json: The provided JSON includes a property whose name is an empty string, this is only supported using the -AsHashTable switch.

PS >

So the JSON response from the Azure RM API clearly contains a "property whose name is an empty string".


Final workaround (it's not pretty, but it works)
# Assets
$Resources = [System.Collections.Generic.List[System.Management.Automation.OrderedHashtable]]::new()
Remove-Variable -Name 'Response' -Force -ErrorAction 'Ignore'
$Query = [string] 'resources
| where type in~ ("Microsoft.Automation/automationAccounts","Microsoft.DataFactory/factories","Microsoft.DocumentDB/databaseAccounts","Microsoft.Logic/workflows","Microsoft.KeyVault/vaults","Microsoft.Network/bastionHosts","Microsoft.Network/networkSecurityGroups","Microsoft.Network/publicIPAddresses","Microsoft.Network/virtualNetworkGateways","Microsoft.ServiceBus/namespaces")
// Add tags from resource group
| join kind = leftouter (
resourcecontainers
| where type == "microsoft.resources/subscriptions/resourcegroups"
| project rgName = name, rgSubscriptionId = subscriptionId, rgTags = tags
) on $left.resourceGroup == $right.rgName and $left.subscriptionId == $right.rgSubscriptionId
| project-away rgName, rgSubscriptionId
// Add tags from subscription
| join kind = leftouter (
resourcecontainers
| where type == "microsoft.resources/subscriptions"
| project subName = name, subId = subscriptionId, subTags = tags
) on $left.subscriptionId == $right.subId
| project-away subId'

# Run resource graph query and handle paging
do {
    $Body = [ordered]@{
        'query'   = [string] $ResourceGraphQuery
        'options' = [ordered]@{
            '$top'               = [byte] 100
            'resultFormat'       = [string] 'objectArray'
            'allowPartialScopes' = [bool] $false
        }
    }
    if (-not [string]::IsNullOrEmpty($Content.'$skipToken')) {
        $Body.'options'.Add('$skipToken', $Content.'$skipToken')
    }
    $Response = Invoke-AzRestMethod -Method 'Post' -Path '/providers/Microsoft.ResourceGraph/resources?api-version=2022-10-01' -Payload (
        ConvertTo-Json -Depth 2 -Compress -InputObject $Body
    )
    if ($Response.'StatusCode' -ne 200) {
        Throw ('That failed with HTTP status code {0}.' -f $Response.'StatusCode')
    }
    $Content = ConvertFrom-Json -InputObject $Response.'content' -AsHashtable
    $Content.'data'.ForEach{$Resources.Add($_)}
    Write-Output -InputObject ('Currently at {0}' -f $Resources.'Count'.ToString())
} until ([string]::IsNullOrEmpty($Content.'$skipToken'))

# Output total of resources
Write-Output -InputObject ('Found a total of {0} resources.' -f $Resources.'Count'.ToString())

Thanks to EnterprisePolicyAsCode for inspiration. 😊

@o-l-a-v o-l-a-v changed the title Query works but Search-AzGraph in Az.Accounts v3.0.4 does not manage to parse the output Query works but Search-AzGraph in Az.ResourceGraph v1.0.0 does not manage to parse the output Oct 24, 2024
@o-l-a-v
Copy link
Author

o-l-a-v commented Oct 25, 2024

Another workaround is to exclude properties you're not interested in, hopefully you'll only end up with parsable output.

Example line:

| project id, name, type, location, resourceGroup, subscriptionId, tags

Full example query:

resources
| where type in~ ("Microsoft.Automation/automationAccounts","Microsoft.DataFactory/factories","Microsoft.DocumentDB/databaseAccounts","Microsoft.Logic/workflows","Microsoft.KeyVault/vaults","Microsoft.Network/bastionHosts","Microsoft.Network/networkSecurityGroups","Microsoft.Network/publicIPAddresses","Microsoft.Network/virtualNetworkGateways","Microsoft.ServiceBus/namespaces")
| project id, name, type, location, resourceGroup, subscriptionId, tags
// Add tags from resource group
| join kind = leftouter (
  resourcecontainers
  | where type == "microsoft.resources/subscriptions/resourcegroups"
  | project rgName = name, rgSubscriptionId = subscriptionId, rgTags = tags
) on $left.resourceGroup == $right.rgName and $left.subscriptionId == $right.rgSubscriptionId
| project-away rgName, rgSubscriptionId
// Add tags from subscription
| join kind = leftouter (
  resourcecontainers
  | where type == "microsoft.resources/subscriptions"
  | project subName = name, subId = subscriptionId, subTags = tags
) on $left.subscriptionId == $right.subId
| project-away subId

@isra-fel
Copy link
Member

Let me loop in ResourceGraph team to understand if the JSON response from the Azure RM API clearly contains a "property whose name is an empty string" is expected behavior.

@isra-fel isra-fel added Service Attention This issue is responsible by Azure service team. Resource Graph and removed needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Oct 31, 2024
Copy link
Contributor

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @venu-l.

1 similar comment
Copy link
Contributor

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @venu-l.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Resource Graph Service Attention This issue is responsible by Azure service team.
Projects
None yet
Development

No branches or pull requests

2 participants