Error in vCenter Scraper: Need to Specify Datacenter for Metric Collection

[sromlee]

First of all, I'm a newbie and this is my very first time using Otel. The testing version is v0.102.0. The collector can produce some metrics for the datastore and host but no VM.
Below is my error:

unable to retrieve ResourcePools with InventoryLists please specify a datacenter;

Also the error about querySpec. Please see below is the full errors.

2024-06-19T12:19:50.542Z error scraperhelper/scrapercontroller.go:197 Error scraping metrics {"kind": "receiver", "name": "vcenter", "data_type": "metrics", "error": "unable to retrieve ResourcePools with InventoryLists: please specify a datacenter; unable to retrieve vApps with InventoryLists: please specify a datacenter; failed to retrieve perf metrics for HostSystems: ServerFaultCode: \nRequired parameter querySpec is missing\n\nwhile parsing call information for method QueryPerf\nat line 2, column 66\n\nwhile parsing SOAP body\nat line 2, column 60\n\nwhile parsing SOAP envelope\nat line 2, column 0\n\nwhile parsing HTTP request for method queryStats\non object of type vim.PerformanceManager\nat line 1, column 0; failed to retrieve perf metrics for VirtualMachines: ServerFaultCode: \nRequired parameter querySpec is missing\n\nwhile parsing call information for method QueryPerf\nat line 2, column 66\n\nwhile parsing SOAP body\nat line 2, column 60\n\nwhile parsing SOAP envelope\nat line 2, column 0\n\nwhile parsing HTTP request for method queryStats\non object of type vim.PerformanceManager\nat line 1, column 0; no inventory path found for collected ResourcePool: Resources; no inventory path found for collected ResourcePool: Resources; no inventory path found for collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected

I'm not sure how to specify the data center or something went wrong with my configuration.

Updated:
My VMware environment has no Resource pool/vApp/Cluster. How to avoid this kind of errors?

[sromlee]

Pinging code owners for receiver/vcenter: @djaglowski @schmikei @StefanKurek.

[StefanKurek]

@sromlee Hello. The user shouldn't have to manually specify a datacenter for collection and this should be picked up automatically. Are you connecting to vCenter or connecting directly to an ESXi host?

[sromlee]

@StefanKurek Thank you for your reply. I’m also confused. Seems vcenter.vm.* are not sent and it could not process the metrics due to no inventory path found.

[StefanKurek]

@sromlee did you say you had 1 VM metric that did return? If that's the case, can you post the resource attributes for that VM?

[sromlee]

@StefanKurek I only get these resource attributes from metric : vcenter.vm.disk.usage I could not found any vm.name in resource_attributes as well.

[StefanKurek]

@sromlee Can you tell me what version of vCenter you are using? I'm also curious if you only have a single datacenter in your environment (I don't think I already asked this)?

[sromlee]

@StefanKurek I have multiple data centers with various VMs in each data center.
Below is the detail of my Center.

Version: | 8.0.2
-- | --
Build: | 23929136
Clusters: | 0
Hosts: | 5
Virtual Machines: | 22

Version:	8.0.2
Build:	23929136
Clusters:	[0](javascript:void(0))
Hosts:	[5](javascript:void(0))
Virtual Machines:	[22](javascript:void(0))

[sromlee]

This is the full error. I'm not sure it's because of my environment is a standalone (not a cluster) ?
@StefanKurek

 error   scraperhelper/scrapercontroller.go:197  Error scraping metrics  {"kind": "receiver", "name": "vcenter", "data_type": "metrics", "error": "unable to retrieve ResourcePools with InventoryLists: please specify a datacenter; unable to retrieve vApps with InventoryLists: please specify a datacenter; failed to retrieve perf metrics for HostSystems: ServerFaultCode: \nRequired parameter querySpec is missing\n\nwhile parsing call information for method QueryPerf\nat line 2, column 66\n\nwhile parsing SOAP body\nat line 2, column 60\n\nwhile parsing SOAP envelope\nat line 2, column 0\n\nwhile parsing HTTP request for method queryStats\non object of type vim.PerformanceManager\nat line 1, column 0; failed to retrieve perf metrics for VirtualMachines: ServerFaultCode: \nRequired parameter querySpec is missing\n\nwhile parsing call information for method QueryPerf\nat line 2, column 66\n\nwhile parsing SOAP body\nat line 2, column 60\n\nwhile parsing SOAP envelope\nat line 2, column 0\n\nwhile parsing HTTP request for method queryStats\non object of type vim.PerformanceManager\nat line 1, column 0; no inventory path found for collected ResourcePool: Resources; no inventory path found for collected ResourcePool: Resources; no inventory path found for collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for collected ResourcePool: Resources; no inventory path found for collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxxxxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for collected ResourcePool: Resources; no inventory path found for VM [test]'s collected ResourcePool: Resources; no inventory path found for VM [test-vm]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [centos_template]'s collected ResourcePool: Resources; no inventory path found for VM [sg-vrep-6002]'s collected ResourcePool: Resources; no inventory path found for VM [centos_ide]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources; no inventory path found for VM [sg-vrep-6001]'s collected ResourcePool: Resources; no inventory path found for VM [win2019]'s collected ResourcePool: Resources; no inventory path found for VM [xxxx]'s collected ResourcePool: Resources", "scraper": "vcenter"}
go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport
        /go/pkg/mod/go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:197
go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1
        /go/pkg/mod/go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:173

[StefanKurek]

@sromlee we have tested with standalone hosts, so that shouldn't be an issue.

[sromlee]

@djaglowski @schmikei @StefanKurek May I know if the scraper will work in case there is no resource pool, vapp and cluster ?
I noticed that the metric just sent only vcenter.host and vcenter.datastore. But for vcenter.vm just sent only 1 metric which is vcenter.vm.disk.usage.

[sromlee]

@StefanKurek The latest version is working! There was still an error for HostSystems. Btw, I can retrieve most of the metrics including all VMs metrics now. Thank you very much for your help :)

Just FYI, here is the error:
"error": "failed to retrieve perf metrics for HostSystems: ServerFaultCode: \nRequired parameter querySpec is missing\n\nwhile parsing call information for method QueryPerf\nat line 2, column 66\n\nwhile parsing SOAP body\nat line 2, column 60\n\nwhile parsing SOAP envelope\nat line 2, column 0\n\nwhile parsing HTTP request for method queryStats\non object of type vim.PerformanceManager\nat line 1, column 0; failed to retrieve perf metrics for VirtualMachines: ServerFaultCode: \nRequired parameter querySpec is missing\n\nwhile parsing call information for method QueryPerf\nat line 2, column 66\n\nwhile parsing SOAP body\nat line 2, column 60\n\nwhile parsing SOAP envelope\nat line 2, column 0\n\nwhile parsing HTTP request for method queryStats\non object of type vim.PerformanceManager\nat line 1, column 0", "scraper": "vcenter"}

[StefanKurek]

Is your data collection set to at least level 2? https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-monitoring-performance/GUID-25800DE4-68E5-41CC-82D9-8811E27924BC.html

[sromlee]

Yes, we set to level 2 already. Still facing the same error.

2024-06-26T12:22:43.951547897+07:00 2024-06-26T12:22:43.951+0700 error scraperhelper/scrapercontroller.go:197 Error scraping metrics {"kind": "receiver", "name": "vcenter", "data_type": "metrics", "error": "failed to retrieve perf metrics for HostSystems: ServerFaultCode: \nRequired parameter querySpec is missing\n\nwhile parsing call information for method QueryPerf\nat line 2, column 66\n\nwhile parsing SOAP body\nat line 2, column 60\n\nwhile parsing SOAP envelope\nat line 2, column 0\n\nwhile parsing HTTP request for method queryStats\non object of type vim.PerformanceManager\nat line 1, column 0; failed to retrieve perf metrics for VirtualMachines: ServerFaultCode: \nRequired parameter querySpec is missing\n\nwhile parsing call information for method QueryPerf\nat line 2, column 66\n\nwhile parsing SOAP body\nat line 2, column 60\n\nwhile parsing SOAP envelope\nat line 2, column 0\n\nwhile parsing HTTP request for method queryStats\non object of type vim.PerformanceManager\nat line 1, column 0", "scraper": "vcenter"}

2024-06-26T12:22:43.951627236+07:00 go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport

2024-06-26T12:22:43.951639027+07:00 go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:197

2024-06-26T12:22:43.951646090+07:00 go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1

2024-06-26T12:22:43.951652943+07:00 go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:173

[StefanKurek]

@sromlee I don't think you should have to do this, but would you temporarily set it to level 4 to see if this makes a difference? I'm reaching at straws a bit here.

[sromlee]

@StefanKurek I update to level 4 already but still the same error

2024-07-01T09:10:08.634711672+07:00 2024-07-01T09:10:08.634+0700 error scraperhelper/scrapercontroller.go:197 Error scraping metrics {"kind": "receiver", "name": "vcenter", "data_type": "metrics", "error": "failed to retrieve perf metrics for HostSystems: ServerFaultCode: \nRequired parameter querySpec is missing\n\nwhile parsing call information for method QueryPerf\nat line 2, column 66\n\nwhile parsing SOAP body\nat line 2, column 60\n\nwhile parsing SOAP envelope\nat line 2, column 0\n\nwhile parsing HTTP request for method queryStats\non object of type vim.PerformanceManager\nat line 1, column 0; failed to retrieve perf metrics for VirtualMachines: ServerFaultCode: \nRequired parameter querySpec is missing\n\nwhile parsing call information for method QueryPerf\nat line 2, column 66\n\nwhile parsing SOAP body\nat line 2, column 60\n\nwhile parsing SOAP envelope\nat line 2, column 0\n\nwhile parsing HTTP request for method queryStats\non object of type vim.PerformanceManager\nat line 1, column 0", "scraper": "vcenter"}