Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

code 400 when collector=Summary #18

Open
yehaotong opened this issue Nov 1, 2021 · 38 comments
Open

code 400 when collector=Summary #18

yehaotong opened this issue Nov 1, 2021 · 38 comments

Comments

@yehaotong
Copy link

When I test pagerduty exporter, there is always this error in the running log:
HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) collector=Summary
ERRO[0195] &{0xc00021a000 map[collector:Summary] 2021-11-01 14:05:32.236131 +0800 CST m=+195.528282675 panic HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) } collector=Summary
Do I need to add some parameters?

@mblaschke
Copy link
Member

mblaschke commented Nov 1, 2021

can you provide the whole log? is there a stack trace?

can you also please deliver all settings/environment variables (except the token)?

@yehaotong
Copy link
Author

can you provide the whole log? is there a stack trace?

can you also please deliver all settings/environment variables (except the token)?

it is my running log :
tarting pagerduty-exporter v (; go1.16.5; by webdevops.io)
INFO[0000] {"Logger":{"Debug":false,"Verbose":false,"LogJson":false},"PagerDuty":{"MaxConnections":4,"Schedule":{"OverrideTimeframe":172800000000000,"EntryTimeframe":259200000000000,"EntryTimeFormat":"Mon, 02 Jan 15:04 MST"},"Incident":{"TimeFormat":"Mon, 02 Jan 15:04 MST"},"Teams":{"Disable":false,"Filter":null},"Summary":{"Since":2628000000000000}},"ServerBind":":8080","ScrapeTime":{"General":300000000000,"Summary":900000000000,"Live":60000000000}}

@yehaotong
Copy link
Author

can you provide the whole log? is there a stack trace?

can you also please deliver all settings/environment variables (except the token)?

This log will appear after running for more than ten seconds:
NFO[0180] starting metrics collection collector=Incident
INFO[0180] starting metrics collection collector=OnCall
PANI[0182] HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) collector=Summary
ERRO[0182] &{0xc0001825b0 map[collector:Summary] 2021-11-02 09:30:12.007774 +0800 CST m=+182.873421728 panic HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) } collector=Summary
INFO[0183] finished metrics collection (duration: 3.211384443s) collector=OnCall duration=3.211384443
INFO[0188] finished metrics collection (duration: 8.952185903s) collector=Incident duration=8.952185903

@yehaotong
Copy link
Author

can you provide the whole log? is there a stack trace?

can you also please deliver all settings/environment variables (except the token)?

I only use the arguments is the authtoken,my start command is : go run .go --pagerduty.authtoken=*****

@mblaschke
Copy link
Member

mblaschke commented Nov 2, 2021

You are building the exporter on your own, right? You dont use the docker images?

can you also run it with --debug?

@yehaotong
Copy link
Author

Yes, I run locally,and my go version is 1.16

@yehaotong
Copy link
Author

and I want to select all incident(include:triggered、acknowledged、resolved,currently only triggered and acknowledged),so I add "resolved" in listOpts.Statuses = []string{"triggered", "acknowledged"}, and an error occurred while running:
PANI[0316] HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) collector=Incident
ERRO[0316] &{0xc0001f91f0 map[collector:Incident] 2021-11-03 13:49:43.614487 +0800 CST m=+316.890111869 panic HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) } collector=Incident
Can't this query include resolved incident?

@mblaschke
Copy link
Member

mblaschke commented Nov 3, 2021

i've added the possibility to define the incident status filter with env var PAGERDUTY_INCIDENT_STATUS or --pagerduty.incident.status

is that solving your issue?
for all statuses use: --pagerduty.incident.status=acknowledged --pagerduty.incident.status=triggered --pagerduty.incident.status=resolved

or use --pagerduty.incident.status=all

@yehaotong
Copy link
Author

i've added the possibility to define the incident status filter with env var PAGERDUTY_INCIDENT_STATUS or --pagerduty.incident.status

is that solving your issue? for all statuses use: --pagerduty.incident.status=acknowledged --pagerduty.incident.status=triggered --pagerduty.incident.status=resolved

or use --pagerduty.incident.status=all

I update the code to the latest, and then use the parameter you changed -- pagerduty. Increment. Status = all. Then I run it locally, and the result still appears

HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) collector=Incident

Just like the error when I put listopts. Statuses = {"triggered", "acknowledged", "resolved"}, can you successfully request incident in all statuses when running locally? I checked the official API of pagerduty. The parameter statuses should support three status queries,it's strange

@mblaschke
Copy link
Member

please check if you made any other changes in the code via git status and then post the full log when running the exporter with --debug

@yehaotong
Copy link
Author

I only change is the go version 1.17 to 1.16

@yehaotong
Copy link
Author

and my pageduty do not use team ,so I comment out the indicators related to team in main.go.Otherwise, it will report an error if he can't find the part about the team

@mblaschke
Copy link
Member

you can disable team support with --pagerduty.disable-teams
please first try it without any code changes and resync vendor folder please.

or in best case try the docker container image.

@yehaotong
Copy link
Author

yehaotong commented Nov 4, 2021 via email

@yehaotong
Copy link
Author

yehaotong commented Nov 4, 2021 via email

@yehaotong
Copy link
Author

the docker stop with this log:
goroutine 337 [running]:
github.com/sirupsen/logrus.(*Entry).log(0xc0004094d0, 0x0, {0xc000028900, 0x2a})
/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:259 +0x26f
github.com/sirupsen/logrus.(*Entry).Log(0xc00021b730, 0x0, {0xc0004094d0, 0x8000101, 0x8eb880})
/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:293 +0x4f
github.com/sirupsen/logrus.(*Entry).Panic(...)
/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:331
main.(*CollectorGeneral).Collect.func1()
/go/src/github.com/webdevops/pagerduty-exporter/collector_general.go:39 +0xa9
panic({0x8f7220, 0xc000afe5b0})
/usr/local/go/src/runtime/panic.go:1038 +0x215
github.com/sirupsen/logrus.(*Entry).log(0xc000409918, 0x0, {0xc000273560, 0x57})
/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:259 +0x26f
github.com/sirupsen/logrus.(*Entry).Log(0xc00021b730, 0x0, {0xc000409918, 0x0, 0x0})
/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:293 +0x4f
github.com/sirupsen/logrus.(*Entry).Panic(...)
/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:331
main.(*MetricsCollectorIncident).Collect(0xc00022da00, {0xc00002b320, 0xc00046a480}, 0xc0002720c0)
/go/src/github.com/webdevops/pagerduty-exporter/metrics_incident.go:87 +0x318
main.(*CollectorGeneral).Collect(0xc000200a20)
/go/src/github.com/webdevops/pagerduty-exporter/collector_general.go:71 +0x153
main.(*CollectorGeneral).Run.func1.1()
/go/src/github.com/webdevops/pagerduty-exporter/collector_general.go:25 +0x1d
created by main.(*CollectorGeneral).Run.func1
/go/src/github.com/webdevops/pagerduty-exporter/collector_general.go:24 +0x29

@yehaotong
Copy link
Author

yehaotong commented Nov 4, 2021 via email

@yehaotong
Copy link
Author

Can you see any conclusions about the status from the log? I hope to find out that the status contains the resolved increment. Thank you

@yehaotong
Copy link
Author

Can you find out the resolved incident on your side? This is very important to me, thank you

@mblaschke
Copy link
Member

Github build from the previous commit was taking longer so it pushed the previous version, the newest docker tag is fine again

please retry but first do a docker pull webdevops/pagerduty-exporter:main and ensure that the version of the exporter matches:

{"file":"main.go:44","func":"main","level":"info","msg":"starting pagerduty-exporter v8834c32 (8834c32; go1.17.2; by webdevops.io)"}

@yehaotong
Copy link
Author

I'm sure I've used the latest docker image and run commands as you said. When there's a problem getting the summary and resolved incident

    docker run --rm -ti -p 8080:8080 webdevops/pagerduty-exporter:main --pagerduty.authtoken=xxxxxxx --pagerduty.incident.status=all --pagerduty.disable-teams
    {"file":"main.go:44","func":"main","level":"info","msg":"starting pagerduty-exporter v8834c32 (8834c32; go1.17.2; by webdevops.io)"}
    {"file":"main.go:45","func":"main","level":"info","msg":"{\"Logger\":{\"Debug\":false,\"Verbose\":false,\"LogJson\":true},\"PagerDuty\":{\"MaxConnections\":4,\"Schedule\":{\"OverrideTimeframe\":172800000000000,\"EntryTimeframe\":259200000000000,\"EntryTimeFormat\":\"Mon, 02 Jan 15:04 MST\"},\"Incident\":{\"Statuses\":[\"triggered\",\"acknowledged\",\"resolved\"],\"TimeFormat\":\"Mon, 02 Jan 15:04 MST\"},\"Teams\":{\"Disable\":true,\"Filter\":null},\"Summary\":{\"Since\":2628000000000000}},\"ServerBind\":\":8080\",\"ScrapeTime\":{\"General\":300000000000,\"Summary\":900000000000,\"Live\":60000000000}}"}

@yehaotong
Copy link
Author

yehaotong commented Nov 5, 2021

Incident and the summary are same error : Arguments Caused Error (code: 2002)
{"collector":"OnCall","file":"metrics_oncall.go:46","func":"Collect","level":"debug","msg":"fetch schedule oncalls (offset: 0, limit:100)"}
{"collector":"Summary","file":"metrics_summary.go:116","func":"collectIncidents","level":"panic","msg":"HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002)"}
{"collector":"Summary","file":"collector_general.go:37","func":"func1","level":"error","msg":"\u0026{0xc000136bd0 map[collector:Summary] 2021-11-05 05:36:30.9047973 +0000 UTC m=+180.192211501 panic 0xc00030de50 HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) \u003cnil\u003e \u003cnil\u003e }"}
{"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3300, limit:100)"}

@mblaschke
Copy link
Member

with the docker container this only happen for the summary collector?

@mblaschke
Copy link
Member

if you now use webdevops/pagerduty-exporter:main@sha256:e6034052004cafe271211e333832619762c99668e0c9ba8d03632ac059bc37ee and start it with --debug the exporter now logs all request URLs.. maybe that's helping

@yehaotong
Copy link
Author

For the debug log and URL information of the incident part, compared with the official API, I think there is no problem with this path, but I don't know why it reports the error of parameter error, which is really strange

      {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}

@yehaotong
Copy link
Author

This is really strange. For the same API, I can return correct results by querying through postman, but whether it is the latest docker or I pull the latest code to run locally, parameter errors will be reported when statuses includes resolved. Even if I manually change the query increment part in the local code to:

c.get(ctx,"/incidents?"+"statses[]=triggered&statses[]=acknowledged&statses[]=resolved")

This really makes me autistic

@mblaschke
Copy link
Member

even with offset and limit?

@yehaotong
Copy link
Author

Whether I add the limit and offset parameters or not, parameter errors will be reported:
limit=100&offset=100&statuses%5B%5D=triggered&statuses%5B%5D=acknowledged&statuses%5B%5D=resolved

@yehaotong
Copy link
Author

ping

@mblaschke
Copy link
Member

are you executing it with the same PagerDuty token in postman and for the exporter?
i'm executing the same queries against my developer account and a bigger account without issues 🤔

be aware that the query log might have a delay as the queries are queued in the collection run to reduce stress on PagerDuty API.
are there any other queries in the log?

i currently don't have a clue from where the error is coming from or what this is causing 🤔

PS: i'm taking this issue serious, but as this is a FOSS project i can only invest my free time to handle issues (and there are more projects) so please be patient.

@mblaschke
Copy link
Member

ah and please post the log again, the last version in docker added some details about the time selection for the summary collector.

@yehaotong
Copy link
Author

Thank you for your energy and time. At this point, I'm a go beginner, so I try to find out whether this phenomenon is caused by improper penetration of my own parameters or other reasons in the code, but I haven't found anything for a long time. In addition, the tokens I use in the code are consistent with those penetrated by docker, In postman, I use the same token (the simulated request in postman finds out that it includes the increment in all States)

@yehaotong
Copy link
Author

yehaotong commented Nov 12, 2021

this is my docker debug log by the version:main and the command
docker run --rm -ti -p 8080:8080 webdevops/pagerduty-exporter:main --pagerduty.authtoken=xxxx --pagerduty.incident.status=all --pagerduty.disable-teams --debug

    {"collector":"Incident","file":"metrics_incident.go:87","func":"Collect","level":"panic","msg":"HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002)"}
    {"collector":"Incident","file":"collector_general.go:37","func":"func1","level":"error","msg":"\u0026{0xc000218000 map[collector:Incident] 2021-11-12 01:09:18.258433 +0000 UTC m=+366.195440701 panic 0xc0001b6190 HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) \u003cnil\u003e \u003cnil\u003e }"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"collector_base.go:122","func":"sleepUntilNextCollection","level":"debug","msg":"sleeping 1m0s"}
    {"collector":"OnCall","file":"collector_base.go:122","func":"sleepUntilNextCollection","level":"debug","msg":"sleeping 1m0s"}
    {"collector":"OnCall","file":"collector_base.go:105","func":"collectionStart","level":"info","msg":"starting metrics collection"}
    {"collector":"Incident","file":"collector_base.go:105","func":"collectionStart","level":"info","msg":"starting metrics collection"}
    {"collector":"OnCall","file":"metrics_oncall.go:46","func":"Collect","level":"debug","msg":"fetch schedule oncalls (offset: 0, limit:100)"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 0, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/oncalls?earliest=true\u0026limit=100"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"OnCall","file":"metrics_oncall.go:46","func":"Collect","level":"debug","msg":"fetch schedule oncalls (offset: 100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/oncalls?earliest=true\u0026limit=100\u0026offset=100"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"OnCall","file":"metrics_oncall.go:46","func":"Collect","level":"debug","msg":"fetch schedule oncalls (offset: 200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/oncalls?earliest=true\u0026limit=100\u0026offset=200"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"OnCall","file":"metrics_oncall.go:46","func":"Collect","level":"debug","msg":"fetch schedule oncalls (offset: 300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/oncalls?earliest=true\u0026limit=100\u0026offset=300"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"OnCall","file":"metrics_oncall.go:46","func":"Collect","level":"debug","msg":"fetch schedule oncalls (offset: 400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/oncalls?earliest=true\u0026limit=100\u0026offset=400"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 10000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=10000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:87","func":"Collect","level":"panic","msg":"HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002)"}
    {"collector":"Incident","file":"collector_general.go:37","func":"func1","level":"error","msg":"\u0026{0xc000218000 map[collector:Incident] 2021-11-12 01:10:17.5645894 +0000 UTC m=+425.570023001 panic 0xc00024ca00 HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) \u003cnil\u003e \u003cnil\u003e }"}
    {"collector":"OnCall","duration":5.4836496,"file":"collector_base.go:116","func":"collectionFinish","level":"info","msg":"finished metrics collection (duration: 5.4836496s)"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 7900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=7900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 4900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=4900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 1900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=1900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 8900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=8900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 5900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=5900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 2900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=2900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6400, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6400\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6500, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6500\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9800, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9800\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6600, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6600\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"OnCall","file":"collector_base.go:122","func":"sleepUntilNextCollection","level":"debug","msg":"sleeping 1m0s"}
    {"collector":"OnCall","file":"collector_base.go:105","func":"collectionStart","level":"info","msg":"starting metrics collection"}
    {"collector":"OnCall","file":"metrics_oncall.go:46","func":"Collect","level":"debug","msg":"fetch schedule oncalls (offset: 0, limit:100)"}
    {"collector":"Incident","file":"collector_base.go:122","func":"sleepUntilNextCollection","level":"debug","msg":"sleeping 1m0s"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/oncalls?earliest=true\u0026limit=100"}
    {"collector":"Incident","file":"collector_base.go:105","func":"collectionStart","level":"info","msg":"starting metrics collection"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 0, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 9900, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=9900\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3200\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"OnCall","file":"metrics_oncall.go:46","func":"Collect","level":"debug","msg":"fetch schedule oncalls (offset: 100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/oncalls?earliest=true\u0026limit=100\u0026offset=100"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 6700, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=6700\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"OnCall","file":"metrics_oncall.go:46","func":"Collect","level":"debug","msg":"fetch schedule oncalls (offset: 200, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/oncalls?earliest=true\u0026limit=100\u0026offset=200"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 100, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=100\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 10000, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=10000\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:81","func":"Collect","level":"debug","msg":"fetch incidents (offset: 3300, limit:100)"}
    {"file":"main.go:233","func":"pagerdutyRequestLogger","level":"debug","msg":"send request to https://api.pagerduty.com/incidents?limit=100\u0026offset=3300\u0026statuses%5B%5D=triggered\u0026statuses%5B%5D=acknowledged\u0026statuses%5B%5D=resolved"}
    {"collector":"Incident","file":"metrics_incident.go:87","func":"Collect","level":"panic","msg":"HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002)"}
    {"collector":"Incident","file":"collector_general.go:37","func":"func1","level":"error","msg":"\u0026{0xc000218000 map[collector:Incident] 2021-11-12 01:11:14.8721068 +0000 UTC m=+482.945588901 panic 0xc002254460 HTTP response failed with status code 400, message: Arguments Caused Error (code: 2002) \u003cnil\u003e \u003cnil\u003e }"}
    {"collector":"Incident","file":"collector_general.go:39","func":"func1","level":"panic","msg":"Error threshold reached, stopping exporter"}
    panic: (*logrus.Entry) 0xc000e03650 [recovered]
        panic: (*logrus.Entry) 0xc000e037a0
    
    goroutine 299 [running]:
    github.com/sirupsen/logrus.(*Entry).log(0xc00052f4d0, 0x0, {0xc0039fc390, 0x2a})
        /go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:259 +0x26f
    github.com/sirupsen/logrus.(*Entry).Log(0xc000219730, 0x0, {0xc00052f4d0, 0x6000103, 0x8eb2e0})
        /go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:293 +0x4f
    github.com/sirupsen/logrus.(*Entry).Panic(...)
        /go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:331
    main.(*CollectorGeneral).Collect.func1()
        /go/src/github.com/webdevops/pagerduty-exporter/collector_general.go:39 +0xa9
    panic({0x8f6a40, 0xc000e03650})
        /usr/local/go/src/runtime/panic.go:1038 +0x215
    github.com/sirupsen/logrus.(*Entry).log(0xc00052f918, 0x0, {0xc0026cbda0, 0x57})
        /go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:259 +0x26f
    github.com/sirupsen/logrus.(*Entry).Log(0xc000219730, 0x0, {0xc00052f918, 0x0, 0x0})
        /go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:293 +0x4f
    github.com/sirupsen/logrus.(*Entry).Panic(...)
        /go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:331
    main.(*MetricsCollectorIncident).Collect(0xc00022ba40, {0xc00002b3c0, 0x618dbe69}, 0x0)
        /go/src/github.com/webdevops/pagerduty-exporter/metrics_incident.go:87 +0x318
    main.(*CollectorGeneral).Collect(0xc0002149c0)
        /go/src/github.com/webdevops/pagerduty-exporter/collector_general.go:71 +0x153
    main.(*CollectorGeneral).Run.func1.1()
        /go/src/github.com/webdevops/pagerduty-exporter/collector_general.go:25 +0x1d
    created by main.(*CollectorGeneral).Run.func1
        /go/src/github.com/webdevops/pagerduty-exporter/collector_general.go:24 +0x29

@mblaschke
Copy link
Member

how many incidents to you have in the system and do you really need all of the 10000 incidents as metric?
It seems that the incident fetch is not even finished when the new run starts, could you please increase the scrape times?

@yehaotong
Copy link
Author

I may not know whether the system in my system reaches 10000. Is there a startup parameter you can set in room 10000? I just want to get the increment 24 hours away from the current,and I see that the default scrape times is 5m ,I update that 10m,but the same mistake still occurred,I cound not get the incident&summary metric.

@mblaschke
Copy link
Member

can you retry with webdevops/pagerduty-exporter@sha256:4a1f4b700b8ee5fc53a0a74e292b370653717aa37a1c333c1987d701c30a7fd1? there was a bugfix in the pagerduty sdk which i've updated and limited the default incidents by 5000.

IMHO the exporter is not made for mirroring the pagerduty database even if this should be possible.. but it should create metrics out of it.

@mblaschke
Copy link
Member

any feedback?

@yehaotong
Copy link
Author

I'm sorry I didn't reply because I was delayed by other things during this period. This problem has always existed. At the same time, I want to ask what the numerical value of each metric represents? The duration of the event? This means that can I use this exporter to count event duration data?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants