Error with Reddit feeds

[guyfedwards]

    - url: https://www.reddit.com/r/golang/.rss

[guyfedwards]

This is due to reddits rate-limiting. Looking at ways to try and bypass it

[Nemoden]

This is due to reddits rate-limiting. Looking at ways to try and bypass it

I don't think it's because of rate-limiting, because once feed is ingested, we save it to local cache. We literally request it once.

This one is to do with http2 somehow.

I'm on the nom branch that I've created to tackle the #13 so I have --feed flag (which is quite handy to test too)

go run cmd/nom/main.go --feed="https://www.reddit.com/r/golang/.rss" -v
commands List: commands List: rss.Fetch: http error: 403 Forbidden

And I'm getting the "blocked" page.

If I disable http2, everything is fine:

GODEBUG=http2client=0 go run cmd/nom/main.go --feed="https://www.reddit.com/r/golang/.rss" -v

There IS rate-limiting, yes, but shouldn't get in a way at least on first fetch of the reddit feed.

So, in order for reddit feeds to work, http2 has to be disabled. The way to disable it is descried here pretty well grafana/k6#936

[guyfedwards]

Great find, was weird we were getting a rate limit response for single requests so glad we now know.

I fixed and tested this quickly and created a branch here, is working for me but would be great if you could check and confirm.