Bors polls too hard

[brson]

Rust's bors instance is close to the API rate limit (though I don't have numbers), and already doesn't update fast enough, with a 5 minute polling interval. This situation impresses GitHub support none-to-much when asking for a rate limit increase.

I assume bors will need to be more stateful.

Note that Rust is still running an old version of bors that doesn't incorporate any of the changes from the last six months, so this behavior may be better now, but I don't think so.

[metajack]

You probably want a real daemon that listens to webhooks. That's basically a rewrite :(

[reiddraper]

We've been running into the same issue at Basho. I have a proof-of-concept to simply call bors from the cmd-line when a webhook fires, but unfortunately there are a few issues with this. For one, it's difficult to associate a commit-comment with the associated pull-request. As @brson says, the 'right' solution probably makes bors more stateful. We've already had to move our bors to 12-minute intervals.

[ghost]

Conditional requests do not count against the rate limit so bors could stay stateless as long as it employs some kind of a persistent cache for the HTTP requests it issues against the GitHub API.

[brson]

Rust is on 10 minute intervals now. @jakub- that sounds like the best short-term solution.

[ghost]

We could also just set-up a local nginx/Varnish proxy that acts as the said cache.

[reiddraper]

I looked into using Squid for us at Basho, but had quite a bit of trouble getting it to cache TLS requests to Github. Maybe nginx or Varnish will be easier.

I also tried to switch Bors to use httplib2, but ran into an issue there as well. httplib2 offers a persistent cache, and uses conditional requests, as suggested by @jakub-.

[ghost]

@brson @reiddraper

So I'm using the following in nginx:

proxy_cache_path /var/data/nginx/cache levels=1:2 keys_zone=cache:50m;
proxy_temp_path /tmp/nginx;

server {
    listen       8080;
    server_name  localhost;
    location / {
        proxy_cache_bypass "";
        proxy_cache cache;
        proxy_cache_revalidate on;
        proxy_cache_valid any 1s;
        proxy_http_version 1.1;
        proxy_ignore_headers "Cache-control";
        proxy_no_cache "";
        proxy_pass https://api.github.com;
    }
}

and then "gh_api": "http://localhost:8080" in bors.cfg.

There's a small issue with this set up, which is that nginx will always serve cached responses that are not older than one second. There doesn't seem to be any way around it as proxy_cache_valid any 0s actually disables caching altogether and greater granuality than seconds is not possible.

@brson For Rust, if we have an nginx instance with the above running on the same machine as bors, we can switch it to poll as often as we see fit.

[ghost]

We really should also be verifying the certificate but since the GitHub client we use doesn't, it's a separate issue.