Infinite redirect loop when attempting to enable SSL on most recent image

[apjoseph]

I'm having an issue with an infinite redirect after logging into the carto app when I attempt to enable ssl on the most recent image on docker hub (2 months ago).

What other specific changes do I need to make for this to work? I'm not necessarily needing the full production configuration (though if a repo with a WORKING version exists, I'd be happy if you could point me towards it), just want to get the app to function as it does now but with https.
My EXACT configuration below -no other changes made:

cartodb.nginx

server {
  listen         80;
  server_name    _;
  return         301 https://$server_name$request_uri;
}

server {

  listen 443 ssl;

  server_name _;

  client_max_body_size 0;

  ssl_certificate           /etc/nginx/ssl/www_example_com.crt;
  ssl_certificate_key       /etc/nginx/ssl/www_example_com.key;
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_session_cache shared:SSL:10m;
  ssl_session_timeout 10m;
  ssl_prefer_server_ciphers on;
  ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK';

  location ~* /(user/.*/)?api/v1/maps {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_pass http://127.0.0.1:3000;
  }

  location ~* /(user/.*/)?api/v1/map {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_pass http://127.0.0.1:8181;
  }

  location ~* /(user/.*)?/api/v2/sql {
     RedHog: Hack to work around bug in cartodb local hosting but using cdn for js libs
    rewrite /(user/.*)?/api/v2/sql(.*) /$1/api/v2/sql$2  break;
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_pass http://127.0.0.1:8080;
  }

  location ^~ /assets {
    root /cartodb/public;
  }

  location / {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_pass http://127.0.0.1:3000;
  }

  error_log /var/log/nginx/cartodb_error.log;
  access_log /var/log/nginx/cartodb_access.log;
}

Dockerfile:

FROM sverhoeven/cartodb:latest

RUN  sed -i '/config.assets.initialize_on_precompile = true/a \ \ config.force_ssl = true' /cartodb/config/environments/development.rb \
     && sed -i 's/base_url: public_url/base_url: public_url.sub(\x27http\x27,\x27https\x27)/' /cartodb/app/models/user/user_decorator.rb
ADD config/cartodb.nginx /etc/nginx/sites-enabled/default

Run command:

docker run --name carto1 -d -p 443:443 -p 80:80 -e CARTO_HOSTNAME=carto.example.com -h carto.example.com \
  --mount type=volume,source=carto-certs,target=/etc/nginx/ssl \
  example/cartodb

[aarontract]

Hi, I had a similar issue and think I have a fix for it.

Sorry I haven't written it up very well, #68

I used this as a starting point: #28

The trick seems to be setting rails environment to production

The main changes was in the Docker file:
ENV RAILS_ENV production
RUN cp /cartodb/config/environments/development.rb /cartodb/config/environments/production.rb
RUN sed -i "s/database: carto_db_production/database: carto_db_development/g" /cartodb/config/database.yml

I also had a few lines to update /cartodb/config/app_config.yml for port 443/https

Seems that now carto have lots of checks for dev environment and will redirect to http://.

[aarontract]

Also you should be able to remove
server {
listen 80;
server_name _;
return 301 https://$server_name$request_uri;
}

I found the change to /cartodb/app/models/user/user_decorator.rb seems to cause errors/crash when i create a new tables.

[bplmp]

Also getting infinite redirects here. It's been really difficult to set up https on the application side.

For me it seems that adding ENV RAILS_ENV=production to the beginning of the Dockerfile is what is triggering the redirect loop.

[aarontract]

I'm happy to try to help you if i can, Could you list out the steps you have done to set it up briefly? I was working on a different version in November and December so my method might be broken now.

Do you have nginx running outside docker on the host machine?

Have you made the changes to the following files to allow for HTTPS+ different/ports?
/cartodb/config/app_config.yml
/Windshaft-cartodb/config/environments/development.js

Also people have been able to get it to work without ENV RAILS_ENV=production but i couldn't.

[bplmp]

Thanks, @aarontract.

I have setup ssl using a load balancer, outside the application and outside nginx.

My steps:

1. Set ENV RAILS_ENV=production at the beginning of Dockerfile
2. Change config/app_config.yml to https and port 443
3. Changed database for production to carto_db_development in config/database.yml
4. Inside the container, copied development.rb to production.rb in /cartodb/config/environments/

But still getting infinite redirects. If I remove ENV RAILS_ENV=production the infinite redirects stop and it works, except that when I log in I keep being redirected to http, like in your issue here #68

[aarontract]

All sounds good to me.

In your nginx rule on your docker server machine do you have something like

server {
server_name YourPublicDomainName
......

location ~* /(user/.*/)?api/v1/maps {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:3000;
}

location ~* /(user/.*/)?api/v1/map {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:8181;
}

location ~* /(user/.)?/api/v2/sql {
rewrite /(user/.)?/api/v2/sql(.*) /$1/api/v2/sql$2 break;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:8080;
}

It could be something in cartodb inside the docker container seeing the requests with a destination address of your servers local network hostname, not your public domain name and it's trying to redirect you to that public address. If this is happening this might be caused by your load balancer routing the requests. I think I experienced this at some point.

Just to be clean, with my setup, I more or less have the same nginx rule more above set on my host machine, with docker I expose ports 3000, 8080, 8181. The nginx inside the docker host isn't doing much.

The only difference between my setup and yours is I have lets encrypt cert on my server, rather than one on a load balancer. But it should still work.

[bplmp]

I have the same nginx config as you... The fact that the ENV RAILS_ENV=production config triggers the infinite redirect makes me think that this is an issue with the cartodb application, not with nginx.

Maybe related to this CartoDB/cartodb#3927

[aarontract]

Sorry, I'm not sure what is going on. I have tried to replicate your setup by putting a separate load balancer in front of my box + pull the latest image and everything is still working.

I have pulled the latest image, ran all my setup.

On AWS
Created a clastic load balancer,
I have it listening on https, and the internal port is 443/https, pointed it at my carto server.

The carto server is only listening on port 443.

I updated the dns details to point to the load balancer, not my carto server's address.

And that was it, I didn't make any changes to the setup.

I will try to clean up my setup document post it up in the next few days.

You might have to just get really creative with your trail and error, I spent a few very frustrating days on it before it started working.

[bdecarne]

Hello there !

Get the same problem here, with Traefik as reverse-proxy/load balancer :(

[bplmp]

@aarontract could you maybe post the config files for your setup with the load balancer, the one you managed to get working?

If you could post your app_config.yml and your cartodb.nginx.proxy.conf that would be very helpful.

[aarontract]

For the load balancer i just used an aws wizard. Might help to take this out, get a lets encrypt cert on the host machine and have it only take request over https, if you can get that working try adding the load balancer as the last step.

I have done a find and replace for app_config.yml, development.js and database.yml,

In /cartodb/config/app_config.yml:
protocol: 'http' to protocol: 'https'
port: '80' to port: '443'
protocol: 'http' to protocol: 'https'
port: 80 to port: 443
"http" to "https"
"80" to "443"

In /Windshaft-cartodb/config/environments/development.js :
http: 'http: ' to http: 'https'
https: 'http: ' to http: 'https'
environment: 'development' to environment: 'production'

in /cartodb/config/database.yml
database: carto_db_production to database: carto_db_development

run:
cp /cartodb/config/environments/development.rb /cartodb/config/environments/production.rb

here is a copy of app_config.yml:
https://textuploader.com/1aipf

nginx rule sitting on the host box is:
https://textuploader.com/1aip7

Also here is my docker-compose file:
https://textuploader.com/1ai7m