art-compose and hosting a docker repository not working together.

[aram535]

I have a 3 note setup (by using art-compose) and then hosting a docker repository in the resulting artifactory instance. I can login to the docker repo if I point the docker client directly at artifactory but if I try to use the nginx the authentication doesn't seem to be making it across and it gives a strange error about v2 docker not supported.

I'm duplicating here but the original question is on slashdot.

• CentOS 7.3
• SELinux Disable
• All activity below is on the "host" machine, just using different IP addresses and ports.
• 10.15.53.21 is the host itself, this is where the art-compose was run.
• 172.19.0.3 is the bridged docker network for the artifactory container. nginx is 172.19.0.4.

We're running artifactory 6.0.2 6.0.3 on-perm as a container (along with nginx and postgresql) as described in running artifactory as a docker instance.

I have set up the basic "docker" instance.

Locally on the same machine as the artifactory container:

# docker login localhost:8081
Username: test
Password: 
Login Succeeded

However, using the nginx port:

# export DOCKER_OPTS=" --insecure-registry docker-local.artifactory.company.com"
# echo password | docker login -u test --password-stdin artifactory.company.com
Error response from daemon: Get https://artifactory.company.com/v2/: unknown: Unsupported docker repository request for 'v2'

I also tried using ~/.docker/config.json with the auth information (base64) however it still giving me the same error.

artifactory.log shows:

2018-06-21 13:25:48,088 [http-nio-8081-exec-3] [ERROR] (o.a.a.d.r.DockerResource:425) - Unsupported docker repository request for 'v2'

access.log shows:

2018-06-21 13:25:48,089 [DENIED LOGIN]  for test/172.19.0.4

request.log shows:

20180625075813|1|REQUEST|172.19.0.2|_internal|GET|/api/system/configuration/reverseProxy/nginx|HTTP/1.1|404|0
20180625075823|1|REQUEST|172.19.0.2|_internal|GET|/api/system/configuration/reverseProxy/nginx|HTTP/1.1|404|0
20180625075823|1|REQUEST|10.15.9.4|**non_authenticated_user**|GET|/api/docker//v2/|HTTP/1.0|401|0
20180625075823|2|REQUEST|10.15.9.4|**non_authenticated_user**|GET|/api/docker/v2/token|HTTP/1.0|400|0

nginx access.log:

ip = 10.15.53.21 user = "-" local_time = "27/Jun/2018:11:28:36 +0000" host = artifactory.company.com request = "GET /v2/ HTTP/1.1" status = 401 bytes = 87 upstream = "172.19.0.3:8081" upstream_time = 0.001 request_time = 0.001 referer = "-" UA = "docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/3.10.0-862.3.3.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \x5C(linux\x5C))"
ip = 10.15.53.121 user = "test" local_time = "27/Jun/2018:11:28:36 +0000" host = artifactory.company.com request = "GET /artifactory/api/docker//v2/token?account=test&client_id=docker&offline_token=true&service=artifactory.company.com%3A443 HTTP/1.1" status = 400 bytes = 122 upstream = "172.19.0.3:8081" upstream_time = 0.001 request_time = 0.001 referer = "-" UA = "docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/3.10.0-862.3.3.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \x5C(linux\x5C))"

[jainishshah17]

@aram535 can you share your nginx.conf?

[aram535]

It's the auto-generated one with just the new set of keys.
The internal artifactory Http Settings are "Tomcat" and "Repository Path"... should have mentioned that in the original post.

ssl_certificate  /var/opt/jfrog/nginx/ssl/artifactory.pem;
ssl_certificate_key  /var/opt/jfrog/nginx/ssl/artifactory.key;
ssl_session_cache shared:SSL:1m;
ssl_prefer_server_ciphers   on;
## server configuration
server {
  listen 443 ssl;
  listen 80 ;
  server_name ~(?<repo>.+)\.artifactory artifactory;

  if ($http_x_forwarded_proto = '') {
    set $http_x_forwarded_proto  $scheme;
  }
  ## Application specific logs
  ## access_log /var/log/nginx/artifactory-access.log timing;
  ## error_log /var/log/nginx/artifactory-error.log;
  rewrite ^/$ /artifactory/webapp/ redirect;
  rewrite ^/artifactory/?(/webapp)?$ /artifactory/webapp/ redirect;
  rewrite ^/(v1|v2)/(.*) /artifactory/api/docker/$repo/$1/$2;
  chunked_transfer_encoding on;
  client_max_body_size 0;
  location /artifactory/ {
    proxy_read_timeout  900;
    proxy_pass_header   Server;
    proxy_cookie_path   ~*^/.* /;
    proxy_pass          http://artifactory:8081/artifactory/;
    proxy_set_header    X-Artifactory-Override-Base-Url $http_x_forwarded_proto://$host:$server_port/artifactory;
    proxy_set_header    X-Forwarded-Port  $server_port;
    proxy_set_header    X-Forwarded-Proto $http_x_forwarded_proto;
    proxy_set_header    Host              $http_host;
    proxy_set_header    X-Forwarded-For   $proxy_add_x_forwarded_for;
  }
}

[jainishshah17]

@aram535 Thank you for sharing config. You are missing one step. add following entry in your etc/hosts file:

127.0.0.1 docker.artifactory docekr-local.artifactory docker-remote.artifactory

then try to pull docker image using

# make sure you have insecure registry configured for `docker.artifactory`, `docekr-local.artifactory` and  `docker-remote.artifactory`
docker pull docker.artifactory/node

[aram535]

Thank you for that update.... I do have the host entry but not for 127.0.0.1 it's pointing to the 10.x address.

10.15.53.21 artifactory.company.com docker.artifactory.company.com docker-remote.artifactory.company.com

now I see that if I do:

# docker login docker.artifactory
Username (test): test
Password: 
Error response from daemon: Get https://docker.artifactory/v2/: x509: certificate is valid for *.company.com, company.com, artifactory.company.com, not docker.artifactory

Is that my problem with accessing this repository over the internet with authentication? Do I need to get a star certificate for *.artifactory.company.com?

[jainishshah17]

Your etc/hosts entry is fine. I thought you are using default SSL certs. In this case you will have to change HTTP settings in Artifactory -> Admin -> Configuration -> HTTP Setting.
change the public server name to artifactory.company.com and Server Provider to Nginx also change other fields accordingly

[aram535]

Public Server Name is set, changed Tomcat to Nginx
Changed the docker access method to subdomain *.artifactory.company.com

When I try to login I still get the

certificate is valid for *.company.com, company.com, artifactory.company.com, not docker.artifactory

[jainishshah17]

change the public server name to company.com

[aram535]

Same cert error.

[aram535]

ah oh, I think it broke the web ui when I changed the servername to company.com.

[jainishshah17]

did it fix your issue?

[aram535]

no same cert error for docker and I have lost access to the web portal. I "think" I can change the public server name back using API, correct?

[jainishshah17]

Yes, Here is the link https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API#ArtifactoryRESTAPI-UpdateReverseProxyConfiguration

[aram535]

Ok recovered. I think the problem is that pem/key that I'm using are *.company.com. Quenstion would be how do you apply one star cert to artifactory.company.com and another star cert to *.artifactory.company.com?

[jainishshah17]

If your cert is valid for *.company.com then you don't need to pass another cert. Just creating two DNS record is enough.

1. artifactory.company.com pointing to IP of Nginx
2. *.company.com pointing to IP of Nginx
   Note: You can not use cert for *.company.com with *.artifactory.company.com.

[jainishshah17]

@aram535 Are you able to resolve your issue?

[aram535]

Sort of. Only works internally (insecure mode). I cannot figure out how to have both a *.company.com cert for the UI and the *.artifactory.company.com for docker.

[jainishshah17]

@aram535 AFAIK you can not achieve this with same certs. You have a cert for the subdomain *.company.com not for sub.subdomain *.*.company.com .

[aram535]

Sorry. I didn't mean to do it with one cert. I'm trying to setup two different certs one for artifactory.company.com (*.company.com) and one for *.artifactory.company.com,, but the HTTPD setup only allows one cert to be applied to the configuration for Nginx.

[jainishshah17]

@aram535 Any update on this issue?

[aram535]

Two part ... adding the docker.artifactory.company.com to the local host file worked for an "insecure" connection. I still haven't been figure out how to apply a second certificate, a star cert, to cover both artifactory.company.com as well as *.artifactory.company.com.

[jainishshah17]

@aram535 You can try something like this -> https://stackoverflow.com/questions/14434120/nginx-set-multiple-server-name-with-ssl-support

[mcgurksean]

Any update on this issue, guys?

I am having a similar issue pulling from

registry-name.artifactory.company-name.com

where I get the error message:

certificate is valid for *.company-name.com, company-name.com, not registry-name.artifactory.company-name.com

So, I have created a DNS entry for

registry-name-artifactory.company-name.com

that points to

registry-name.artifactory.company-name.com (note the - replacing . in the DNS entry)

Attempting to log into this registry/ pull images from the registry gives the error message:

Error response from daemon: login attempt to https://registry-name-artifactory.company-name.com/v2/ failed with status: 400 Bad Request

My nginx configuration is as follows:

###########################################################
## this configuration was generated by JFrog Artifactory ##
###########################################################
## add ssl entries when https has been set in config
ssl_certificate      /etc/nginx/ssl/wildcard.company-name.com_02222016.crt;
#/etc/nginx/ssl/artifactory-wildcard.crt;
ssl_certificate_key  /etc/nginx/ssl/wildcard.company-name.com_02222016.key;
#/etc/nginx/ssl/artifactory-wildcard.key;
ssl_session_cache shared:SSL:1m;
ssl_prefer_server_ciphers   on;
## server configuration
server {
    listen 443 ssl;
    listen 80 ;
    server_name ~(?<repo>.+)\.artifactory.company-name.com artifactory.company-name.com registry-name-artifactory.company-name.com;

    if ($http_x_forwarded_proto = '') {
        set $http_x_forwarded_proto  $scheme;
    }
    ## Application specific logs
    ## access_log /var/log/nginx/artifactory.company-name.com-access.log timing;
    ## error_log /var/log/nginx/artifactory.company-name.com-error.log;
    rewrite ^/$ /artifactory/webapp/ redirect;
    rewrite ^/artifactory/?(/webapp)?$ /artifactory/webapp/ redirect;
    rewrite ^/(v1|v2)/(.*) /artifactory/api/docker/$repo/$1/$2;
    chunked_transfer_encoding on;
    client_max_body_size 0;
    location /artifactory/ {
    proxy_read_timeout  900;
    proxy_pass_header   Server;
    proxy_cookie_path   ~*^/.* /;
    if ( $request_uri ~ ^/artifactory/(.*)$ ) {
        proxy_pass          http://localhost:8081/artifactory/$1;
    }
    proxy_pass          http://localhost:8081/artifactory/;
    proxy_set_header    X-Artifactory-Override-Base-Url $http_x_forwarded_proto://$host:$server_port/artifactory;
    proxy_set_header    X-Forwarded-Port  $server_port;
    proxy_set_header    X-Forwarded-Proto $http_x_forwarded_proto;
    proxy_set_header    Host              $http_host;
    proxy_set_header    X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_redirect      http://localhost:8081 https://artifactory.company-name.com;
    }
}

Thanks,

Sean

[mcgurksean]

BTW, I think that I have solved this by amending the server_name part of the nginx configuration to:

server_name ~(?<repo>.+)\.artifactory.company-name.com artifactory.company-name.com ~(?<repo>.+)\-artifactory.company-name.com;

[aram535]

I have tried all of the above even with semi-similar DNS tricks, however, I think passing through Azure as my auth proxy is wrecking havoc with the certs and DNS. Funny enough everything else seems to work fine, it seems docker is the only one that has issues.

[aram535]

@mcgurksean how did you manage to get artifactory from overwriting your nginx config file? Every time I restart my docker-compose I have to re-edit the file (at least it works internally) but I cannot figure out how to stop it from re-generating the config file.

[jainishshah17]

@aram535 You can skip re-generating the config file by setting SKIP_AUTO_UPDATE_CONFIG=true in nginx service.

[aram535]

@jainishshah17 thanks. Do you happen to know if that's a ENV variable I can through to the docker instance? I used art-compose tool to generate my docker-compose.yml file.

[jainishshah17]

@aram535 You can put it after https://github.com/jfrog/artifactory-docker-examples/blob/master/docker-compose/artifactory/artifactory-pro.yml#L59

[eldada]

@aram535 - is this still relevant?

[aram535]

@eldada AFAIK it's still a manual change and you have to change the "auto-generate nginx config" in the app. Other than that it works.