Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to initialize Metadata client #21

Open
Braintelligence opened this issue Apr 13, 2017 · 29 comments
Open

Failed to initialize Metadata client #21

Braintelligence opened this issue Apr 13, 2017 · 29 comments

Comments

@Braintelligence
Copy link

Hi there.
I tried running RAP but when I fire docker-compose up I see this error in the logs:

Failed to initialize Rancher Metadata client: Get http://rancher-metadata/latest/version: dial tcp: lookup rancher-metadata on 127.0.0.11:53 no such host

I think this is due to using rancher-proxy without having a rancher-agent running.
My problem is, though, that I want to use a rancher-server and -agent on the same host and I need some best practice on what my initial docker-compose with the rancher-server and RAP should be, so I can reach the rancher-server on my domain, before I create the agent.

Any ideas on how I should tackle this? =/

Thank you a lot for your hard work!

@adi90x
Copy link
Owner

adi90x commented Apr 14, 2017

Hello,

To setup rancher-server with RAP : you can start Rancher Server like that :
docker run -d -v /home/data/mysql:/var/lib/mysql --restart=unless-stopped -p 8080:8080 --name=rancher-server -l rap.host=rap.foo.bar -l rap.port=8080 -l rap.le_host=rap.foo.bar -l [email protected] -l io.rancher.container.pull_image=always rancher/server
This will start rancher server on port 8080, connect to foo.bar:8080 and add an host ( do not forget to setup the IP if you want to have server and agent on the same host ) and run the command on you host.

Then you can go to ADMIN > SETTING >> Add a Catalog part :
https://github.com/adi90x/rancher-active-proxy.git

Then go to CATALOG and add Rancher Active Proxy Stack

And it should start Rancher Active Proxy !

Do not hesitate , if you need some more information.

@Braintelligence
Copy link
Author

Thanks for your response @adi90x! Can you elaborate on what part of your command automatically adds a host?
It looks like it only spins up a Rancher Server.

@adi90x
Copy link
Owner

adi90x commented Apr 14, 2017

No part of the command add a host.
When you start rancher server , you can go to the managing interface and then Infrastructure > Host > Add a host .
And you simply follow rancher procedure to setup a new host.

@adi90x
Copy link
Owner

adi90x commented Apr 14, 2017

I recommend not using docker-compose, but using the Rancher catalogue entry directly.

@Braintelligence
Copy link
Author

I understand. Your comment sounded like your command would start a host as well :).
This way, if I restart the whole machine, will rancher-server restart automatically or is this considered "stopping" the container in the sense of "unless-stopped"?

Thanks a lot for your help :).

@adi90x
Copy link
Owner

adi90x commented Apr 14, 2017

If you have /home/data/mysql:/var/lib/mysql in you rancher server start command , it will save all parameters on the host. And yes , if you restart you the host , it will rancher will restart and everything will come back in the state it was !

@Braintelligence
Copy link
Author

Thanks a lot! I will try this out and ping you up if I need further help 👍 .

@Braintelligence
Copy link
Author

A question came up:
If I do it like this, then the initial connection will be without SSL encryption. If I spin up the rancher-agent with the localhost IP I think this is no problem, because it will stay internal communication, right?
All other rancher-agents should run with the RAP-enabled https-connection, then, correct?

@Braintelligence
Copy link
Author

Braintelligence commented Apr 14, 2017

I started rancher-server with your command and then added a host to the static IP and port 8080.
Then I added your catalog and started the rancher active proxy stack but it looks like RAP can't see the rancher-server.
I tried docker restart rancher-server, but it didn't help. Is this maybe the same problem like with nginx-proxy and letsencrypt-companion? Do they have to be on the same docker network?

I want to reach rancher-server on my.domain.com, is this not possible with RAP?

@adi90x
Copy link
Owner

adi90x commented Apr 14, 2017

it is , when you start the rancher-server containers at the beggining , you can add rap label to it in the example it was : -l rap.host=rap.foo.bar that would lead to rancher server available at rap.foo.bar

@Braintelligence
Copy link
Author

The label is already contained in the docker run command. Do I have to add it again while the container is running?

@ghost
Copy link

ghost commented Apr 14, 2017

Hi guys, I just found this interesting & still hot discussion, so let me summarize how I would do the setup without ssl ( in virtual box) and later with ssl somewhere out there..

  1. start rap on port 80
  2. start rancher-server with the label rap.host=xyz.local (later also with the letsencrypt labels)
  3. Navigate to xyz.local ( rancher-server ) a: add a host; b: add rap repo, c: install rap
  4. Start other services proxied by rap

Right?

@adi90x
Copy link
Owner

adi90x commented Apr 14, 2017

@dropfen No step 1 !

Setup rancher server as normal, grab the catalog, add it , fire a new RAP stack from the catalog.
when you setup rancher server , you may add to the container some rap label that will only be active after rap start.
However , you still need to acces rancher server with an external port at least to set it up !

@Braintelligence
Copy link
Author

@adi90x I did just as you say, but it doesn't work. The rancher-server is on a different subnet than RAP and I don't see RAP recognizing the rancher-server container in the logs, when I restart it.
I can't access the rancher-server on the rap.host and a certificate isn't being generated.

@ghost
Copy link

ghost commented Apr 14, 2017

@adi90x thx. I got it. If something goes wrong, I'll come back, hopefully not :)

@Braintelligence
Copy link
Author

@dropfen If you can make this work (aka accessing the rancher-server via https on a domain with RAP container running on the same host) then please come back and describe how you did it ☹️ .

@ghost
Copy link

ghost commented Apr 14, 2017

@Braintelligence 👌

@adi90x
Copy link
Owner

adi90x commented Apr 14, 2017

That is my exact setup ! It should be working !
To summarize, I have one physical host running docker,
Fire rancher with the first command, access rancher server using IP of the physical host:8080, configure an host in rancher specifying the IP of the physical host ( as explained by rancher doc). Then when the agent is running go to catalog to activate RAP.
If there is problem between server and agent, double check in rancher server Admin > host registration URL and double check that it is ending by 8080 to match the port use by rancher server

@Braintelligence
Copy link
Author

Can you Plesse show an example of rap container logs how it looks like if it Sees a New container with rap.host? Also what are the internal IP adresses of rancher-server and rap? (You can See that in infrastructure > containers.

@ghost
Copy link

ghost commented Apr 14, 2017

Hey, I got it up and running on VirtualBox at least :)

My Setup:

Container Host -> RancherOS

  • local IP => 192.168.99.100 its important cause I used it, to add hosts and agents
  1. start rancher/server like already mentioned with the -l rap.host=rancher.local
  2. add a Host
  • use the IP:8080 as the host registration domain
  • set the same IP for the agent in the GUI
  • copy the command & run it
  1. Add the RAP catalogue
  2. Launch the RAP service
  3. Now you can access the rancher-server on rancher.local(:80) 🎉
    Hint: of course if its local env you should have a /etc/hosts entry for rancher.local -> IP
  4. Further services should also get the rap.host label to be accessible on port 80

@Braintelligence
Copy link
Author

Hm. I will try again later completely from scratch. Maybe I've missed something.

@ghost
Copy link

ghost commented Apr 14, 2017

Its also important that your server/vm has enough capacities to run the server & the host. With 1GB of RAM it just crashed.. So now with 2GBs its at least running, but some people are saying, that after some time it eats all the memory and so on. So its not related to RAP, btw. thank you @adi90x, but rancher is not a small fancy toy for small projects.

Actually, I am going to setup a complete GitLab Infrastructure on the same host as the server, and for production it will be some other host. I am planning to do it on a 6GB machine, and I am also not quite sure if its enough or not for the builds and so on.

However

Good luck @Braintelligence
Thanks a lot for RAP @adi90x

@Braintelligence
Copy link
Author

I'm running this stuff on a VM with 4 Xeon Cores and 16GB ECC RAM on a RAID SSD... Hardware is definitely not the problem.

@Braintelligence
Copy link
Author

Ok, so I'm retrying from scratch. Let's make a history of everything I'm doing, so others won't run into the same problems.

I'm running this on a ubuntu server VM with the latest docker and docker-compose.

I killed all (also dangling) docker containers, volumes and images. I disabled the ufw firewall (even though I made it work with everything up till now).

I have two scripts, so I don't need to type it everytime I want to fiddle around.
One of them is like this:

docker run -d -v /home/USER/rancher-mysql-data:/var/lib/mysql --restart=unless-stopped -p 8080:8080 --name=rancher-server -l rap.host=my.domain.com -l rap.port=8080 -l rap.le_host=my.domain.com -l [email protected] -l io.rancher.container.pull_image=always rancher/server

The other one is like this:

docker run -e CATTLE_AGENT_IP="IP.OF.RANCHER.SERVER" -d --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.2 http://IP.OF.RANCHER.SERVER:8080/v1/scripts/yadayadayada:012345678:yadayada

copied from the rancher-server.

  1. I fire up the rancher-server script with sudo.
  2. I visit :8080 and set up local auth.
  3. I save the rancher-server address as http://IP.OF.RANCHER.SERVER:8080 in settings.
  4. I add the RAP catalog.
  5. I create a RAP stack, only change the PATH to /etc/nginx and the default mail to [email protected]
  6. Look into RAP logs, nothing about my.domain.com to be seen.
  7. When I try to use my normal firefox browser to get there nothing happens, because it tries to enforce https from memory. I use other browsers and am able to reach http://my.domain.com but it doesn't work with https. I don't see anything about letsencrypt in the logs and the ssl folder only contains dhparam.pem.

@adi90x Any idea why no letsencrypt certificate is being fetched?

@Braintelligence
Copy link
Author

I added /etc/nginx/conf.d to the bound volumes. The default.conf doesn't even contain anything about SSL connections.
I added -l rap.proto=https to the server start script, killed the container and started it up again but that didn't change anything about the default.conf.

@Braintelligence
Copy link
Author

Wow, ok.
I tried to start up another stack (Wekan docker) just for shits and giggles.
When that one started up RAP fetched the certificate for rancher-server.domain.com and wekan.domain.com. Now everything works.

So now... why does it need for another stack to spin up for RAP to recognize the rancher-server needs a letsencrypt certificate, too?

@ghost
Copy link

ghost commented Apr 16, 2017

@Braintelligence I finally finished the setup for my infra environment.
Everything works fine, rancher-server & gitlab are running behind ssl, great! 🎉

To trigger the vhost and certs creation for rancher-server (without other services), I had to stop it, remove it, and run it again. When I then started Gitlab, it triggered the creation of the corresponding rap stuff immediately.

@Braintelligence
Copy link
Author

Yeah, so it looks like you have to create another stack for RAP to be triggered for the rancher-server. Maybe this should be added to the readme for this use-case.

@ghost
Copy link

ghost commented Apr 16, 2017

good point 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants