community.docker.docker_compose failed with Docker SDK for Python 7.0.0 (Released: Dec 8, 2023)

[A-Mahla]

SUMMARY

community.docker.docker_compose fails to run my containers with Docker SDK for Python 7.0.0 (Released: Dec 8, 2023).

The error is related to a function call to "kwargs from_env()" which is passed an argument called "ssl_version" which the function no longer recognizes or supports.

Everything works with the penultimate stable version (Docker SDK for Python: 6.1.3).

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.docker.docker_compose

ANSIBLE VERSION

ansible 2.10.8
  config file = /home/vagrant/ansible.cfg
  configured module search path = ['/home/vagrant/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]

COLLECTION VERSION

none

CONFIGURATION

DEFAULT_HOST_LIST(/home/vagrant/ansible.cfg) = ['/home/vagrant/inventory']
DEFAULT_PRIVATE_KEY_FILE(/home/vagrant/ansible.cfg) = /home/vagrant/.ssh/ansible

OS / ENVIRONMENT

Linux 5.15.0-83-generic #92-Ubuntu SMP Mon Aug 14 09:30:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

STEPS TO REPRODUCE

Run this yaml in an Ubuntu 22.04 vagrant VM to install the packages and build up /app/docker/docker_compose.yml

- hosts: servers
  become: true

  tasks:
  - name: install dependencies
    tags: docker, boot
    apt:
      name:
        - apt-transport-https
        - ca-certificates
        - curl
        - gnupg-agent
        - software-properties-common
        - python3-pip
        - virtualenv
        - python3-setuptools
      state: present
      update_cache: yes

  - name: add docker GPG key
    tags: docker, boot
    apt_key:
      url: https://download.docker.com/linux/ubuntu/gpg
      state: present

  - name: add docker repository to apt
    tags: docker, boot
    apt_repository:
      repo: deb [arch=amd64] https://download.docker.com/linux/ubuntu jammy stable
      state: present

  - name: install Docker
    tags: docker, boot
    apt:
      name:
        - docker-ce
      state: present
      update_cache: yes

  - name:  install docker module for python
    tags: docker, boot
    pip:
      name:
        - docker == 7.0.0
        - docker-compose >= 1.7.0, < 2.0.0
        
  - name: build images and up containers
    tags: docker, up
    docker_compose:
      project_src: /app/docker
      state: present

EXPECTED RESULTS

My containers defined in /app/docker/docker_compose.yml are up.

ACTUAL RESULTS

The full traceback is:
  File "/tmp/ansible_docker_compose_payload_ekkjzyn7/ansible_docker_compose_payload.zip/ansible_collections/community/general/plugins/modules/docker_compose.py", line 670, in __init__
  File "/usr/local/lib/python3.10/dist-packages/compose/cli/command.py", line 60, in project_from_options
    return get_project(
  File "/usr/local/lib/python3.10/dist-packages/compose/cli/command.py", line 152, in get_project
    client = get_client(
  File "/usr/local/lib/python3.10/dist-packages/compose/cli/docker_client.py", line 41, in get_client
    client = docker_client(
  File "/usr/local/lib/python3.10/dist-packages/compose/cli/docker_client.py", line 124, in docker_client
    kwargs = kwargs_from_env(environment=environment, ssl_version=tls_version)
fatal: [192.168.56.111]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_version": "auto",
            "build": false,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "debug": false,
            "definition": null,
            "dependencies": true,
            "docker_host": "unix://var/run/docker.sock",
            "files": null,
            "hostname_check": false,
            "nocache": false,
            "project_name": null,
            "project_src": "/app/docker",
            "pull": false,
            "recreate": "smart",
            "remove_images": null,
            "remove_orphans": false,
            "remove_volumes": false,
            "restarted": false,
            "scale": null,
            "services": null,
            "ssl_version": null,
            "state": "present",
            "stopped": false,
            "timeout": 10,
            "tls": false,
            "tls_hostname": "localhost",
            "validate_certs": false
        }
    },
    "msg": "Configuration error - kwargs_from_env() got an unexpected keyword argument 'ssl_version'"
}

[madewithpat]

Encountered this same issue. Had to check my versions, as I use geerlingguy's roles for both the docker and pip installs

• ansible 2.15.6
• docker 7.0.0
• docker-compose 1.29.2

[felixfontein]

Ah, the always very popular major releases with breaking chnages on Friday evenings :D There will be some more modules/plugins in this collection that have trouble with this, I'll take a look at it later today.

Fortunately the breaking changes are very limited (see https://github.com/docker/docker-py/releases/tag/7.0.0), otherwise you might have quite a few more breakages, especially with docker-compose itself, which is EOL for quite some time now and won't get adjusted to such changes anymore.

[felixfontein]

I'm afraid this wlil be a wontfix, since this is a problem of docker-compose itself:
https://github.com/docker/compose/blob/v1/compose/cli/docker_client.py#L124

So basically docker-compose is incompatible with Docker SDK for Python >= 7. Since there will be no further release of docker-compose v1 (according to Docker Inc.), there's no way to fix this (except by forking docker-compose and releasing it under a new name, with all the problems that brings...).

The only thing we can do is update the documentation of the module to point this incompatibility out.

[marcsc13]

I am in contact with the maintainers of the docker-py project in this issue docker/docker-py#3194.
I identified a change in the kwargs_from_env function signature described in the linked issue.

[felixfontein]

Now that there's a docker_compose_v2 module, I would suggest stop using Docker Compose v1 and switch to Docker Compose v2. That one has a lot less problems, also since the module doesn't have any Python requirements (except Python 2.7 or Python 3.6+ itself :) ).

[christian-heusel]

Now that there's a docker_compose_v2 module

Where can I find this? 😊

[felixfontein]

It's part of community.docker 3.6.0 and 3.7.0. If you're using the ansible PyPI package, you have to wait a few more days for the 9.2.0 release, which probably happens on Tuesday.

[pgassmann]

ansible 9.2.0 was released which contains the docker_compose_v2 and docker_compose_v2_pull modules