Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support different python versions (3.12, 3.13) #1703

Open
gulldan opened this issue Oct 24, 2024 · 5 comments
Open

support different python versions (3.12, 3.13) #1703

gulldan opened this issue Oct 24, 2024 · 5 comments

Comments

@gulldan
Copy link

gulldan commented Oct 24, 2024

Hello, is there any more allegiant solution for using python 3.12, 3.13 than
#1543 (comment) ?

now i do understand nothing about bazel to do it myself.

Maybe you can recommend some step by step documentation how to do it?
readme doesnt help me

thank

@loosebazooka
Copy link
Member

There's are a few suggestions in other issues. But it's not really our mission to provide this.

@daaain
Copy link

daaain commented Nov 11, 2024

Not sure if more elegant, but I ended up copying binaries and libs into Distroless CC from the official Python image, which also serves as local development image.

In my case production is always amd64 anyway, so didn't bother to make it work with arm.

I understand if the Distroless team doesn't want to maintain multiple Python images, but would be good to document which version it is (3.11.2) so people at least know what to expect.

# Development dependencies stage
FROM python:3.12.7-bookworm as deps

SHELL ["/bin/bash", "-o", "pipefail", "-c"]
WORKDIR /projects/app

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

ENTRYPOINT ["/bin/bash"]

# Production stage using CC Distroless
FROM --platform=linux/amd64 gcr.io/distroless/cc-debian12:nonroot AS prod

ARG PYTHON_VERSION=3.12
ENV PYTHONPATH=/usr/local/lib/python${PYTHON_VERSION}/site-packages
ENV PATH="/usr/local/bin:${PATH}"
ENV LD_LIBRARY_PATH=/usr/local/lib:/lib/x86_64-linux-gnu:${LD_LIBRARY_PATH:-}
WORKDIR /projects/app

# Copy Python dependencies from deps stage
COPY --from=deps /usr/local/lib/python${PYTHON_VERSION}/site-packages /usr/local/lib/python${PYTHON_VERSION}/site-packages
# Copy the Python interpreter with a specific name
COPY --from=deps /usr/local/bin/python${PYTHON_VERSION} /usr/local/bin/pythonapp
# Copy Python library files including shared libraries
COPY --from=deps /usr/local/lib/python${PYTHON_VERSION} /usr/local/lib/python${PYTHON_VERSION}
COPY --from=deps /usr/local/lib/libpython${PYTHON_VERSION}.so* /usr/local/lib/
# Copy system libraries for x86_64
COPY --from=deps /lib/x86_64-linux-gnu/lib* /lib/x86_64-linux-gnu/

# Copy application code
COPY . ./

USER nonroot

ENTRYPOINT ["/usr/local/bin/pythonapp"]
CMD ["/projects/app/main.py"]

@gulldan
Copy link
Author

gulldan commented Nov 12, 2024

Not sure if more elegant, but I ended up copying binaries and libs into Distroless CC from the official Python image, which also serves as local development image.

In my case production is always amd64 anyway, so didn't bother to make it work with arm.

I understand if the Distroless team doesn't want to maintain multiple Python images, but would be good to document which version it is (3.11.2) so people at least know what to expect.

# Development dependencies stage
FROM python:3.12.7-bookworm as deps

SHELL ["/bin/bash", "-o", "pipefail", "-c"]
WORKDIR /projects/app

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

ENTRYPOINT ["/bin/bash"]

# Production stage using CC Distroless
FROM --platform=linux/amd64 gcr.io/distroless/cc-debian12:nonroot AS prod

ARG PYTHON_VERSION=3.12
ENV PYTHONPATH=/usr/local/lib/python${PYTHON_VERSION}/site-packages
ENV PATH="/usr/local/bin:${PATH}"
ENV LD_LIBRARY_PATH=/usr/local/lib:/lib/x86_64-linux-gnu:${LD_LIBRARY_PATH:-}
WORKDIR /projects/app

# Copy Python dependencies from deps stage
COPY --from=deps /usr/local/lib/python${PYTHON_VERSION}/site-packages /usr/local/lib/python${PYTHON_VERSION}/site-packages
# Copy the Python interpreter with a specific name
COPY --from=deps /usr/local/bin/python${PYTHON_VERSION} /usr/local/bin/pythonapp
# Copy Python library files including shared libraries
COPY --from=deps /usr/local/lib/python${PYTHON_VERSION} /usr/local/lib/python${PYTHON_VERSION}
COPY --from=deps /usr/local/lib/libpython${PYTHON_VERSION}.so* /usr/local/lib/
# Copy system libraries for x86_64
COPY --from=deps /lib/x86_64-linux-gnu/lib* /lib/x86_64-linux-gnu/

# Copy application code
COPY . ./

USER nonroot

ENTRYPOINT ["/usr/local/bin/pythonapp"]
CMD ["/projects/app/main.py"]

ty also i can find a different way to do same

FROM python:3.13-slim-bookworm AS builder
ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy UV_PYTHON_INSTALL_DIR=/python
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
RUN uv python install 3.13

WORKDIR /app
RUN --mount=type=bind,source=uv.lock,target=uv.lock \
    --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
    uv sync --frozen --no-dev --no-cache --no-editable --no-install-project --python-preference only-managed
ADD . /app

# Then, use a final image without uv
FROM --platform=linux/amd64 gcr.io/distroless/cc-debian12:nonroot as prod
ARG PYTHON_VERSION=3.13
ENV PATH="/usr/local/bin:${PATH}"

# Copy system libraries for x86_64
COPY --from=builder /lib/x86_64-linux-gnu/lib* /lib/x86_64-linux-gnu/
COPY --from=builder --chown=python:python /python /python
# Copy the application from the builder
COPY --from=builder --chown=app:app /app /app

# Place executables in the environment at the front of the path
ENV PATH="/app/.venv/bin:$PATH"

# Run the FastAPI application by default
ENTRYPOINT ["granian", "--interface", "asgi", "/app/main:app", "--port", "8000", "--host", "0.0.0.0"]

@daaain
Copy link

daaain commented Nov 12, 2024

Oh cool, so by installing Python with uv you can get all the binaries and libraries under one directory (UV_PYTHON_INSTALL_DIR) so it's fewer paths to copy? It's probably much less disk space too because you only get stuff that Python will actually use?

Do you even need the Python image for the builder or a C (or even base Debian) one might be enough because you're copying uv from another image anyway?

I just realised that my Dockerfile might need another stage as currently the dev dependencies will get copied into the prod image too 🤔

@gulldan
Copy link
Author

gulldan commented Nov 12, 2024

im not sure will need investigate.

Oh cool, so by installing Python with uv you can get all the binaries and libraries under one directory (UV_PYTHON_INSTALL_DIR) so it's fewer paths to copy? It's probably much less disk space too because you only get stuff that Python will actually use?

Do you even need the Python image for the builder or a C (or even base Debian) one might be enough because you're copying uv from another image anyway?

I just realised that my Dockerfile might need another stage as currently the dev dependencies will get copied into the prod image too 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants
@daaain @loosebazooka @gulldan and others