Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rhel9 binaries #4086

Merged
merged 21 commits into from
Jul 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build-staging-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
latest=false
tags: |
type=sha
type=branch,event=workflow_dispatch
type=ref,event=branch

- name: Login to registry
uses: docker/login-action@v2
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -172,4 +172,4 @@ RUN windmill cache

EXPOSE 8000

CMD ["windmill"]
CMD ["windmill"]
2 changes: 1 addition & 1 deletion backend/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion backend/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ mysql_async = { version = "*", default-features = false, features = ["minimal",
postgres-native-tls = "^0"
native-tls = "^0"
# samael will break compilation on MacOS. Use this fork instead to make it work
# samael = { git="https://github.com/njaremko/samael", rev="464d015e3ae393e4b5dd00b4d6baa1b617de0dd6", features = ["xmlsec"] }
# samael = { git="https://github.com/njaremko/samael", rev="464d015e3ae393e4b5dd00b4d6baa1b617de0dd6", features = ["xmlsec"] }
samael = { version="0.0.14", features = ["xmlsec"] }
gcp_auth = "0.9.0"
rust_decimal = { version = "^1", features = ["db-postgres"]}
Expand Down
77 changes: 77 additions & 0 deletions docker/RHEL9/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
ARG DEBIAN_IMAGE=debian:bookworm-slim
ARG RUST_IMAGE=registry.access.redhat.com/ubi9/ubi:latest
ARG PYTHON_IMAGE=python:3.11.8-slim-bookworm

FROM ${RUST_IMAGE} AS rust_base

RUN yum install -y rust-toolset

RUN yum update -y && \
yum install -y git openssl-devel npm nodejs rustfmt

RUN CARGO_NET_GIT_FETCH_WITH_CLI=true cargo install cargo-chef --version ^0.1

WORKDIR /windmill

ENV SQLX_OFFLINE=true
# ENV CARGO_INCREMENTAL=1

FROM node:20-alpine as frontend

# install dependencies
WORKDIR /frontend
COPY ./frontend/package.json ./frontend/package-lock.json ./
RUN npm ci

# Copy all local files into the image.
COPY frontend .
RUN mkdir /backend
COPY /backend/windmill-api/openapi.yaml /backend/windmill-api/openapi.yaml
COPY /openflow.openapi.yaml /openflow.openapi.yaml
COPY /backend/windmill-api/build_openapi.sh /backend/windmill-api/build_openapi.sh

RUN cd /backend/windmill-api && . ./build_openapi.sh
COPY /backend/parsers/windmill-parser-wasm/pkg/ /backend/parsers/windmill-parser-wasm/pkg/
COPY /typescript-client/docs/ /frontend/static/tsdocs/

RUN npm run generate-backend-client
ENV NODE_OPTIONS "--max-old-space-size=8192"
RUN npm run build


FROM rust_base AS planner

COPY ./openflow.openapi.yaml /openflow.openapi.yaml
COPY ./backend ./

RUN --mount=type=cache,target=/usr/local/cargo/registry \
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo chef prepare --recipe-path recipe.json

FROM rust_base AS builder
ARG features=""

COPY --from=planner /windmill/recipe.json recipe.json

RUN --mount=type=secret,id=rh_username \
--mount=type=secret,id=rh_password \
subscription-manager register --username $(cat /run/secrets/rh_username) --password $(cat /run/secrets/rh_password)

RUN subscription-manager repos --enable codeready-builder-for-rhel-9-$(arch)-rpms

RUN yum update -y && \
yum install -y libxml2-devel xmlsec1-devel xmlsec1-openssl-devel clang llvm-devel cmake libtool-ltdl-devel

RUN --mount=type=cache,target=/usr/local/cargo/registry \
CARGO_NET_GIT_FETCH_WITH_CLI=true RUST_BACKTRACE=1 cargo chef cook --release --features "$features" --recipe-path recipe.json

COPY ./openflow.openapi.yaml /openflow.openapi.yaml
COPY ./backend ./

COPY --from=frontend /frontend /frontend
COPY --from=frontend /backend/windmill-api/openapi-deref.yaml ./windmill-api/openapi-deref.yaml
COPY .git/ .git/

RUN --mount=type=cache,target=/usr/local/cargo/registry \
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo build --release --features "$features"

RUN subscription-manager unregister
19 changes: 19 additions & 0 deletions docker/RHEL9/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Red Hat Linux Binaries

This directory contains the Dockerfiles for building Windmill binaries for Red Hat Linux 9.

We build Windmill on the Red Hat Universal Base Image 9. Windmill requires the xmlsec1-devel package which is not available in the default UBI9 repositories. It is however included in the CodeReady Builder for RHEL9 repository which requires a RedHat subscription.
Moreover, only rust v1.75 is supported on Red Hat Linux 9. To make Windmill compatible with Rust v1.75, you need to pin the following libraries:
```
aws-config = "=1.4.0"
aws-sdk-sts = "=1.25.0"
aws-sdk-ssooidc = "=1.25.0"
aws-sdk-sso = "=1.25.0"
```

Make sure to include `aws-sdk-ssooidc` and `aws-sdk-sso` in the Cargo.toml of windmill-common as well to enforce the correct versions of the nested dependencies. Make them optional and include them in the `parquet` feature.
It's also possible that you need to add `#[async_recursion]` to the `lock_modules` function in the `backend/windmill-worker/src/worker_lockfiles.rs` file for it to compile.

Once the image is built, you can simply copy the binary on any Red Hat Linux 9 machine and run it. You will just need to install the xmlsec1 package which can be installed directly using `yum/dnf install xmlsec1`.


Loading