Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adjust Dockerfile & entrypoint and add Dependabot, CI & docker release #51

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions .dockerignore
100755 → 100644
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
__pycache__
.mypy_cache
.pytest_cache
.vscode
cvdupdate.egg-info
/build
/dist
/tests
.github
__pycache__
.git
.github
.mypy_cache
.pytest_cache
.vscode
*cvdupdate.egg-info
/build
/dist
/tests
30 changes: 30 additions & 0 deletions .github/dependabot.yml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dpendabot.yml Seems good to me.

Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
version: 2
updates:
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"
time: "09:00"
timezone: "Europe/Berlin"

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
time: "09:00"
timezone: "Europe/Berlin"

- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
time: "09:00"
timezone: "Europe/Berlin"

- package-ecosystem: "pip"
directory: "/cvdupdate"
schedule:
interval: "weekly"
time: "09:00"
timezone: "Europe/Berlin"
37 changes: 37 additions & 0 deletions .github/release-drafter.yml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the suggestion. It's neat tech. It shifts some work from release engineering over to grooming our PR titles, descriptions and labels... which I should probably do more of anyways.

But I'm not sure I can add Release Drafter to the org/repo. I'm not keen on trying to do this for a project that we update so infrequently. If we do go for it, then I'd want to add it to all of our projects in a consistent way. I'm not ready to commit to that.

At least for now, please remove this from this PR.

Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name-template: '$RESOLVED_VERSION'
tag-template: '$RESOLVED_VERSION'
categories:
- title: '🚀 Features'
labels:
- 'feature'
- 'enhancement'
- title: '🐛 Bug Fixes'
labels:
- 'fix'
- 'bugfix'
- 'bug'
- title: '🧹 Maintenance'
labels:
- 'chore'
- 'dependencies'
version-resolver:
major:
labels:
- 'feature'
minor:
labels:
- 'enhancement'
patch:
labels:
- 'fix'
- 'bugfix'
- 'bug'
- 'chore'
- 'dependencies'
default: patch
template: |
## Changes

$CHANGES

**Full Changelog**: https://github.com/$OWNER/$REPOSITORY/compare/$PREVIOUS_TAG...$RESOLVED_VERSION
42 changes: 42 additions & 0 deletions .github/workflows/ci.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice. It would be very good to have CI.

Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: ci

on:
pull_request:

jobs:
docker-build:
runs-on: ubuntu-22.04
steps:
- name: Checkout Code
uses: actions/checkout@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Docker metadata action
id: meta
uses: docker/metadata-action@v4
with:
images:
cvdupdate-local
tags: |
type=raw,latest

- name: Build Dockerimage
id: docker_build
uses: docker/build-push-action@v4
with:
context: .
file: ./Dockerfile
labels: ${{ steps.meta.outputs.labels }}
outputs: type=docker,dest=/tmp/cvdupdate-local.tar
platforms: linux/amd64
push: false
tags: ${{ steps.meta.outputs.tags }}

- name: Run Dockerimage
run: |
docker load --input /tmp/cvdupdate-local.tar
docker run -d --net=host cvdupdate-local serve
sleep 30
curl --fail --silent --output /dev/null http://localhost:8000/main.cvd
57 changes: 57 additions & 0 deletions .github/workflows/docker-release.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our existing CI to build/publish docker images is done from an internal Jenkins and publishes to the Docker Hub registry.

I have mixed feelings about publishing a cvdupdate container, let alone publishing to a different registry.

Then again, this appears to be really simple, and easy to maintain. I have very mixed feelings.

Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: docker-release

on:
push:
branches:
- main
tags:
- '*'
schedule:
- cron: '0 0 * * *'

jobs:
docker-build-push:
runs-on: ubuntu-22.04
steps:
- name: Checkout Code
uses: actions/checkout@v3

- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.repository_owner }}

- name: Docker metadata action
id: meta
uses: docker/metadata-action@v4
with:
images: |
ghcr.io/${{ github.repository_owner }}/cvdupdate
tags: |
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
type=raw,value={{date 'YYYYMMDD-HHmmss' tz='Europe/Berlin'}},enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
type=ref,event=tag
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}

- name: Build and push
id: docker_build
uses: docker/build-push-action@v4
with:
context: .
file: ./Dockerfile
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}

- name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
2 changes: 1 addition & 1 deletion .github/workflows/pypi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
- uses: actions/checkout@master

- name: Set up Python 3.7
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: 3.7

Expand Down
22 changes: 22 additions & 0 deletions .github/workflows/release-drafter.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned above, I am not ready to commit to Release Drafter for all our projects, and am not interested in adopting it just for cvdupdate, which we rarely update.

Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Release Drafter

on:
push:
branches:
- main
pull_request:
types: [opened, reopened, synchronize]

permissions:
contents: read

jobs:
update_release_draft:
permissions:
contents: write
pull-requests: write
runs-on: ubuntu-22.04
steps:
- uses: release-drafter/release-drafter@v5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Empty file modified .gitignore
100755 → 100644
Empty file.
31 changes: 24 additions & 7 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,24 @@
FROM python:3-slim
RUN apt-get -y update \
&& apt-get -y --no-install-recommends install cron gosu \
&& rm -rf /var/lib/apt/lists/*
COPY . /dist
RUN pip install --no-cache-dir /dist
ENTRYPOINT [ "/dist/scripts/docker-entrypoint.sh" ]
FROM python:3.12.0b1-slim
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am curious to know why you suggested this change. I believe it is better with python:3-slim. Using a fine-grain image tag will be more work for maintenance, or else will leave people stuck on an unmaintained tag.


WORKDIR /cvdupdate

RUN apt-get -y update && \
apt-get -y --no-install-recommends install cron sudo && \
apt-get -y clean && \
rm -rf /var/lib/apt/lists/* && \
useradd --no-create-home --home-dir /cvdupdate --uid 1000 cvdupdate && \
echo '30 */4 * * * /usr/local/bin/cvdupdate update > /proc/1/fd/1 2>&1' >> /etc/cron.d/cvdupdate && \
echo '@reboot /usr/local/bin/cvdupdate update >/proc/1/fd/1 2>/proc/1/fd/2' >> /etc/cron.d/cvdupdate && \
crontab -u cvdupdate /etc/cron.d/cvdupdate && \
echo "cvdupdate\tALL=(ALL:ALL) NOPASSWD: /usr/sbin/cron" >> /etc/sudoers

COPY . .

RUN pip install --no-cache-dir . && \
chown cvdupdate:cvdupdate -R /cvdupdate

USER cvdupdate:cvdupdate

RUN cvd update

ENTRYPOINT [ "./scripts/docker-entrypoint.sh" ]
28 changes: 3 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -295,34 +295,12 @@ Run image, that will automaticly update databases in folder `/srv/cvdupdate` and

```bash
docker run -d \
-v /srv/cvdupdate:/cvdupdate/database \
-v /var/log/cvdupdate:/cvdupdate/logs \
-v /srv/cvdupdate:/cvdupdate/.cvdupdate/database \
-v /var/log/cvdupdate:/cvdupdate/.cvdupdate/logs \
cvdupdate:latest
```

Run image, that will automaticly update databases in folder `/srv/cvdupdate`, write logs to `/var/log/cvdupdate` and set owner of files to user with ID 1000

```bash
docker run -d \
-v /srv/cvdupdate:/cvdupdate/database \
-v /var/log/cvdupdate:/cvdupdate/logs \
-e USER_ID=1000 \
cvdupdate:latest
```

Default update interval is `30 */4 * * *` (see [Cron Example](#cron-example))

You may pass custom update interval in environment variable `CRON`

For example - update every day in 00:00

```bash
docker run -d \
-v /srv/cvdupdate:/cvdupdate/database \
-v /var/log/cvdupdate:/cvdupdate/logs \
-e CRON='0 0 * * *' \
cvdupdate:latest
```
Update interval is `30 */4 * * *` (see [Cron Example](#cron-example))

## Contribute

Expand Down
47 changes: 12 additions & 35 deletions scripts/docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -1,41 +1,18 @@
#!/bin/bash
USER_ID="${USER_ID:-0}"
#
# cvdupdate & cron entrypoint
#

set -e

SCRIPT_PATH=$(readlink -f "$0")
echo "ClamAV Private Database Mirror Updater Cron ${SCRIPT_PATH}"
if [ "${USER_ID}" -ne "0" ]; then
echo "Creating user with ID ${USER_ID}"
useradd --create-home --home-dir /cvdupdate --uid "${USER_ID}" cvdupdate
chown -R "${USER_ID}" /cvdupdate
gosu cvdupdate cvdupdate config set --logdir /cvdupdate/logs
gosu cvdupdate cvdupdate config set --dbdir /cvdupdate/database
else
mkdir -p /cvdupdate/{logs,database}
cvdupdate config set --logdir /cvdupdate/logs
cvdupdate config set --dbdir /cvdupdate/database
fi

if [ $# -eq 0 ]; then
set -e
if [ $# -eq 0 ]; then
echo "ClamAV Private Database Mirror Updater Cron ${SCRIPT_PATH}"

echo "Adding crontab entry"
if [ "${USER_ID}" -ne "0" ]; then
crontab -l | {
cat
echo "${CRON:-"30 */4 * * *"} /usr/sbin/gosu cvdupdate /usr/local/bin/cvdupdate update >/proc/1/fd/1 2>/proc/1/fd/2"
echo "@reboot /usr/sbin/gosu cvdupdate /usr/local/bin/cvdupdate update >/proc/1/fd/1 2>/proc/1/fd/2"
} | crontab -
else
crontab -l | {
cat
echo "${CRON:-"30 */4 * * *"} /usr/local/bin/cvdupdate update >/proc/1/fd/1 2>/proc/1/fd/2"
echo "@reboot /usr/local/bin/cvdupdate update >/proc/1/fd/1 2>/proc/1/fd/2"
} | crontab -
fi
cron -f
sudo cron -f
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong opinion on using gosu versus sudo. But it's the sort of change that someone (at least the person who added gosu to begin with) will be upset about.

It seems to me that sudo or gosu are necessary in order for cvdupdate to modify any user owned mounted directory. Sadly, we can't just strip them out and run without root privs.

I'm not comfortable with my current insight into how this works and sudo vs gosu in order to bless/merge this.

Can someone smarter than me please give me a breakdown on why I should/shouldn't accept the change to use sudo?

else
if [ "${USER_ID}" -ne "0" ]; then
exec gosu cvdupdate "$@"
else
exec "$@"
fi
echo "ClamAV Private Database Mirror Updater "$@" ${SCRIPT_PATH}"

cvdupdate "$@"
fi