Skip to content

Commit

Permalink
Merge branch 'master' into feature/overlay-middleware
Browse files Browse the repository at this point in the history
  • Loading branch information
SwitchTV-BenBettridge authored Jan 19, 2023
2 parents e01852f + 65725c4 commit fee1343
Show file tree
Hide file tree
Showing 37 changed files with 3,812 additions and 253 deletions.
11 changes: 11 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

version: 2
updates:
- package-ecosystem: "maven" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "monthly"
181 changes: 181 additions & 0 deletions .github/workflows/ci-main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
name: Main CI
on:
push:
branches:
- "master"
- "develop"
pull_request:
branches:
- "*"
jobs:
meta:
runs-on: ubuntu-latest
outputs:
dockerhub-publish: ${{ steps.dockerhub-publish.outputs.defined }}
registry: ghcr.io/${{ github.repository }}/container:${{ fromJSON(steps.docker_action_meta.outputs.json).labels['org.opencontainers.image.version'] }}
container_tags: ${{ steps.docker_action_meta.outputs.tags }}
container_labels: ${{ steps.docker_action_meta.outputs.labels }}
container_buildtime: ${{ fromJSON(steps.docker_action_meta.outputs.json).labels['org.opencontainers.image.created'] }}
container_version: ${{ fromJSON(steps.docker_action_meta.outputs.json).labels['org.opencontainers.image.version'] }}
container_revision: ${{ fromJSON(steps.docker_action_meta.outputs.json).labels['org.opencontainers.image.revision'] }}
container_base: ${{ fromJSON(steps.docker_action_meta.outputs.json).tags[0] }}
new_release_version: ${{ steps.version.outputs.new_release_version }}
steps:
- name: Checkout
uses: actions/checkout@v3
with:
submodules: false
persist-credentials: false
- id: dockerhub-publish
if: "${{ env.MY_KEY != '' }}"
run: echo "::set-output name=defined::true"
env:
MY_KEY: ${{ secrets.DOCKER_PASS }}
- uses: actions/setup-node@v2
with:
node-version: "14"
- name: Docker meta
id: docker_action_meta
uses: docker/[email protected]
with:
images: ghcr.io/${{ github.repository }}/container
flavor: |
latest=false
tags: |
type=sha,format=long
type=sha
type=semver,pattern={{version}},value=${{ steps.version.outputs.new_release_version }}
type=semver,pattern={{major}},value=${{ steps.version.outputs.new_release_version }}
type=semver,pattern={{major}}.{{minor}},value=${{ steps.version.outputs.new_release_version }}
type=ref,event=branch
type=ref,event=pr
type=ref,event=tag
labels: |
org.opencontainers.image.licenses=Apache-2.0
runTests:
runs-on: ubuntu-latest
needs: [meta]
steps:
- uses: actions/checkout@v2
with:
submodules: "recursive"

# These steps are quick and will work or if fail only because of external issues
- uses: actions/setup-java@v2
with:
distribution: "temurin"
java-version: "11"
cache: "maven"
- uses: actions/setup-python@v4
with:
python-version: "3.8"
cache: "pip"

#Run tests
- name: Maven Set version
run: |
mvn versions:set -DnewVersion=${{ needs.meta.outputs.new_release_version }}
- name: Maven Package
run: |
mvn package -DskipTests
- name: Maven Test
run: |
mvn test
- name: Other Test
run: |
./src/test/resources/run-s3-tests.sh
#Store the target
- uses: actions/upload-artifact@v2
with:
name: s3proxy
path: target/s3proxy
- uses: actions/upload-artifact@v2
with:
name: pom
path: pom.xml

Containerize:
runs-on: ubuntu-latest
needs: [runTests, meta]
steps:
#Yes we need code
- uses: actions/checkout@v2
- uses: actions/download-artifact@v2
with:
name: s3proxy
path: target
- uses: actions/download-artifact@v2
with:
name: pom
path: .
# These steps are quick and will work or if fail only because of external issues
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Login to DockerHub
uses: docker/[email protected]
if: github.event_name != 'pull_request' && needs.meta.outputs.dockerhub-publish == 'true'
with:
username: ${{ secrets.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}

- name: Login to DockerHub
uses: docker/[email protected]
if: github.event_name != 'pull_request'
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

#Generate Meta
- name: Build and push
uses: docker/build-push-action@v3
with:
context: .
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ needs.meta.outputs.container_base }}
labels: ${{ needs.meta.outputs.container_labels }}
build-args: |
BUILDTIME=${{ needs.meta.outputs.container_buildtime }}
VERSION=${{ needs.meta.outputs.container_version }}
REVISION=${{ needs.meta.outputs.container_revision }}
cache-from: type=registry,ref=${{ needs.meta.outputs.container_base }}
cache-to: type=inline
- uses: actions/setup-node@v2
with:
node-version: "14"
- name: Setup regctl
if: github.event_name != 'pull_request' && needs.meta.outputs.dockerhub-publish == 'true'
run: |
curl -L https://github.com/regclient/regclient/releases/download/v0.3.5/regctl-linux-amd64 >/tmp/regctl
chmod 755 /tmp/regctl
- name: Docker meta
if: github.event_name != 'pull_request' && needs.meta.outputs.dockerhub-publish == 'true'
id: docker_action_meta
uses: docker/[email protected]
with:
images: andrewgaul/s3proxy
flavor: |
latest=false
tags: |
type=sha
type=semver,pattern={{version}},value=${{ needs.meta.outputs.new_release_version }}
type=semver,pattern={{major}},value=${{ needs.meta.outputs.new_release_version }}
type=semver,pattern={{major}}.{{minor}},value=${{ needs.meta.outputs.new_release_version }}
type=ref,event=branch
type=ref,event=pr
type=ref,event=tag
labels: |
org.opencontainers.image.licenses=Apache-2.0
- name: Publish to Docker
if: github.event_name != 'pull_request' && needs.meta.outputs.dockerhub-publish == 'true'
run: |
for line in $CONTAINER_DEST_TAGS; do echo working on "$line"; /tmp/regctl image copy $SOURCE_CONTAINER $line; done
env:
SOURCE_CONTAINER: ${{ needs.meta.outputs.new_release_version }}
CONTAINER_DEST_TAGS: ${{ steps.docker_action_meta.outputs.tags }}
54 changes: 0 additions & 54 deletions .github/workflows/run-tests.yml

This file was deleted.

29 changes: 29 additions & 0 deletions .releaserc
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"tagFormat": 's3proxy-${version}',
"branches": [
{
"name": 'master',
prerelease: false
},
{
"name": 'releases\/+([0-9])?(\.\d+)(\.\d+|z|$)',
prerelease: false
},
{
"name": 'next',
prerelease: false
},
{
name: 'next-major',
prerelease: true
},
{
name: 'develop',
prerelease: true
},
{
name: 'develop\/.*',
prerelease: true
}
]
}
8 changes: 5 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,8 @@ LABEL maintainer="Andrew Gaul <[email protected]>"
WORKDIR /opt/s3proxy

COPY \
--from=s3proxy-builder \
/opt/s3proxy/target/s3proxy \
/opt/s3proxy/src/main/resources/run-docker-container.sh \
target/s3proxy \
src/main/resources/run-docker-container.sh \
/opt/s3proxy/

ENV \
Expand All @@ -33,6 +32,9 @@ ENV \
S3PROXY_OVERLAY_BLOBSTORE="false" \
S3PROXY_OVERLAY_BLOBSTORE_MASK_SUFFIX="__deleted" \
S3PROXY_OVERLAY_BLOBSTORE_PATH="/tmp" \
S3PROXY_ENCRYPTED_BLOBSTORE="" \
S3PROXY_ENCRYPTED_BLOBSTORE_PASSWORD="" \
S3PROXY_ENCRYPTED_BLOBSTORE_SALT="" \
JCLOUDS_PROVIDER="filesystem" \
JCLOUDS_ENDPOINT="" \
JCLOUDS_REGION="" \
Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ and has instructions on how to run it.

Users can [download releases](https://github.com/gaul/s3proxy/releases)
from GitHub. Developers can build the project by running `mvn package` which
produces a binary at `target/s3proxy`. S3Proxy requires Java 8 or newer to run.
produces a binary at `target/s3proxy`. S3Proxy requires Java 11 or newer to
run.

Configure S3Proxy via a properties file. An example using the local
file system as the storage backend with anonymous access:
Expand Down Expand Up @@ -155,12 +156,12 @@ for specific storage backends.

* [Apache jclouds](https://jclouds.apache.org/) provides storage backend support for S3Proxy
* [Ceph s3-tests](https://github.com/ceph/s3-tests) help maintain and improve compatibility with the S3 API
* [fake-s3](https://github.com/jubos/fake-s3), [gofakes3](https://github.com/johannesboyne/gofakes3), [S3 ninja](https://github.com/scireum/s3ninja), and [s3rver](https://github.com/jamhall/s3rver) provide functionality similar to S3Proxy when using the filesystem backend
* [fake-s3](https://github.com/jubos/fake-s3), [gofakes3](https://github.com/johannesboyne/gofakes3), [minio](https://github.com/minio/minio), [S3 ninja](https://github.com/scireum/s3ninja), and [s3rver](https://github.com/jamhall/s3rver) provide functionality similar to S3Proxy when using the filesystem backend
* [GlacierProxy](https://github.com/bouncestorage/glacier-proxy) and [SwiftProxy](https://github.com/bouncestorage/swiftproxy) provide similar functionality for the Amazon Glacier and OpenStack Swift APIs
* [minio](https://github.com/minio/minio) and [Zenko](https://www.zenko.io/) provide similar multi-cloud functionality
* [s3mock](https://github.com/findify/s3mock) mocks the S3 API for Java/Scala projects
* [sbt-s3](https://github.com/localytics/sbt-s3) runs S3Proxy via the Scala Build Tool
* [swift3](https://github.com/openstack/swift3) provides an S3 middleware for OpenStack Swift
* [Zenko](https://www.zenko.io/) provide similar multi-cloud functionality

## License

Expand Down
Loading

0 comments on commit fee1343

Please sign in to comment.