adds SeqFu #1047
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
|
|
||
| ##### ------------------------------------------------------------------------------------------------ ##### | ||
| ##### This caller workflow tests, builds, and pushes the image to Docker Hub and Quay using the most ##### | ||
| ##### recent version of Freyja and downloading the most recent variant information. ##### | ||
| ##### It takes no manual input. ##### | ||
| ##### ------------------------------------------------------------------------------------------------ ##### | ||
|
|
||
| name: Update Freyja | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| schedule: | ||
| - cron: '30 7 * * *' | ||
|
|
||
| run-name: Updating Freyja | ||
|
|
||
| jobs: | ||
| update: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
|
|
||
| # Keeping here in case we want to use it to keep Freyja's version up-to-date as well | ||
| # - name: Get latest release | ||
| # uses: rez0n/actions-github-release@main | ||
| # id: latest_release | ||
| # with: | ||
| # token: ${{ secrets.GITHUB_TOKEN }} | ||
| # repository: andersen-lab/Freyja | ||
| # type: "stable" | ||
| # - name: Remove V | ||
| # id: strip | ||
| # run: | | ||
| # version=$(echo "${{ steps.latest_release.outputs.release }}" | sed 's/V//g' | sed 's/v//g' ) | ||
| # echo "The version is $version" | ||
| # echo "version=$version" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: pull repo | ||
| uses: actions/checkout@v3 | ||
|
|
||
| # Keeping here in case we want to figure out why this doesn't work | ||
| # - name: get latest version of freyja in docker-builds repo | ||
| # id: latest_version | ||
| # run: | | ||
| # file=$(git log -1 --name-only --format=%cd --date=iso freyja/*/Dockerfile | grep Dockerfile | head -n 1 ) | ||
| # echo "the latest file is $file" | ||
| # echo "file=$file" >> $GITHUB_OUTPUT | ||
| # | ||
| # version=$(echo $file | cut -f 2 -d "/" | cut -f 1 -d "_") | ||
| # echo "the latest version is $version" | ||
| # echo "version=$version" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: set freyja version | ||
| id: latest_version | ||
| run: | | ||
| version=1.4.8 | ||
| echo "version=$version" >> $GITHUB_OUTPUT | ||
|
|
||
| file=freyja/$version/Dockerfile | ||
| ls $file | ||
| echo "file=$file" >> $GITHUB_OUTPUT | ||
| - name: set up docker buildx | ||
| id: buildx | ||
| uses: docker/setup-buildx-action@v2 | ||
|
|
||
| - name: cache docker layers | ||
| uses: actions/cache@v3 | ||
| with: | ||
| path: /tmp/.buildx-cache-freyja | ||
| key: ${{ runner.os }}-buildx-freyja-${{ github.sha }} | ||
| restore-keys: ${{ runner.os }}-buildx-freyja | ||
|
|
||
| - name: build to test | ||
| id: docker_build_to_test | ||
| uses: docker/build-push-action@v3 | ||
| with: | ||
| file: ${{ steps.latest_version.outputs.file }} | ||
| target: test | ||
| load: true | ||
| push: false | ||
| cache-from: type=local,src=/tmp/.buildx-cache-freyja | ||
| cache-to: type=local,mode=max,dest=/tmp/.buildx-cache-freyja-new | ||
| tags: freyja:update | ||
|
|
||
| - name: get freyja database version | ||
| id: db_version | ||
| run: | | ||
| docker run freyja:update freyja demix --version | ||
| version=$(docker run freyja:update freyja demix --version | grep . | grep -v Barcode | head -n 1) | ||
| echo "the latest version is $version" | ||
| echo "version=$version" >> $GITHUB_OUTPUT | ||
| - name: Get current date | ||
| id: date | ||
| run: | | ||
| date=$(date '+%Y-%m-%d') | ||
| echo "the date is $date" | ||
| echo "date=$date" >> $GITHUB_OUTPUT | ||
|
|
||
| # - name: Login to DockerHub | ||
| # uses: docker/login-action@v2 | ||
| # with: | ||
| # username: ${{ secrets.DOCKER_HUB_USERNAME }} | ||
| # password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} | ||
|
|
||
| - name: Login to Quay | ||
| uses: docker/login-action@v2 | ||
| with: | ||
| registry: quay.io | ||
| username: ${{ secrets.QUAY_USERNAME }} | ||
| password: ${{ secrets.QUAY_ROBOT_TOKEN }} | ||
|
|
||
| - name: Build and push user-defined tag to DockerHub | ||
| id: docker_build_user_defined_tag | ||
| uses: docker/build-push-action@v3 | ||
| with: | ||
| file: ${{ steps.latest_version.outputs.file }} | ||
| target: app | ||
| push: true | ||
| cache-from: type=local,src=/tmp/.buildx-cache-freyja | ||
| cache-to: type=local,mode=max,dest=/tmp/.buildx-cache-freyja-new # mode=max means export layers from all stage to cache | ||
| tags: staphb/freyja:${{ steps.latest_version.outputs.version }}-${{ steps.db_version.outputs.version }}-${{ steps.date.outputs.date }} | ||
|
|
||
| - name: Build and push to Quay | ||
| id: build | ||
| uses: docker/build-push-action@v3 | ||
| with: | ||
| file: ${{ steps.latest_version.outputs.file }} | ||
| target: app | ||
| push: true | ||
| cache-from: type=local,src=/tmp/.buildx-cache-freyja | ||
| cache-to: type=local,mode=max,dest=/tmp/.buildx-cache-freyja-new # mode=max means export | ||
| tags: quay.io/staphb/freyja:${{ steps.latest_version.outputs.version }}-${{ steps.db_version.outputs.version }}-${{ steps.date.outputs.date }} | ||
|
|
||
| - name: Move cache # apparently prevents the cache from growing in size forever | ||
| run: | | ||
| rm -rf /tmp/.buildx-cache-freyja | ||
| mv /tmp/.buildx-cache-freyja-new /tmp/.buildx-cache-freyja | ||
| - name: Image digest | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| # Set global variables | ||
| ARG SEQFU_VER="1.20.3" | ||
|
|
||
| # Stage 1: Build Dockerfile | ||
| FROM ubuntu:focal AS builder | ||
| ARG SEQFU_VER | ||
|
|
||
| # Metadata | ||
| LABEL base.image="ubuntu:focal" \ | ||
| dockerfile.version="1" \ | ||
| software="SeqFu" \ | ||
| software.version="${SEQFU_VER}" \ | ||
| description="A general-purpose program to manipulate and parse information from FASTA/FASTQ files, supporting gzipped input files." \ | ||
| website="https://github.com/telatin/seqfu2" \ | ||
| license.url="https://github.com/telatin/seqfu2?tab=GPL-3.0-1-ov-file#readme" \ | ||
| maintainer="Taylor K. Paisie" \ | ||
| maintainer.email="[email protected]" | ||
|
|
||
| # Set non-interactive frontend and update PATH | ||
| ENV DEBIAN_FRONTEND=noninteractive \ | ||
| PATH="/root/.nimble/bin:${PATH}" | ||
|
|
||
| # Install required dependencies | ||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| wget \ | ||
| unzip \ | ||
| build-essential \ | ||
| zlib1g-dev \ | ||
| curl \ | ||
| python3 \ | ||
| python3-pip \ | ||
| git \ | ||
| ca-certificates && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Set the working directory | ||
| WORKDIR /root | ||
|
|
||
| # Install Nim and SeqFu | ||
| RUN curl https://nim-lang.org/choosenim/init.sh -sSf | sh -s -- -y && \ | ||
| wget https://github.com/telatin/seqfu2/archive/refs/tags/v${SEQFU_VER}.tar.gz && \ | ||
| tar -xzf v${SEQFU_VER}.tar.gz && \ | ||
| rm v${SEQFU_VER}.tar.gz && \ | ||
| cd seqfu2-${SEQFU_VER} && \ | ||
| make && \ | ||
| make test VERBOSE=1 || (echo "Tests failed, but continuing build"; exit 0) | ||
|
There was a problem hiding this comment. Do you know why the tests fail? There was a problem hiding this comment. oh oops, i can take that out, just part of my debugging of the dockerfile, doesn't fail in the container now. |
||
|
|
||
| # Copy the built binary to a location in the PATH | ||
| RUN ls -l /root/seqfu2-${SEQFU_VER}/bin && \ | ||
| cp -R /root/seqfu2-${SEQFU_VER}/bin/* /usr/local/bin/ && \ | ||
| chmod +x /usr/local/bin/seqfu | ||
|
|
||
| # Stage 2: Create the final image | ||
| FROM ubuntu:focal AS app | ||
| ARG SEQFU_VER | ||
|
|
||
| # Copy the necessary files from the builder | ||
| COPY --from=builder /usr/local/bin /usr/local/bin | ||
| COPY --from=builder /root/seqfu2-${SEQFU_VER}/data/tests /root/seqfu2-${SEQFU_VER}/data/tests | ||
|
There was a problem hiding this comment. Do you think it's worthwhile to include the test files in the final image? It might be helpful to just have these copied into the test stage for testing if they aren't needed during run time. |
||
|
|
||
| # Install minimal runtime dependencies | ||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| curl \ | ||
| wget && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Set working directory | ||
| WORKDIR /data | ||
|
|
||
| # Verify installation | ||
| RUN seqfu --version && seqfu --help | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # SeqFu | ||
|
|
||
| Main tool: [SeqFu](https://github.com/telatin/seqfu2) | ||
|
|
||
| Code repository: https://github.com/telatin/seqfu2 | ||
|
|
||
| Basic information on how to use this tool: | ||
| - executable: | | ||
| ``` | ||
| · count [cnt] : count FASTA/FASTQ reads, pair-end aware | ||
| · deinterleave [dei] : deinterleave FASTQ | ||
| · derep [der] : feature-rich dereplication of FASTA/FASTQ files | ||
| · interleave [ilv] : interleave FASTQ pair ends | ||
| · lanes [mrl] : merge Illumina lanes | ||
| · list [lst] : print sequences from a list of names | ||
| · metadata [met] : print a table of FASTQ reads (mapping files) | ||
| · sort [srt] : sort sequences by size (uniques) | ||
| · stats [st] : statistics on sequence lengths | ||
|
|
||
| · cat : concatenate FASTA/FASTQ files | ||
| · grep : select sequences with patterns | ||
| · head : print first sequences | ||
| · rc : reverse complement strings or files | ||
| · tab : tabulate reads to TSV (and viceversa) | ||
| · tail : view last sequences | ||
| · view : view sequences with colored quality and oligo matches | ||
| ``` | ||
|
|
||
| - help: `seqfu --help` | ||
| - version: `seqfu --version` | ||
| - description: | | ||
| > A general-purpose program to manipulate and parse information from FASTA/FASTQ files, supporting gzipped input files. | ||
|
|
||
|
|
||
| Full documentation: https://telatin.github.io/seqfu2/ | ||
|
|
||
|
|
||
| # Testing SeqFU analysis | ||
| ``` | ||
| seqfu check \ | ||
| --deep \ | ||
| --verbose \ | ||
| /root/seqfu2-1.20.3/data/tests/sample1_R1.fq \ | ||
| /root/seqfu2-1.20.3/data/tests/sample1_R2.fq | ||
| ``` | ||
|
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think file was included by accident