Skip to content

Commit

Permalink
Set machine-type of the PAPI worker (#567)
Browse files Browse the repository at this point in the history
* Set machine-type for the runner worker

All this worker does is to launch the Dataflow pipeline, monitor the result, and finlay load AVRO files to the destination BQ tables.
Using a smaller machine would help our customers to save on extra cost of running this worker.

* Rollback #471
and use gcloud instead of pipelines tools. This will remove an extra dependency and also make our docker image smaller.
Here we also update our base image to cloud-sdk:286.0.0-slim

* Keep using piplines tools

Decided to not move back to gcloud command in this PR.
  • Loading branch information
samanvp authored Mar 30, 2020
1 parent 89826f3 commit b569e63
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 2 deletions.
5 changes: 3 additions & 2 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ FROM golang:1.11

RUN go get -ldflags '-extldflags "-fno-PIC -static"' -buildmode pie -tags 'osusergo netgo static_build' github.com/googlegenomics/pipelines-tools/pipelines

FROM google/cloud-sdk:276.0.0-slim
FROM google/cloud-sdk:286.0.0-slim

COPY --from=0 /go/bin/pipelines /usr/bin

Expand All @@ -44,7 +44,8 @@ RUN apt-get update && apt-get install -y \
libssl-dev \
make \
perl \
zlib1g-dev
zlib1g-dev \
python-pip

# Install dependencies.
RUN python -m pip install --upgrade pip && \
Expand Down
1 change: 1 addition & 0 deletions docker/pipelines_runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ function main {
--scopes "https://www.googleapis.com/auth/cloud-platform" \
--regions "${region}" \
--image "${vt_docker_image}" \
--machine-type "g1-small" \
--pvm-attempts 0 \
--attempts 1 \
--disk-size 10
Expand Down

0 comments on commit b569e63

Please sign in to comment.