Skip to content
This repository has been archived by the owner on Jul 10, 2024. It is now read-only.

Commit

Permalink
SUBMARINE-1396. Split experiment-prehandler test into separate action
Browse files Browse the repository at this point in the history
### What is this PR for?
The image build of `experiment-prehandler` resulted in a `no space left on device` exception. Now split `experiment-prehandler` build to a standalone script.
Currently, the e2e test already contains `experiment-prehandler` experiment related codes (a new issue has been created to handle unit test https://issues.apache.org/jira/browse/SUBMARINE-1397), so for now remove pyhton test.

### What type of PR is it?
Hot Fix

### Todos
* [x] - Strip experiment-prehandler from python tests

### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-1396

### How should this be tested?
CI test

### Screenshots (if appropriate)
NA

### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? No

Author: cdmikechen <[email protected]>
Author: cdmikechen <[email protected]>

Signed-off-by: cdmikechen <[email protected]>

Closes #1093 from cdmikechen/SUBMARINE-1396 and squashes the following commits:

31f058d [cdmikechen] use sh
9ef4ad0 [cdmikechen] remove space
bc96ee1 [cdmikechen] add new scripts
5f99e47 [cdmikechen]  Remove experiment-prehandler in build image scripts
  • Loading branch information
cdmikechen committed Jul 31, 2023
1 parent 0a554c4 commit 5a5e3a6
Show file tree
Hide file tree
Showing 4 changed files with 32 additions and 6 deletions.
24 changes: 24 additions & 0 deletions .github/scripts/build-image-experiment-prehandler.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/usr/bin/env bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

SUBMARINE_VERSION="0.8.0-SNAPSHOT"
echo "Build Image apache/submarine:experiment-prehandler-${SUBMARINE_VERSION}"
echo "Execute ./dev-support/docker-images/experiment-prehandler/build.sh"
./dev-support/docker-images/experiment-prehandler/build.sh
kind load docker-image apache/submarine:experiment-prehandler-${SUBMARINE_VERSION}
docker rmi apache/submarine:experiment-prehandler-${SUBMARINE_VERSION}
3 changes: 1 addition & 2 deletions .github/scripts/build-image-locally-v3.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,13 @@
#

SUBMARINE_VERSION="0.8.0-SNAPSHOT"
FOLDER_LIST=("database" "mlflow" "submarine" "operator-v3" "agent" "experiment-prehandler")
FOLDER_LIST=("database" "mlflow" "submarine" "operator-v3" "agent")
IMAGE_LIST=(
"apache/submarine:database-${SUBMARINE_VERSION}"
"apache/submarine:mlflow-${SUBMARINE_VERSION}"
"apache/submarine:server-${SUBMARINE_VERSION}"
"apache/submarine:operator-${SUBMARINE_VERSION}"
"apache/submarine:agent-${SUBMARINE_VERSION}"
"apache/submarine:experiment-prehandler-${SUBMARINE_VERSION}"
)

for i in "${!IMAGE_LIST[@]}"
Expand Down
3 changes: 1 addition & 2 deletions .github/scripts/build-image-locally.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,13 @@
#

SUBMARINE_VERSION="0.8.0-SNAPSHOT"
FOLDER_LIST=("database" "mlflow" "submarine" "operator" "agent" "experiment-prehandler")
FOLDER_LIST=("database" "mlflow" "submarine" "operator" "agent")
IMAGE_LIST=(
"apache/submarine:database-${SUBMARINE_VERSION}"
"apache/submarine:mlflow-${SUBMARINE_VERSION}"
"apache/submarine:server-${SUBMARINE_VERSION}"
"apache/submarine:operator-${SUBMARINE_VERSION}"
"apache/submarine:agent-${SUBMARINE_VERSION}"
"apache/submarine:experiment-prehandler-${SUBMARINE_VERSION}"
)

for i in "${!IMAGE_LIST[@]}"
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/master.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,9 @@ jobs:
echo "current-context:" $(kubectl config current-context)
echo "environment-kubeconfig:" ${KUBECONFIG}
- name: Build Image locally
run: .github/scripts/build-image-locally-v3.sh
run: |
.github/scripts/build-image-locally-v3.sh
sh .github/scripts/build-image-experiment-prehandler.sh
- name: Install Golang Dependencies
working-directory: submarine-cloud-v3
run: go mod vendor
Expand Down Expand Up @@ -260,7 +262,9 @@ jobs:
echo ">>> mvn ${BUILD_FLAG} -B"
mvn ${BUILD_FLAG} -B
- name: Build Image locally
run: .github/scripts/build-image-locally-v3.sh
run: |
.github/scripts/build-image-locally-v3.sh
sh .github/scripts/build-image-experiment-prehandler.sh
- name: Start submarine
run: bash ./.github/scripts/start-submarine.sh
- name: Test
Expand Down

0 comments on commit 5a5e3a6

Please sign in to comment.