Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More controlled environment in the build #52

Merged
merged 3 commits into from
Apr 12, 2024

Conversation

andreaTP
Copy link
Contributor

Improvements to make the build self-contained.

Description

Various minor improvements to make sure that we are using the same versions of the tools and more.

How Has This Been Tested?

Building locally on Mac and building the container image with podman

Merge criteria:

  • The commits and have meaningful messages; the author will squash them after approval or will ask to merge with squash.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

@andreaTP andreaTP force-pushed the build-improvements branch 4 times, most recently from 1e34226 to fb574a0 Compare March 26, 2024 16:42
@andreaTP
Copy link
Contributor Author

Thanks to the help of @tarilabs we have verified that with those changes the build and the docker-compose-local.yaml are working as expected.

Copy link
Member

@tarilabs tarilabs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I like about this PR:

  • parametrize in bin/ the protoc toolchain, this was an issue with other contributors and seems this improves
  • parametrize in bin/ the Go toolchain, as above
  • avoid network mgt in docker-compose-local.yml

Things I haven't been able to confirm we need:

  • buildvcs flag
  • network: host settings/changes

per comments below.

Will add docker-compose.yml changes which I found beneficial when working with @andreaTP

Makefile Outdated
@@ -204,7 +212,7 @@ endif
# build docker image
.PHONY: image/build
image/build:
${DOCKER} build . -f ${DOCKERFILE} -t ${IMG}:$(IMG_VERSION)
${DOCKER} build . -f ${DOCKERFILE} -t ${IMG}:$(IMG_VERSION) --network host
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tbh I'm not sure we need this change 🤔 , further it does not work with Podman desktop on my end:
image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On podman Desktop on my Mac I get this error:

No routable interface for IPv6: IPv6 is disabled
Couldn't open network namespace /proc/4047/ns/net: Permission denied

@@ -13,8 +16,13 @@ services:
build:
context: .
dockerfile: Dockerfile
command: ["proxy", "--mlmd-hostname", "localhost", "--mlmd-port", "9090"]
network: host
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we need this network: host 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's used only during the build of the container and it was needed on my machine with podman to successfully build the image.

@@ -13,8 +16,13 @@ services:
build:
context: .
dockerfile: Dockerfile
command: ["proxy", "--mlmd-hostname", "localhost", "--mlmd-port", "9090"]
network: host
command: ["proxy", "--hostname", "0.0.0.0", "--mlmd-hostname", "mlmd-server", "--mlmd-port", "8080"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confirming "--hostname", "0.0.0.0", is needed 😅

Comment on lines +166 to +170
${GO} build -buildvcs=false

.PHONY: build/odh
build/odh: vet
go build
${GO} build -buildvcs=false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was -buildvcs=false needed only for the build-in-docker? 🤔 Because on my end the make build works without issue as-is, without needing -buildvcs=false

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it fails in CI without this flag ...

Copy link
Contributor

@dhirajsb dhirajsb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
I'm just not sure what's going on with the whole docker network thing. Is that necessary or not? Also, it's going to affect the DB as it'll have to be on the same network.

Copy link

@dhirajsb: changing LGTM is restricted to collaborators

In response to this:

/lgtm
I'm just not sure what's going on with the whole docker network thing. Is that necessary or not? Also, it's going to affect the DB as it'll have to be on the same network.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@andreaTP
Copy link
Contributor Author

andreaTP commented Apr 3, 2024

I'm just not sure what's going on with the whole docker network thing.

Using host network is not a "best practice" and with those changes, we are improving it.

Also, it's going to affect the DB as it'll have to be on the same network.

The DB should be exposed on the docker compose network too, should we modify any additional files?

@dhirajsb
Copy link
Contributor

dhirajsb commented Apr 3, 2024

@andreaTP looks like the default store config file is using an in-memory SQLite DB, so the network is not an issue.

Copy link
Contributor

@dhirajsb dhirajsb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Copy link

@dhirajsb: changing LGTM is restricted to collaborators

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tarilabs tarilabs mentioned this pull request Apr 11, 2024
3 tasks
@tarilabs
Copy link
Member

With e50cd37 I've added support for protoc in ARM and avoiding the use of network-host as per comment.

Tested with:

  1. make build (the build succeeds):
    Screenshot 2024-04-11 at 13 59 29

  2. docker-compose-local (the robotframework test passes):
    Screenshot 2024-04-11 at 14 02 57

  3. docker-compose (the robotframework test passes):
    Screenshot 2024-04-11 at 14 05 04

based on OP and review, will eventually merge this one.

@tarilabs
Copy link
Member

Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreaTP, dhirajsb, lampajr, tarilabs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit 8745364 into kubeflow:main Apr 12, 2024
10 checks passed
rkubis added a commit to rkubis/model-registry that referenced this pull request Apr 23, 2024
* Update(openshift-ci/scripts): Increase API version of Model Registry

* Update(test/scripts): Increase API version of Model Registry
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants