-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update dev docs #1141
Merged
Merged
Update dev docs #1141
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -6,16 +6,13 @@ and are published to https://docs.sematic.dev/ | |
## Setup | ||
|
||
The developer tools need to be installed by running this command once (and subsequently | ||
whenever `requirements/ci-requirements.txt`) will be updated: | ||
whenever `uv.lock` will be updated): | ||
```bash | ||
$ make install-dev-deps | ||
$ make py-prep | ||
``` | ||
|
||
### Bazel installation | ||
|
||
Be sure to install a compatible version of bazel: `sudo apt update && sudo apt install bazel-6.1.1` | ||
|
||
You may also want to add the following to your .bashrc: `export USE_BAZEL_VERSION=6.1.1` | ||
This will result in a python virtual environment prepared at `./.venv`. You must | ||
activate into that virtual environment with `source .venv/bin/activate` if you | ||
wish to have an interpreter with Sematic and its dependencies installed. | ||
|
||
## Testing | ||
|
||
|
@@ -31,7 +28,7 @@ In order to ensure the PR review goes swiftly, please: | |
|
||
- Add a comprehensive description to your PR | ||
- Ensure your code is properly formatted and type checked | ||
- Make sure you have the dev tools installed by running `make install-dev-deps` (you only ever need to do this once) | ||
- Make sure you have the dev tools installed by running `make py-prep` (you only ever need to do this once) | ||
- Use `make pre-commit` to run the linter and code formatter | ||
- Use `make update-schema` to make sure any DB changes you made are accounted for | ||
- Make sure the CircleCI build passes for your branch (linked in the checks section at the bottom of the GitHub PR page) | ||
|
@@ -63,23 +60,29 @@ When developing, we often need to switch the Server where we submit pipelines, | |
sometimes bundled together with other settings as well. In order to avoid | ||
constantly editing this file manually, we can define one file per environment | ||
or profile, under the names `~/.sematic/settings.yaml.<profile>`, and switch | ||
between them using `bazel run //tools:switch-settings -- <profile>`. | ||
between them using `cp ~/.sematic/settings.yaml.<profile> ~/.sematic/settings.yaml`. | ||
|
||
This actually copies these profile files over the main settings file. Example: | ||
Example: | ||
|
||
```bash | ||
$ bazel run //tools:switch-settings -- prod1 | ||
[...] | ||
Copying previous settings to /Users/tudorscurtu/.sematic/settings.yaml_bck | ||
Copying /Users/tudorscurtu/.sematic/settings.yaml.prod1 to /Users/tudorscurtu/.sematic/settings.yaml | ||
|
||
Successfully switched to prod1! | ||
|
||
$ cp ~/.sematic/settings.yaml ~/.sematic/settings.yaml.prod1 | ||
$ cp ~/.sematic/settings.yaml ~/.sematic/settings.yaml.prod2 | ||
$ vim ~/.sematic/settings.yaml.prod1 # edit whatever you need to | ||
$ vim ~/.sematic/settings.yaml.prod2 # edit whatever you need to | ||
$ # To use prod1: | ||
$ cp ~/.sematic/settings.yaml.prod1 ~/.sematic/settings.yaml | ||
$ sematic settings show | ||
Active user settings: | ||
|
||
SEMATIC_API_ADDRESS: https://<my_prod1_host:port> | ||
SEMATIC_API_KEY: <my_prod1_api_key> | ||
$ # To use prod2: | ||
$ cp ~/.sematic/settings.yaml.prod2 ~/.sematic/settings.yaml | ||
$ sematic settings show | ||
Active user settings: | ||
|
||
SEMATIC_API_ADDRESS: https://<my_prod2_host:port> | ||
SEMATIC_API_KEY: <my_prod2_api_key> | ||
``` | ||
|
||
## Building the Wheel | ||
|
@@ -88,7 +91,7 @@ If you want to build the wheel from source, do: | |
|
||
```bash | ||
$ make ui | ||
$ make wheel | ||
$ make uv-wheel | ||
``` | ||
|
||
## SQLite vs Postgres Migrations | ||
|
@@ -115,10 +118,8 @@ performed in order, without skipping any. If any errors occur, you must fix | |
them and then go back and redo all the steps that have been affected by the | ||
changes. | ||
|
||
1. Announce internally that a `main` merge freeze has started. | ||
|
||
1. Bump the version in: | ||
- `wheel_constants.bzl` - `wheel_version_string` | ||
- `pyproject.yaml` - `version` | ||
- `helm/sematic-server/Chart.yaml` - `appVersion` | ||
- `sematic/versions.py` - `CURRENT_VERSION`; bump `MIN_CLIENT_SERVER_SUPPORTS` if | ||
there are any TODOs mentioning this should be done | ||
|
@@ -138,12 +139,12 @@ changes. | |
|
||
1. Build the wheel: | ||
```bash | ||
$ make wheel | ||
$ make uv-wheel | ||
``` | ||
|
||
1. Test the Server wheel locally for all supported versions of Python. | ||
|
||
1. Copy the wheel from `bazel-bin/sematic/sematic-*.whl` into a scratch | ||
1. Copy the wheel from `./dist/sematic-*.whl` into a scratch | ||
directory. | ||
|
||
1. For each supported version of Python, use a virtual env to test: | ||
|
@@ -159,7 +160,7 @@ changes. | |
|
||
1. Test the [internal Helm charts](/helm/sematic-server) deployment. | ||
|
||
1. **Test an upgrade deployment** in the `stage` environment, **where the previous | ||
1. **Test an upgrade deployment** in a staging environment, **where the previous | ||
version is already deployed**, use | ||
[`serve-dev`](https://github.com/sematic-ai/infrastructure/tree/main/bin) to upgrade | ||
the deployment with a Docker image built on-the-fly from the current release commit. | ||
|
@@ -175,20 +176,13 @@ changes. | |
1. Smoke test new features that were included or significantly updated in the | ||
release. | ||
|
||
1. Ensure you have a compatible version of bazel: | ||
|
||
```bash | ||
$ export USE_BAZEL_VERSION=6.1.1 | ||
$ sudo apt update && sudo apt install bazel-6.1.1 | ||
``` | ||
|
||
1. Run the Testing Pipeline with the test cases listed below on this deployment, and | ||
check that it completes successfully, while perusing its outputs and logs to check | ||
that they render correctly. | ||
|
||
```bash | ||
$ # STAGE: | ||
$ bazel run sematic/examples/testing_pipeline:__main__ -- \ | ||
$ sematic run --build sematic/examples/testing_pipeline/__main__.py -- \ | ||
--cloud \ | ||
--detach \ | ||
--max-parallelism 10 \ | ||
|
@@ -198,19 +192,16 @@ changes. | |
--sleep 10 \ | ||
--spam-logs 1000 \ | ||
--fan-out 10 \ | ||
--raise-retry 0.7 \ | ||
--raise-retry 0.1 \ | ||
--external-resource \ | ||
--expand-shared-memory \ | ||
--mount-host-path /tmp /test \ | ||
--mount-host-path /tmp /test2 \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Our current staging environment doesn't have host path mounting enabled, and that is a required server config for this part of the test to work. |
||
--cache-namespace test \ | ||
--images \ | ||
--virtual-funcs \ | ||
--fork-subprocess return 0 \ | ||
--fork-subprocess exit 0 \ | ||
--fork-subprocess exit 1 \ | ||
--fork-subprocess signal 2 \ | ||
--fork-subprocess signal 15 | ||
--fork-subprocess signal 2 | ||
``` | ||
|
||
1. Test client backwards compatibility. Install the version of Sematic that matches | ||
|
@@ -219,11 +210,11 @@ changes. | |
the new Server. | ||
```bash | ||
$ sematic version # check that the correct min version is installed | ||
$ STAGE: | ||
$ # STAGE: | ||
$ sematic run examples/mnist/pytorch | ||
``` | ||
|
||
1. **Test a clean installation** in the same `stage` environment. | ||
1. (Optional) **Test a clean installation** in the same `stage` environment. | ||
```bash | ||
$ # STAGE: | ||
$ helm uninstall sematic-server -n stage | ||
|
@@ -237,7 +228,7 @@ changes. | |
aspect as well. | ||
```bash | ||
$ # STAGE: | ||
$ bazel run sematic/examples/testing_pipeline:__main__ | ||
$ sematic run --build sematic/examples/testing_pipeline/__main__.py | ||
``` | ||
|
||
1. Test publishing the wheel. Check if the generated webpage on `test.pypi.org` is | ||
|
@@ -271,17 +262,15 @@ changes. | |
|
||
1. Add the git tag. | ||
```bash | ||
$ export RELEASE_VERSION=v$(python3.9 ./sematic/versions.py) | ||
$ export RELEASE_VERSION=v$(python3 ./sematic/versions.py) | ||
$ git tag $RELEASE_VERSION | ||
$ git push origin $RELEASE_VERSION | ||
``` | ||
|
||
1. Announce internally that the `main` merge freeze is over. | ||
|
||
1. Build and push the Server Docker image. Use the Dockerfile at | ||
`docker/Dockerfile.server`. | ||
```bash | ||
$ TAG=v$(python3.9 ./sematic/versions.py) make release-server | ||
$ TAG=v$(python3 ./sematic/versions.py) make release-server | ||
``` | ||
|
||
2. Next you can generate the Helm package and publish it to the Helm repository. | ||
|
@@ -331,19 +320,6 @@ changes. | |
repo. Commit and push all of these to a new release branch, and create a PR for the change | ||
based on `gh-pages`. Wait for approval, and merge it. | ||
|
||
1. Deploy this new official release to the `stage` environment, in order to leave it in a | ||
consistent and expected state, and to test the actual commands users will be using to | ||
deploy the release. | ||
```bash | ||
$ # Run this step if it has never been done before | ||
$ helm repo add sematic-ai https://sematic-ai.github.io/helm-charts | ||
|
||
$ # STAGE: | ||
$ helm repo update | ||
$ helm upgrade sematic-server sematic-ai/sematic-server -n stage -f /path/to/stage_values.yml | ||
$ helm list -n stage # check that the expected APP VERSION was deployed | ||
``` | ||
|
||
1. Finally, draft the release on GitHub, from | ||
[the tag you previously committed](https://github.com/sematic-ai/sematic/tags): | ||
- Pick the `"previous tag"` from the dropdown to refer to the previous release. | ||
|
@@ -382,11 +358,3 @@ cases, instead of performing the release from the `main` branch, we: | |
- cherry-pick the commits we want to include | ||
- follow all the other steps from the [Releasing](#releasing) section | ||
- switch to the `main` branch, make a commit that contains the previous version | ||
increase and reconciles the changelog, and merge a PR with this commit | ||
|
||
## Updating the CircleCi Image | ||
|
||
The image used for most of our CircleCi steps is built using `docker/Dockerfile.ci`. | ||
After updating it, open a PR with the change, push the image to Dockerhub using the PR | ||
number as a tag (in order to be able to maintain a history, and revert, if necessary), | ||
and update the `SEMATIC_CI_IMAGE` env var in the CircleCi project settings. |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
retries are flaky, likely due to the issues described in #609