refactor: use heroku.yml and run migrations in release phase #50
Conversation
| COPY . . | ||
|
|
||
| # Install indexer | ||
| RUN pip3 install poetry==1.6.1 |
There was a problem hiding this comment.
note: this is an adjustment that i needed to make because previously i was using the python3-poetry from debian package manager, and that version of poetry did not support the export command in line 16 below
There was a problem hiding this comment.
Maybe worth adding a note to the Install indexer comment about the version and use of export.
| # Install indexer | ||
| RUN pip3 install poetry==1.6.1 | ||
| RUN poetry install | ||
| RUN poetry export --without-hashes --format=requirements.txt > requirements.txt |
There was a problem hiding this comment.
this is to work around the weird heroku "run", heroku.yml and poetry bug i mentioned.
this file does not need to be tracked, since it's being generated by poetry.
There was a problem hiding this comment.
As mentioned above, maybe worth making note of this within the file.
| release: | ||
| image: index-regen | ||
| command: | ||
| - PGSSLMODE=no-verify yarnpkg migrate |
There was a problem hiding this comment.
PGSSLMODE was needed because without it the underlying node postgres library throws an error.
this tells the pg library under the hood "don't worry about verifying the ssl certificate, but still make use of it"
the node pg library expects you to add certificates to verify ssl connections by default (and that can be tricky).
we do this in the regen-server as well, seems fairly low stakes in this case (this being "use ssl but don't verify")
wgwz
force-pushed
the
kyle/23-pg-migrate
branch
from
d900157
to
88b3f6f
Compare
wgwz
force-pushed
the
kyle/24-heroku-migrations
branch
from
c9cd11e
to
c0dead9
Compare
wgwz
force-pushed
the
kyle/24-heroku-migrations
branch
from
c0dead9
to
ea4b12e
Compare
|
i also re-deployed this to the staging analytics environment today: |
There was a problem hiding this comment.
LGTM and verified the successful deployment.
i also re-deployed this to the staging analytics environment today
We could set up review apps in heroku (same as we have set up for origination-server), which would have allowed you to test the heroku setup without manually deploying to the staging environment. Ideally we would only deploy to staging what has been reviewed, approved, and merged but not a major concern atm.
| COPY . . | ||
|
|
||
| # Install indexer | ||
| RUN pip3 install poetry==1.6.1 |
There was a problem hiding this comment.
Maybe worth adding a note to the Install indexer comment about the version and use of export.
| # Install indexer | ||
| RUN pip3 install poetry==1.6.1 | ||
| RUN poetry install | ||
| RUN poetry export --without-hashes --format=requirements.txt > requirements.txt |
There was a problem hiding this comment.
As mentioned above, maybe worth making note of this within the file.
👍 |
Closes: #24
Before deploying this, the following adjustment would need to be made for the production heroku app:
The reason that I had to change the heroku deployment method is because graphile-migrate requires a node environment.
The build pack we were using previously was a poetry-python specific build pack, with no easy way to add node support there.
This approach utilizes a version of the dockerfile for the indexer that copied what @ryanchristo put together for the tests.
For a few different reasons, we need a different version of this dockerfile for these changes:
When using the "container" stack (which is heroku's way to let you deploy custom containers), the "build context" for a relative path is always the directory of the dockerfile (this is a heroku feature that can't be changed). So if we tried using the
docker/indexer.Dockerfile, when the copy step gets run in heroku, then it would only copy files within thedocker/folder (see heroku's known issues and limitations for containers documented here)The heroku.yml "run" method was not working with poetry for some reason. It only seemed to be an issue in heroku. When I built and ran the container locally, it was working without issue. So there is some detail about how heroku and "run" command work that was incompatible with how the
docker/indexer.Dockerfileworks. I had to hack around this by using poetry to export to the "requirements.txt" format and then having the container install the dependencies from there. This is set up in a way where we can still manage our dependencies with poetry and there should be no interruptions otherwise. Just a weird annoying bug.