redockerize and deproxify #58
Conversation
So I can try manually what is going wrong.
This reverts commit 5576b495b30fef08b310d6414a0936e39e4d2a98.
This reverts commit d0b59f9.
There was a problem hiding this comment.
@BeritJanssen here is a description of what I'm currently running into. More details in the comments below.
The generation of the Angular frontend proceeds roughly as follows. Notice that it is a back-and-forth between Python on the host machine (odd steps) and node.js inside Docker containers (even steps).
- The
frontend.angulardirectory comes rendered out of thecookiecuttermain template run. All steps after this happen during the post-generation hook. - A new Angular project directory with the name substituted for
{{cookiecutter.npm_slug}}is created usingng new. The contents offrontend.angularare copied into this directory after it is created. This happens in thebootstrapstage in theDockerfile-postgenerate. A host directory named justfrontendis also bind mounted into the container. - The properties in the
frontend.angular/package.overwrite.jsonare merged into the{{cookiecutter.npm_slug}}/package.json. This is what happens in between Dockerfile stages withCMD cp, the first call tooverride_jsonin thebootstrap.py:activate_frontendand finallyCOPY. ng add @angular/localizeis run inside the{{cookiecutter.npm_slug}}directory. This happens in thebootstrap-2stage.- The
angular.jsonis customized with a procedure like step 3. ng extract-i18nis run inside the{{cookiecutter.npm_slug}}directory in thebootstrap-3stage. After this, all contents of the{{cookiecutter.npm_slug}}directory (which only exists in the container) are copied to thefrontendbind mount.- Further modifications to the locale files happen inside the
frontenddirectory in Python on the host machine.
Steps 4-7 are currently commented out, so you can run step 4 manually and reproduce the issue that I run into. Steps to reproduce:
- Clone the cookiecutter-webapp-deluxe repo (if necessary) and checkout the
feature/redockerize-deproxifybranch. - Use your local clone to generate a scrap project using
cookiecutter ../relative/path/to/cookiecutter-webapp-deluxe. Tip: you can run this again with the exact same settings, without having to answer all the questions again, by adding the--replayflag. cdinto the generated project.- Run
docker-compose -f compose-postgenerate.yml --profile bootstrap-2 run frontend-2 sh. This is the manual version of step 4 in the previous list. - Observe the contents of the
/usr/src/app/{{cookiecutter.npm_slug}}directory inside the container. Notice that theCOPYcommand on line 16 of theDockerfile-postgenerateappears to have been undone. In other words, the project directory appears to be pristine as it was right afterng new, with all subsequent customizations somehow having disappeared.
I hope you understand what is going on here!
| venv = create_virtualenv() | ||
| pip_tools = backreq = backpack = clone_req = funcreq = funcpack = False | ||
| if venv: | ||
| adopt_virtualenv(VIRTUALENV) | ||
| pip_tools = install_pip_tools() | ||
| if pip_tools: | ||
| check_version_pip() | ||
| check_version_pip_compile() | ||
| backreq = compile_backend_requirements() | ||
| if backreq: | ||
| backpack = install_backend_packages() | ||
| clone_req = copy_backreq_to_func() | ||
| if clone_req: | ||
| funcreq = compile_functest_requirements() | ||
| if funcreq: | ||
| funcpack = install_functest_packages() | ||
| frontpack = install_frontend_packages() | ||
| git = setup_git() | ||
| gitflow = develop = stage = commit = remote = False | ||
| if git: | ||
| gitflow = setup_gitflow() | ||
| if not gitflow: | ||
| develop = create_develop_branch() | ||
| if funcreq and frontpack: | ||
| stage = git_add() | ||
| if stage: | ||
| commit = initial_commit() | ||
| remote = add_remote() | ||
| db = create_db() | ||
| if db: | ||
| db = grant_db() | ||
| migrate = superuser = False | ||
| if db and backpack: | ||
| migrate = run_migrations() | ||
| if migrate: | ||
| superuser = create_superuser() | ||
| if not all([superuser, remote, commit, gitflow, funcpack]): | ||
| print('\nPlease read {} for information on failed commands.'.format(LOGFILE_NAME)) | ||
| print('\nAlmost ready to go! Just a couple more commands to run:') | ||
| print(cd_into_project) | ||
| if not venv: print(create_virtualenv) | ||
| print(activate_venv) | ||
| if not pip_tools: print(install_pip_tools) | ||
| if not backreq: print(compile_backend_requirements) | ||
| if not clone_req: print(copy_backreq_to_func) | ||
| if not funcreq: print(compile_functest_requirements) | ||
| if not (backpack and frontpack and funcpack): print(install_all_packages) | ||
| if not git: print(setup_git) | ||
| if not gitflow and not develop: print(create_develop_branch) | ||
| if not stage: print(git_add) | ||
| if not commit: print(initial_commit) | ||
| if not remote: print(add_remote) | ||
| if not db: | ||
| print(create_db) | ||
| print(grant_db) | ||
| if not migrate: print(run_migrations) | ||
| if not superuser: print(create_superuser) | ||
| print(git_push) | ||
| print(yarn_start) |
There was a problem hiding this comment.
Some, but not all, of this logic is or will be restored via the bootstrap_subprojects command, which relies on docker-compose.
hooks/post_gen_project.py
Outdated
| @@ -121,6 +66,8 @@ def generate_backbone_translations(): | |||
| return True | |||
|
|
|||
|
|
|||
| bootstrap_subprojects = make_bootstrap_command('bootstrap') | |||
There was a problem hiding this comment.
make_bootstrap_command is currently defined in the bootstrap.py, but will move here.
{{cookiecutter.slug}}/bootstrap.py
Outdated
| project_name = '{{cookiecutter.slug}}'.replace('_', '-') | ||
| Command( | ||
| 'Install dependencies', | ||
| ['yarn', 'install', '--ignore-scripts'] | ||
| )() | ||
| Command( | ||
| 'Creating project', | ||
| ['yarn', 'ng', 'new', project_name, '--prefix={{cookiecutter.app_prefix}}', | ||
| '--ssr', | ||
| '--skip-git=true', | ||
| '--skip-install=true', | ||
| '--package-manager=yarn', | ||
| '--style=scss', | ||
| '--routing=true'] | ||
| )() | ||
| shutil.copytree('frontend.angular', project_name, dirs_exist_ok=True) | ||
| os.rename(project_name, 'frontend') | ||
| shutil.move(op.join('frontend', 'proxy.conf.json'), 'proxy.conf.json') |
There was a problem hiding this comment.
This part of the old activation logic is implemented in the bootstrap stage of the frontend.angular/Dockerfile-postgenerate. This already works as intended.
{{cookiecutter.slug}}/bootstrap.py
Outdated
| )() | ||
| shutil.copytree('frontend.angular', project_name, dirs_exist_ok=True) | ||
| os.rename(project_name, 'frontend') | ||
| shutil.move(op.join('frontend', 'proxy.conf.json'), 'proxy.conf.json') | ||
| override_json('package') |
There was a problem hiding this comment.
This part is simpy retained and (still) works as intended. The implementation will be moved to the hooks/post_gen_project.py, though.
| Command( | ||
| 'Install frontend dependencies using Yarn', | ||
| ['yarn'], | ||
| cwd="frontend" | ||
| )() | ||
| # Remove favicon.ico | ||
| os.remove(os.path.join('frontend', 'src', 'favicon.ico')) | ||
| # Remove editorconfig | ||
| os.remove(os.path.join('frontend', '.editorconfig')) | ||
| Command( | ||
| 'ng add @angular/localize', | ||
| ['yarn', 'ng', 'add', '@angular/localize', '--skip-confirmation'], | ||
| cwd="frontend" | ||
| )() |
There was a problem hiding this comment.
This part is implemented in the bootstrap-2 stage of the frontend.angular/Dockerfile-postgenerate. It doesn't work as intended for a surprising reason, which I'll highlight in the Dockerfile.
{{cookiecutter.slug}}/bootstrap.py
Outdated
| ['yarn', 'ng', 'config', "projects.{{cookiecutter.slug | replace('_', '-')}}.architect.serve.options.port", '{{cookiecutter.frontend_port}}'], | ||
| cwd="frontend" | ||
| )() | ||
| # if not angular_bootstrap_2(): |
There was a problem hiding this comment.
This is the part where the bootstrap-2 stage of the Dockerfile-postgenerate gets to do its tricks.
{{cookiecutter.slug}}/bootstrap.py
Outdated
| # if not angular_bootstrap_2(): | ||
| # return false | ||
| # override_json('angular') | ||
| # if not angular_bootstrap_3(): |
There was a problem hiding this comment.
Likewise for the bootstrap-3 stage.
{{cookiecutter.slug}}/bootstrap.py
Outdated
| def make_bootstrap_command(profile): | ||
| return Command( | ||
| f'Finalize subproject package configuration ({profile})', | ||
| ['docker', 'compose', | ||
| '-f', 'compose-postgenerate.yml', | ||
| '--profile', profile, | ||
| 'up'] | ||
| ) | ||
|
|
||
|
|
||
| angular_bootstrap_2 = make_bootstrap_command('bootstrap-2') | ||
| angular_bootstrap_3 = make_bootstrap_command('bootstrap-3') |
There was a problem hiding this comment.
Like activate_frontend, this logic will move to the hooks/post_gen_project.py.
There was a problem hiding this comment.
Note that this compose file has three profiles:
bootstrap, which runs as part ofhooks/post_gen_project.py:main.bootstrap-2, which runs as part ofbootstrap.py:activate_frontend.bootstrap-3, which also runs as part ofbootstrap.py:activate_frontend.
The names of the profiles are the same as, and correspond to, the stages in the frontend.angular/Dockerfile-postgenerate. bootstrap-2 and bootstrap-3 are only needed for Angular.
| yarn ng add @angular/localize --skip-confirmation | ||
|
|
||
| VOLUME /usr/src/app/frontend | ||
| CMD cp angular.json angular.overwrite.json ../frontend/ |
There was a problem hiding this comment.
Here's the surprising part that I'm currently running into. At this point in the image layering, the contents of /usr/src/app/{{cookiecutter.npm_sug}} appear to have been reset to how they were at the WORKDIR on line 15. @BeritJanssen do you understand what is going on here?
|
@BeritJanssen and I discussed the above issue on Teams. Some notes, mostly as a reminder to self:
|
This is a work in progress. This description will likely be updated at least a few times in the future. I will certainly rebase and force-push the branch multiple times.
I am trying to close #16 and close #35 at the same time. My intention is to first get everything working in Docker and then git rid of the proxy situation. I'm currently having some trouble with Docker and am hoping that @BeritJanssen can take a look. See my own review below for details.
cookiecutter, node.js 18 and@angular/cliinstalledcookiecutterand Docker installedI envision the global workflow as follows:
cookiecuttergenerates initial files based on template directory. This part doesn't change much.ng newis taken care of in the post-generation hook usingdocker-compose. To this end, the frontend and backend both have a specialDockerfile-postgenerateand the project root has a specialcompose-postgenerate.yml, which get deleted right after generation is done. This is the part where I'm currently having trouble.docker-compose up. It is no longer necessary to runpython bootstrap.pyon the first run after cloning. Some of the logic in thebootstrap.pyis still needed during generation, but it can be merged into thehooks/post_gen_project.py.