Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option learn-ocaml build serve --serve-during-build and fix related minor issues #597

Merged
merged 5 commits into from
Apr 19, 2024

Conversation

erikmd
Copy link
Member

@erikmd erikmd commented Apr 9, 2024

Description

Good news, @AltGr @yurug !

I was able (with Louis' advice from last week) to implement #594.

Regarding the naming of the CLI command, I now propose --serve-during-build (better than my initial try, --fork-replace)

Each commit comes with a detailed description (or motivation), in particular note that:

  • I fixed a minor issue (that originated in the --replace PR) due to the fact alpine's default lsof supports no option; unlike the lsof package;
  • I fixed another minor issue regarding a missing flush in Learnocaml_server.kill_running;
  • and I changed the formatting of the displayed version (→ if you don't object, it will print Learnocaml v1.1.0 running instead of Learnocaml v.1.1.0 running, which was making less sense given our current git tag convention).

I thoroughly tested the new option as summarized in the message of the main commit f0108a3
but as suggested by @AltGr, I didn't turn the option on by default in the Dockerfile.

Regarding performance, I ran an experiment on learn-ocaml-corpus endowed with the following docker-compose.yml:

docker-compose.yml
services:
  lo-test:
    image: learn-ocaml  # build using ( sudo make docker-images )
    volumes:
      - ../learn-ocaml-corpus:/repository
      - lo-test-serve-during-build-sync:/sync
    ports:
      - '8080:8080'
    restart: unless-stopped
    environment:
      - 'LEARNOCAML_SERVE_DURING_BUILD=true'

volumes:
  lo-test-serve-during-build-sync:
    driver: local

along with the following script:

shell session
$ sudo make docker-images
$ sudo docker compose up -d
$ sudo docker compose logs -t lo-test
$ sudo docker compose stop
$ touch ../learn-ocaml-corpus/exercises/fpottier/*/test.ml  # touch 26 exercises
$ sudo docker compose restart
$ sudo docker compose logs -t lo-test

And I got:

  • (around 10' to build the docker image locally)
  • 54s to build learn-ocaml-corpus (offline)
  • (<1s to start the server)
  • (immediate stop)
  • (<1s to restart the child server from the previous www)
  • 19s to rebuild learn-ocaml-corpus/exercises/fpottier (online)
  • (<1s to kill the child server and start the main one)

So I believe the effort was worth it for this quite standard use case :)

Now, let's wait for the CI ! and let me know if you have review comments.

And last hint: we shouldn't squash-merge this PR but rather do a regular merge (to preserve the atomic commits).

Checklist

Note to maintainers

  • Read this wiki page.
  • Make sure the PR has a milestone.
  • Assign yourself before merging.
  • Either do a regular merge:
    • for PRs containing several commits following conventional-commits,
    • or for PRs containing 1 commit shared with a later PR (to preserve the SHA1)
  • Or do a squash-merge:
    • for PRs containing only 1 commit (not shared with a later PR),
    • or for PRs containing several commits that need not be kept in the history;
    • Update the commit message header with a conventional-commit type,
    • Add a footer Close #… if a related issue exists.

@erikmd erikmd added the kind: feature New user-facing feature. label Apr 9, 2024
@erikmd erikmd force-pushed the feat/serve-during-rebuild branch 4 times, most recently from 2bb83c6 to 6707bc9 Compare April 9, 2024 21:09
.ci-macosx.sh Outdated Show resolved Hide resolved
Copy link
Collaborator

@AltGr AltGr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, this is more complex than I anticipated.

It's fine overall but I think some points should be clarified, in particular I have trouble following the meaning and processing of the child_pid variable/flag.
I also added some suggestions to make the user-facing information easier to grasp.

Dockerfile Outdated Show resolved Hide resolved
src/main/learnocaml_main.ml Outdated Show resolved Hide resolved
src/main/learnocaml_main.ml Show resolved Hide resolved
src/main/learnocaml_main.ml Outdated Show resolved Hide resolved
src/main/learnocaml_main.ml Outdated Show resolved Hide resolved
src/main/learnocaml_main.ml Outdated Show resolved Hide resolved
src/main/learnocaml_main.ml Outdated Show resolved Hide resolved
src/main/learnocaml_server_args.ml Outdated Show resolved Hide resolved
src/main/learnocaml_server_main.ml Outdated Show resolved Hide resolved
src/main/learnocaml_main.ml Outdated Show resolved Hide resolved
@erikmd erikmd mentioned this pull request Apr 11, 2024
5 tasks
@erikmd
Copy link
Member Author

erikmd commented Apr 11, 2024

Hi @AltGr @yurug, I believe I addressed all your suggestions.

Copy link
Collaborator

@AltGr AltGr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, besides nitpicking on the manpage section the new option appears in, this seems all fine! :)

Motivation:

- Makes it possible to provide a feature similar to the `--replace` option
  (namely, `learn-ocaml build serve --replace`) within a Docker context.

  As `--replace` needed to successively start 2 learn-ocaml processes
  listening to the same port, but if we spin two different containers,
  ./www is not shared between containers, nor the local TCP interface.

- If tweaking the default entrypoint could be an alternative solution,
  the shell script would be involved to cope with the need to handle
  signals properly.

- The solution implemented in this commit is simpler and can be enabled
  in a docker-compose context by passing:
  ```
  environment:
    LEARNOCAML_SERVE_DURING_BUILD: 'true'
  ```
  or:
  ```
  environment:
    - 'LEARNOCAML_SERVE_DURING_BUILD=true'
  ```
  then run a command such as `docker restart learn-ocaml`.

Remarks:

- Using docker-compose, to restart a server and benefit from this feature, use
  ( docker compose stop ; docker compose restart )
  rather than
  ( docker compose down ; docker compose up -d )

- This commit has been double-checked with both native server
  ( make ; make opaminstall ;
    learn-ocaml build serve --serve-during-build --repo=$REPO )
  and bytecode server
  ( make ; make opaminstall ; mv $(which learn-ocaml-server){,~} ;
    learn-ocaml build serve --serve-during-build --repo=$REPO )

- For uniformity, this commit also introduces an environment variable
  'LEARNOCAML_REPLACE=true' as a fallback for `--replace`.

Close ocaml-sf#594
Beforehand:
- Learnocaml v.1.0.0 running.
- Learnocaml server v.1.0.0 starting on port 8080

Henceforth:
- Learnocaml v1.0.0 running.
- Learnocaml server v1.0.0 starting on port 8080
This makes it possible to fix the error:
```
Waiting for process 9 to terminate...  0 Error: process didn't terminate in time
```
triggered by either `--serve-during-build` or `--replace` CLI options.
@erikmd
Copy link
Member Author

erikmd commented Apr 19, 2024

@AltGr

while I agree that code-wise they are pretty different, from a user-perspective, they're both options concerning the server though (as the name implies!) -- and with similar use-cases so it would be best to have them closer to each other.

We could cheat by keeping this organisation in the code, but hardcoding ~docs to still put it in the SERVER section of the manpage (COMMON OPTIONS implicitely means the union of the commands, here we're dealing with the intersection :) )

Very good point, thanks! I fixed it as suggested.

Thanks again for your advice and help!

@erikmd
Copy link
Member Author

erikmd commented Apr 19, 2024

With this last tweak (and rebase), CI is green.

@AltGr @yurug If you don't object, I will merge the PR today at 16:00 or so.

@erikmd erikmd merged commit dc6f569 into ocaml-sf:master Apr 19, 2024
12 checks passed
@erikmd erikmd deleted the feat/serve-during-rebuild branch April 19, 2024 16:20
@erikmd erikmd self-assigned this Apr 20, 2024
@erikmd erikmd added this to the learn-ocaml 1.1.0 milestone Apr 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: feature New user-facing feature.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: new option à la --replace to leverage this existing feature in a docker restart context
3 participants