[feat](docker)Add a BE ENV item 'SKIP_CHECK_ULIMIT' for Docker to start quickly #45267
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
FreeOnePlus
changed the title
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run buildall |
16 tasks
TPC-H: Total hot run time: 40068 ms |
|
run buildall |
TPC-H: Total hot run time: 39724 ms |
|
TeamCity be ut coverage result: |
TPC-DS: Total hot run time: 189468 ms |
ClickBench: Total hot run time: 33.56 s |
morningman
approved these changes
Dec 12, 2024
|
clang-tidy review says "All clean, LGTM! 👍" |
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
catpineapple
approved these changes
Dec 12, 2024
zy-kkk
pushed a commit
that referenced
this pull request
Dec 13, 2024
…uirements for rapid Docker startup. (#45269) Related PR: #45267 Problem Summary: To meet the needs of rapid Docker startup, I have made adjustments to two related scripts in the Docker startup process. First, I added a env `SKIP_CHECK_ULIMIT` to the `start_be.sh` script, which will skip the size checks for `swap`, `ulimit`, and `max_map_count`. At the same time, I used `--console` to start the process and print logs. The reason why I did not use the `--daemon` daemon command to execute is that starting with a foreground log printing method in a Docker container is the correct and reliable approach. At the same time, I added a check logic for a `be.conf` configuration item in the `init_be.sh` script: if it is the first time starting, append the export `SKIP_CHECK_ULIMIT=true` to skip the `ulimit` value check in the BE process. In summary, these adjustments can meet the basic requirements for rapid Docker startup usage.
|
run buildall |
github-actions bot
pushed a commit
that referenced
this pull request
Dec 13, 2024
…uirements for rapid Docker startup. (#45269) Related PR: #45267 Problem Summary: To meet the needs of rapid Docker startup, I have made adjustments to two related scripts in the Docker startup process. First, I added a env `SKIP_CHECK_ULIMIT` to the `start_be.sh` script, which will skip the size checks for `swap`, `ulimit`, and `max_map_count`. At the same time, I used `--console` to start the process and print logs. The reason why I did not use the `--daemon` daemon command to execute is that starting with a foreground log printing method in a Docker container is the correct and reliable approach. At the same time, I added a check logic for a `be.conf` configuration item in the `init_be.sh` script: if it is the first time starting, append the export `SKIP_CHECK_ULIMIT=true` to skip the `ulimit` value check in the BE process. In summary, these adjustments can meet the basic requirements for rapid Docker startup usage.
|
TeamCity be ut coverage result: |
github-actions bot
pushed a commit
that referenced
this pull request
Dec 16, 2024
…rt quickly (#45267) In the storage_engine.cpp of the BE process, it is mandatory to check that the `ulimit` value must be greater than 60,000. When starting with Docker or Docker-Compose, it is necessary to preemptively change the corresponding value on the host machine. This change is very unfriendly to Docker, as it loses its unique advantage of being able to "build quickly and anywhere," and cannot become a fundamental capability for rapid startup. Therefore, a new environment variable has been added to control whether to skip this check value. The default value is false, which enforces the check of this value. When set to true, it skips the check and starts directly.
github-actions bot
pushed a commit
that referenced
this pull request
Dec 16, 2024
…rt quickly (#45267) In the storage_engine.cpp of the BE process, it is mandatory to check that the `ulimit` value must be greater than 60,000. When starting with Docker or Docker-Compose, it is necessary to preemptively change the corresponding value on the host machine. This change is very unfriendly to Docker, as it loses its unique advantage of being able to "build quickly and anywhere," and cannot become a fundamental capability for rapid startup. Therefore, a new environment variable has been added to control whether to skip this check value. The default value is false, which enforces the check of this value. When set to true, it skips the check and starts directly.
yiguolei
pushed a commit
that referenced
this pull request
Dec 19, 2024
…ocker to start quickly #45267 (#45468) Cherry-picked from #45267 Co-authored-by: FreeOnePlus <[email protected]>
morningman
pushed a commit
that referenced
this pull request
Dec 23, 2024
…ocker to start quickly #45267 (#45467) Cherry-picked from #45267 Co-authored-by: FreeOnePlus <[email protected]>
16 tasks
yiguolei
pushed a commit
that referenced
this pull request
Dec 25, 2024
…uirements for rapid Docker startup(Merge 2.1). (#45858) ### What problem does this PR solve? Issue Number: close #xxx Related PR: #45267 Master PR: #45269 Problem Summary: To meet the needs of rapid Docker startup, I have made adjustments to two related scripts in the Docker startup process. First, I added a env `SKIP_CHECK_ULIMIT` to the `start_be.sh` script, which will skip the size checks for `swap`, `ulimit`, and `max_map_count`. At the same time, I used `--console` to start the process and print logs. The reason why I did not use the `--daemon` daemon command to execute is that starting with a foreground log printing method in a Docker container is the correct and reliable approach. At the same time, I added a check logic for a `be.conf` configuration item in the `init_be.sh` script: if it is the first time starting, append the export `SKIP_CHECK_ULIMIT=true` to skip the `ulimit` value check in the BE process. In summary, these adjustments can meet the basic requirements for rapid Docker startup usage.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
In the storage_engine.cpp of the BE process, it is mandatory to check that the
ulimitvalue must be greater than 60,000. When starting with Docker or Docker-Compose, it is necessary to preemptively change the corresponding value on the host machine. This change is very unfriendly to Docker, as it loses its unique advantage of being able to "build quickly and anywhere," and cannot become a fundamental capability for rapid startup. Therefore, a new environment variable has been added to control whether to skip this check value. The default value is false, which enforces the check of this value. When set to true, it skips the check and starts directly.Release note
None
Check List (For Author)
ulimit -n 50000to make it less than the BE check value of 60000.export SKIP_CHECK_ULIMIT = trueconfiguration item.export SKIP_CHECK_ULIMIT = falseconfiguration item.Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)