Skip to content

Commit

Permalink
[feat](docker)Modify the init_be and start_be scripts to meet the req…
Browse files Browse the repository at this point in the history
…uirements for rapid Docker startup. (#45269)

Related PR: #45267

Problem Summary:

To meet the needs of rapid Docker startup, I have made adjustments to two related scripts in the Docker startup process. First, I added a env `SKIP_CHECK_ULIMIT` to the `start_be.sh` script, which will skip the size checks for `swap`, `ulimit`, and `max_map_count`. At the same time, I used `--console` to start the process and print logs. The reason why I did not use the `--daemon` daemon command to execute is that starting with a foreground log printing method in a Docker container is the correct and reliable approach.

At the same time, I added a check logic for a `be.conf` configuration item in the `init_be.sh` script: if it is the first time starting, append the export `SKIP_CHECK_ULIMIT=true` to skip the `ulimit` value check in the BE process. In summary, these adjustments can meet the basic requirements for rapid Docker startup usage.
  • Loading branch information
FreeOnePlus authored and Your Name committed Dec 13, 2024
1 parent 0f95d59 commit f8a20f1
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 17 deletions.
36 changes: 19 additions & 17 deletions bin/start_be.sh
Original file line number Diff line number Diff line change
Expand Up @@ -175,29 +175,31 @@ if [[ "${RUN_VERSION}" -eq 1 ]]; then
exit 0
fi

if [[ "$(uname -s)" != 'Darwin' ]]; then
MAX_MAP_COUNT="$(cat /proc/sys/vm/max_map_count)"
if [[ "${MAX_MAP_COUNT}" -lt 2000000 ]]; then
echo "Set kernel parameter 'vm.max_map_count' to a value greater than 2000000, example: 'sysctl -w vm.max_map_count=2000000'"
exit 1
if [[ "${SKIP_CHECK_ULIMIT:- "false"}" != "true" ]]; then
if [[ "$(uname -s)" != 'Darwin' ]]; then
MAX_MAP_COUNT="$(cat /proc/sys/vm/max_map_count)"
if [[ "${MAX_MAP_COUNT}" -lt 2000000 ]]; then
echo "Set kernel parameter 'vm.max_map_count' to a value greater than 2000000, example: 'sysctl -w vm.max_map_count=2000000'"
exit 1
fi

if [[ "$(swapon -s | wc -l)" -gt 1 ]]; then
echo "Disable swap memory before starting be"
exit 1
fi
fi

if [[ "$(swapon -s | wc -l)" -gt 1 ]]; then
echo "Disable swap memory before starting be"
MAX_FILE_COUNT="$(ulimit -n)"
if [[ "${MAX_FILE_COUNT}" -lt 60000 ]]; then
echo "Set max number of open file descriptors to a value greater than 60000."
echo "Ask your system manager to modify /etc/security/limits.conf and append content like"
echo " * soft nofile 655350"
echo " * hard nofile 655350"
echo "and then run 'ulimit -n 655350' to take effect on current session."
exit 1
fi
fi

MAX_FILE_COUNT="$(ulimit -n)"
if [[ "${MAX_FILE_COUNT}" -lt 60000 ]]; then
echo "Set max number of open file descriptors to a value greater than 60000."
echo "Ask your system manager to modify /etc/security/limits.conf and append content like"
echo " * soft nofile 655350"
echo " * hard nofile 655350"
echo "and then run 'ulimit -n 655350' to take effect on current session."
exit 1
fi

# add java libs
# Must add hadoop libs, because we should load specified jars
# instead of jars in hadoop libs, such as avro
Expand Down
1 change: 1 addition & 0 deletions docker/runtime/be/resource/init_be.sh
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,7 @@ _main() {
fi
check_be_status
doris_note "Ready to start BE!"
export SKIP_CHECK_ULIMIT=true
${DORIS_HOME}/be/bin/start_be.sh --console &
child_pid=$!
wait $child_pid
Expand Down

0 comments on commit f8a20f1

Please sign in to comment.