Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'Initialization scripts' like in https://hub.docker.com/_/postgres implemented #538

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,3 +82,24 @@ using KMS.
### KMS Encryption Key
Which key to use to encrypt the passwords in the template. By using KMS encryption we avoid storing the passwords
in plain text anywhere. They will only be available to the EC2 instances running Spilo.

### Initialization scripts

If you would like to do additional initialization in an image derived from this one, add one or more `*.sql`, `*.sql.gz`, or `*.sh` scripts under `/docker-entrypoint-initdb.d` (creating the directory if necessary). After the entrypoint calls `initdb` to create the default `postgres` user and database, it will run any `*.sql` files, run any executable `*.sh` scripts, and source any non-executable `*.sh` scripts found in that directory to do further initialization before starting the service.

**Warning**: scripts in `/docker-entrypoint-initdb.d` are only run if you start the container with a data directory that is empty; any pre-existing database will be left untouched on container startup. One common problem is that if one of your `/docker-entrypoint-initdb.d` scripts fails (which will cause the entrypoint script to exit) and your orchestrator restarts the container with the already initialized data directory, it will not continue on with your scripts.

For example, to add an additional user and database, add the following to `/docker-entrypoint-initdb.d/init-user-db.sh`:

```bash
#!/bin/bash
set -e

psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE USER docker;
CREATE DATABASE docker;
GRANT ALL PRIVILEGES ON DATABASE docker TO docker;
EOSQL
```

These initialization files will be executed in sorted name order as defined by the current locale, which defaults to `en_US.utf8`. Any `*.sql` files will be executed by `POSTGRES_USER`, which defaults to the `postgres` superuser. It is recommended that any `psql` commands that are run inside of a `*.sh` script be executed as `POSTGRES_USER` by using the `--username "$POSTGRES_USER"` flag. This user will be able to connect without a password due to the presence of `trust` authentication for Unix socket connections made inside the container.
66 changes: 59 additions & 7 deletions postgres-appliance/scripts/post_init.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
#!/bin/bash

cd "$(dirname "${BASH_SOURCE[0]}")" || exit 1
export POSTGRES_USER="$1"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first parameter is actually not a postgres user, but HUMAN_ROLE.

export POSTGRES_DB="$2"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The second parameter is not the database name, but the connection string in the format "host=localhost port=5432 user=postgres dbname=postgres".

There is one exception though, the post_init.sh could be called from on_role_change.sh. In this case the parameter would be just "postgres".


PGVER=$(psql -d "$2" -XtAc "SELECT pg_catalog.current_setting('server_version_num')::int/10000")
PGVER=$(psql -d "$POSTGRES_DB" -XtAc "SELECT pg_catalog.current_setting('server_version_num')::int/10000")
if [ "$PGVER" -ge 12 ]; then RESET_ARGS="oid, oid, bigint"; fi

(echo "DO \$\$
Expand All @@ -17,11 +19,11 @@ END;\$\$;

DO \$\$
BEGIN
PERFORM * FROM pg_catalog.pg_authid WHERE rolname = '$1';
PERFORM * FROM pg_catalog.pg_authid WHERE rolname = '$POSTGRES_USER';
IF FOUND THEN
ALTER ROLE $1 WITH NOCREATEDB NOLOGIN NOCREATEROLE NOSUPERUSER NOREPLICATION INHERIT;
ALTER ROLE $POSTGRES_USER WITH NOCREATEDB NOLOGIN NOCREATEROLE NOSUPERUSER NOREPLICATION INHERIT;
ELSE
CREATE ROLE $1;
CREATE ROLE $POSTGRES_USER;
END IF;
END;\$\$;

Expand Down Expand Up @@ -149,7 +151,7 @@ while IFS= read -r db_name; do
echo "SELECT public.postgis_extensions_upgrade();"
fi
fi
sed "s/:HUMAN_ROLE/$1/" create_user_functions.sql
sed "s/:HUMAN_ROLE/$POSTGRES_USER/" create_user_functions.sql
echo "CREATE EXTENSION IF NOT EXISTS pg_stat_statements SCHEMA public;
CREATE EXTENSION IF NOT EXISTS pg_stat_kcache SCHEMA public;
CREATE EXTENSION IF NOT EXISTS set_user SCHEMA public;
Expand All @@ -158,5 +160,55 @@ GRANT EXECUTE ON FUNCTION public.set_user(text) TO admin;
GRANT EXECUTE ON FUNCTION public.pg_stat_statements_reset($RESET_ARGS) TO admin;"
if [ "x$ENABLE_PG_MON" = "xtrue" ] && [ "$PGVER" -ge 11 ]; then echo "CREATE EXTENSION IF NOT EXISTS pg_mon SCHEMA public;"; fi
cat metric_helpers.sql
done < <(psql -d "$2" -tAc 'select pg_catalog.quote_ident(datname) from pg_catalog.pg_database where datallowconn')
) | PGOPTIONS="-c synchronous_commit=local" psql -Xd "$2"
done < <(psql -d "$POSTGRES_DB" -tAc 'select pg_catalog.quote_ident(datname) from pg_catalog.pg_database where datallowconn')
) | PGOPTIONS="-c synchronous_commit=local" psql -Xd "$POSTGRES_DB"


#The folloving code is taken from https://github.com/docker-library/postgres/blob/master/docker-entrypoint.sh

# usage: docker_process_init_files [file [file [...]]]
# ie: docker_process_init_files /always-initdb.d/*
# process initializer files, based on file extensions and permissions
docker_process_init_files() {
echo
local f
for f; do
case "$f" in
*.sh)
# https://github.com/docker-library/postgres/issues/450#issuecomment-393167936
# https://github.com/docker-library/postgres/pull/452
if [ -x "$f" ]; then
echo "$0: running $f"
"$f"
else
echo "$0: sourcing $f"
# shellcheck disable=SC1090
source "$f"
fi
;;
*.sql) echo "$0: running $f"; docker_process_sql -f "$f"; echo ;;
*.sql.gz) echo "$0: running $f"; gunzip -c "$f" | docker_process_sql; echo ;;
*.sql.xz) echo "$0: running $f"; xzcat "$f" | docker_process_sql; echo ;;
*) echo "$0: ignoring $f" ;;
esac
echo
done
}

# Execute sql script, passed via stdin (or -f flag of pqsl)
# usage: docker_process_sql [psql-cli-args]
# ie: docker_process_sql --dbname=mydb <<<'INSERT ...'
# ie: docker_process_sql -f my-file.sql
# ie: docker_process_sql <my-file.sql
docker_process_sql() {
local query_runner=( psql -v "ON_ERROR_STOP=1" --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" --no-password )

"${query_runner[@]}" "$@"
}



# check dir permissions to reduce likelihood of half-initialized database
ls /docker-entrypoint-initdb.d/ > /dev/null

docker_process_init_files /docker-entrypoint-initdb.d/*
Comment on lines +256 to +258
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since in the readme you tell that scripts will be executed only for the new cluster, it makes sense to execute this block only if we know that post_init is not executed from on_role_change:

if [ "$2" != "postgres" ]; then
    ...
fi