Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move from redis to valkey #89

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ actinia-data/grassdb
actinia-data/resources
actinia-data/userdata
actinia-data/workspace
redis_data/dump.rdb
valkey_data/dump.rdb
8 changes: 4 additions & 4 deletions .vscode/tasks.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,17 @@
}
},
{
"label": "start-redis",
"label": "start-valkey",
"type": "shell",
"command": "docker-compose -f actinia-docker/docker-compose-dev.yml up -d --force-recreate redis",
"command": "docker-compose -f actinia-docker/docker-compose-dev.yml up -d --force-recreate valkey",
"isBackground": true,
},
{
"type": "docker-run",
"label": "docker-run: debug",
"dependsOn": [
"docker-build",
"start-redis"
"start-valkey"
],
"python": {
"module": "flask",
Expand All @@ -39,7 +39,7 @@
},
"dockerRun": {
"remove": true,
// network is needed when connecting to redis
// network is needed when connecting to valkey
// while not using docker-compose for startup
"network": "actinia-docker_actinia-dev",
"ports": [
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,9 @@ __actinia-data__

<a id="startup-errors"></a>
# How to fix common startup errors
* if a redis db is running locally this will fail. Run and try again:
* if a valkey db is running locally this will fail. Run and try again:
```
/etc/init.d/redis-server stop
/etc/init.d/valkey-server stop
```
* if you see an error like "max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]", run
```
Expand Down Expand Up @@ -207,15 +207,15 @@ See this [README](https://github.com/actinia-org/actinia-core/tree/main/docker#t
<a id="production-deployment"></a>
# Production deployment

To run actinia_core in production systems, you can use the docker-compose-prod.yml. Please change before the default redis password in redis_data/config/.redis and inside the actinia.cfg. Also incomment the `build` block in the `docker-compose.yml`. Additional change the keycloak and postgis passwords in `.env` file.
To run actinia_core in production systems, you can use the docker-compose-prod.yml. Please change before the default valkey password in valkey_data/config/.valkey and inside the actinia.cfg. Also incomment the `build` block in the `docker-compose.yml`. Additional change the keycloak and postgis passwords in `.env` file.

To start the server, run:

```
docker-compose -f docker-compose.yml build
docker-compose -f docker-compose.yml up -d
```
Then actinia runs at 'http://127.0.0.1:8088' and depending on your server settings might be accessible from outside. Because of this the start.sh is overwritten to not create any user to avoid security vulnerability. You will have to use a clean redis database to avoid stored actinia credentials from previous runs. You have to create the user by yourself by using the built-in actinia-user cli. __Please change below username (-u) and password (-w)__:
Then actinia runs at 'http://127.0.0.1:8088' and depending on your server settings might be accessible from outside. Because of this the start.sh is overwritten to not create any user to avoid security vulnerability. You will have to use a clean key-value database to avoid stored actinia credentials from previous runs. You have to create the user by yourself by using the built-in actinia-user cli. __Please change below username (-u) and password (-w)__:
```
# list help about the cli tool:
docker-compose -f docker-compose.yml exec actinia \
Expand Down
2 changes: 1 addition & 1 deletion actinia-alpine/actinia.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ plugins = ["actinia_tiling_plugin", "actinia_statistic_plugin", "actinia_satelli
force_https_urls = True

[REDIS]
redis_server_url = redis
redis_server_url = valkey
redis_server_pw = pass
redis_resource_expire_time = 864001
worker_logfile = /actinia_core/workspace/tmp/actinia_worker.log
Expand Down
8 changes: 4 additions & 4 deletions actinia-dev/actinia.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ force_https_urls = False
# # attr_prefix = actinia

[REDIS]
redis_server_url = redis
redis_server_url = valkey
redis_server_pw = pass
redis_resource_expire_time = 864001
worker_logfile = /actinia_core/workspace/tmp/actinia_worker.log
Expand All @@ -28,19 +28,19 @@ queue_type = local
# [QUEUE]
# queue_type = redis
# number_of_workers = 3
# redis_queue_server_url = redis-queue
# redis_queue_server_url = valkey-queue
# redis_queue_server_password = pass
# worker_queue_prefix = job_queue

# [QUEUE]
# queue_type = per_job
# redis_queue_server_url = redis-queue
# redis_queue_server_url = valkey-queue
# redis_queue_server_password = pass
# worker_queue_prefix = job_queue

# [QUEUE]
# queue_type = per_user
# redis_queue_server_url = redis-queue
# redis_queue_server_url = valkey-queue
# redis_queue_server_password = pass
# worker_queue_prefix = job_queue

Expand Down
2 changes: 1 addition & 1 deletion actinia-prod/actinia.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ plugins = ["actinia_statistic_plugin"]
force_https_urls = False

[REDIS]
redis_server_url = redis
redis_server_url = valkey
redis_server_pw = pass
worker_logfile = /actinia_core/workspace/tmp/actinia_worker.log

Expand Down
4 changes: 0 additions & 4 deletions actinia-ubuntu/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,6 @@ RUN apt-get update && apt-get upgrade -y && \
python3-magic \
python3-pip \
python3-ply \
redis-server \
redis-tools \
rsync \
subversion \
unzip \
Expand Down Expand Up @@ -97,8 +95,6 @@ RUN python3 -m pip install --user --upgrade setuptools wheel
RUN python3 setup.py sdist bdist_wheel -d /build
RUN pip3 install /build/*

## install (Cython must be installed before requirements.txt)
#RUN pip3 install Cython cython-setuptools setuptools_cython
RUN pip3 install scikit-learn && pip3 install .
## TODO: fix tests
#\
Expand Down
2 changes: 1 addition & 1 deletion actinia-ubuntu/actinia.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ plugins = ["actinia_statistic_plugin"]
force_https_urls = False

[REDIS]
redis_server_url = redis
redis_server_url = valkey
redis_server_pw = pass
redis_resource_expire_time = 864001
worker_logfile = /actinia_core/workspace/tmp/actinia_worker.log
Expand Down
14 changes: 7 additions & 7 deletions docker-compose-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,24 +22,24 @@ services:
ports:
- "8088:8088"
depends_on:
- redis
- valkey
cap_add:
- SYS_PTRACE
networks:
- actinia-dev

redis:
image: redis:5.0.4-alpine
valkey:
image: valkey/valkey:8.0-alpine
volumes:
- ./redis_data:/data
- ./valkey_data:/data
environment:
- REDIS_PASS_FILE=/data/config/.redis
- VALKEY_PASS_FILE=/data/config/.valkey
command: [
"sh", "-c",
'
docker-entrypoint.sh
"/data/config/redis.conf"
--requirepass "$$(cat $$REDIS_PASS_FILE)"
"/data/config/valkey.conf"
--requirepass "$$(cat $$VALKEY_PASS_FILE)"
'
]
ports:
Expand Down
14 changes: 7 additions & 7 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,19 @@ services:
ports:
- "8088:8088"
depends_on:
- redis
- valkey

redis:
image: redis:5.0.4-alpine
valkey:
image: valkey/valkey:8.0-alpine
volumes:
- ./redis_data:/data
- ./valkey_data:/data
environment:
- REDIS_PASS_FILE=/data/config/.redis
- VALKEY_PASS_FILE=/data/config/.valkey
command: [
"sh", "-c",
'
docker-entrypoint.sh
"/data/config/redis.conf"
--requirepass "$$(cat $$REDIS_PASS_FILE)"
"/data/config/valkey.conf"
--requirepass "$$(cat $$VALKEY_PASS_FILE)"
'
]
4 changes: 2 additions & 2 deletions docker-swarm.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Put this node into the docker swarm
docker swarm init --advertise-addr docker0

# Create the overlay network that connects the actinia container with the redis container
# Create the overlay network that connects the actinia container with the key-value database container
docker network create \
--driver overlay \
--subnet 172.20.128.0/24 \
Expand All @@ -21,7 +21,7 @@ docker stack deploy -c docker-swarm.yml actinia_swarm
docker service ls
# List infos about each docker run of the swarm
docker service ps actinia_swarm_actinia
docker service ps actinia_swarm_redis
docker service ps actinia_swarm_valkey

# Remove the stack
# outcommented, otherwise this script is useless ;)
Expand Down
14 changes: 7 additions & 7 deletions docker-swarm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ services:
networks:
- backend
depends_on:
- redis
- valkey

redis:
image: redis:5.0.4-alpine
valkey:
image: valkey/valkey:8.0-alpine
deploy:
replicas: 1
resources:
Expand All @@ -37,15 +37,15 @@ services:
placement:
constraints: [node.role == manager]
volumes:
- ./redis_data:/data
- ./valkey_data:/data
environment:
- REDIS_PASS_FILE=/data/config/.redis
- VALKEY_PASS_FILE=/data/config/.valkey
command: [
"sh", "-c",
'
docker-entrypoint.sh
"/data/config/redis.conf"
--requirepass "$$(cat $$REDIS_PASS_FILE)"
"/data/config/valkey.conf"
--requirepass "$$(cat $$VALKEY_PASS_FILE)"
'
]
networks:
Expand Down
Loading