Unclear how to revert changes after attempting to add Nvidia gpu

[orrinwitt]

Steps to reproduce

1. Starting from a working installation, attempted to add nvidia gpu by adding NEXTCLOUD_ENABLE_NVIDIA_GPU=TRUE to env variables in compose.yaml file.

Expected behavior

All containers start up, with gpu passthrough enabled for apps or containers that could use it.

Actual behavior

1. nextcloud container fails to start (and therefore some others as well.)
2. In mastercontainer logs, I get {"message":"unknown or invalid runtime: nvidia"} (this makes sense, as it's the only thing I had changed.

Other information

Host OS

Truenas SCALE, electric eel 24.10.1

Output of sudo docker info

Client: Docker Engine - Community
 Version:    27.1.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 20
  Running: 13
  Paused: 0
  Stopped: 7
 Images: 33
 Server Version: 27.1.1
 Storage Driver: overlay2
  Backing Filesystem: zfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.6.44-production+truenas
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 20
 Total Memory: 94.25GiB
 Name: truenas
 ID: 02a0e34c-1b9c-46c2-aaf6-ab89967da331
 Docker Root Dir: /mnt/.ix-apps/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.16.0.0/12, Size: 24

Docker run command or docker-compose file that you used

networks: {}
services:
  nextcloud:
    container_name: nextcloud-aio-mastercontainer
    environment:
      - APACHE_PORT=11000
      - APACHE_IP_BINDING=0.0.0.0
      - NEXTCLOUD_DATADIR=/mnt/tank/nextcloud_data/
      - NEXTCLOUD_MEMORY_LIMIT=8192M
      - NEXTCLOUD_MAX_TIME=36000
      - SKIP_DOMAIN_VALIDATION=true
      - NEXTCLOUD_ENABLE_NVIDIA_GPU=true <---(this was added, then later removed)
    image: nextcloud/all-in-one:latest
    ports:
      - '8080:8080'
    restart: always
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /var/run/docker.sock:/var/run/docker.sock:ro
volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer

Other valuable info

I have tried completely removing and recreating the mastercontainer, but the nvidia error persists. I'd like to know where else that change has been written so as to remove it completely.

[orrinwitt]

I believe I found the answer! It appears the configs in the yaml file get written to configuration.json in this location:
/mnt/.ix-apps/docker/volumes/nextcloud_aio_mastercontainer/_data/data

I thought I had removed the volumes associated with the mastercontainer, which would have fixed it, but apparently I had not.

Anyway, at the very bottom of that file was the NEXTCLOUD_ENABLE_NVIDIA_GPU=true entry, which I simply deleted, saved the file, and was off to the races. One small hiccup was that I apparently needed to also delete the trailing comma from the line above it as well.