new pgadmin 7.8 in namespace scoped mode gives error ERROR\tpgadmin:\t'pinged'

[mzwettler2]

PGO: postgres-operator:ubi8-5.5.0-0
PGADMIN: crunchy-pgadmin4:ubi8-7.8-0
POSTGRES: crunchy-postgres:ubi8-16.1-0

We deployed pgadmin in namespace scoped mode.

We exposed pgadmin using an virtualservice and istio service mesh.

We get a bunch of this error messages: ERROR\tpgadmin:\t'pinged'

Any idea?

---

ci_id: "9200005945"
ci_name: ""
swid: "cmpgdb"
instance: "dev003"
stage: "dev"
k8s-cluster: "szh-svc-01-nonprod"
namespace: "cmpgdb-pg-dev003"
pod: "pgadmin-2051351b-a3d9-4a1a-a620-5d76c2b035d2-0"
container: "pgadmin"
[pg-cluster]: ""
[pg-role]: "pgadmin"
message: "2024-05-23 14:10:23,191: ERROR\tpgadmin:\t'pinged'"

---

ci_id: "9200005945"
ci_name: ""
swid: "cmpgdb"
instance: "dev003"
stage: "dev"
k8s-cluster: "szh-svc-01-nonprod"
namespace: "cmpgdb-pg-dev003"
pod: "pgadmin-2051351b-a3d9-4a1a-a620-5d76c2b035d2-0"
container: "pgadmin"
[pg-cluster]: ""
[pg-role]: "pgadmin"
message: "2024-05-23 14:05:23,382: ERROR\tpgadmin:\t'pinged'"

[benjaminjb]

Hi @mzwettler2, sorry you're running into this. I'm afraid I've never seen this particular error before.

Just to make sure I understand the situation, the pgadmin pod is running fine? If so, can you hit it directly from inside the k8s cluster?

And the way that you've exposed it (virtualservice and istio service mesh) is something that's working fine for other pods in the namespace?

[mzwettler2]

@benjaminjb

Two times yes. I suspect it is related to istio. I am just surprised because the old, cluster-scoped version 4.6 has no problems.

[benjaminjb]

Hey @mzwettler2, would you be able to change from crunchy-pgadmin4:ubi8-7.8-0 to crunchy-pgadmin4:ubi8-8.6-1?

Actually, before you do, I want to point to some changes we made between those two images: https://access.crunchydata.com/documentation/postgres-operator/latest/releases/5.6.x#560

(IMHO, the biggest change that might impact people moving from the 7.8 image to a 8.6 image is probably user management, so here's the relevant doc: https://access.crunchydata.com/documentation/postgres-operator/latest/guides/pgadmin/user-management. My quick summary of that change is we no longer automatically set up an admin user for you to manage your own users; now you can define users in the pgadmin CR (and set them as users or admins).)

OK, now that I've gone over the changes that might make the 8.6 experience different, I want to discuss the one change that I think might affect this weird error. One change we recently made was that we are now using gunicorn in the newer image (and gunicorn was what we were using in the older, cluster-scoped image). I am wondering if serving the app through gunicorn might help relieve whatever is happening between pgadmin and istio. (Though also: I don't know for sure that that's where the issue is coming from, but like you, I suspect it is.)

[benjaminjb]

Hey @mzwettler2, just realized -- I have never run an update from 7.8 to 8.6, so I do not know how that will go. Rather than deal with that now, you could create a new pgadmin using the 8.6 image and see if that clears up that noise. If it does, then we can move on to figuring out a best practice for migrating. (Someone else may have that experience, I just wanted to note that I haven't done that yet.)

[mzwettler2]

@benjaminjb

I am not able to update from 7.8 to 8.6 right now. But I will keep this in mind.

Regarding the new PGAdmin user spec. Cool feature. Thanks. But I have one small criticism. The secrets for the database users are created by pgo automatically. The secrets for the PGAdmin users have to be created manually. That's a bit inconsistent, isn't it?

[benjaminjb]

Hey @mzwettler2, circling back to this issue now -- were you ever able to update the pgadmin version?

(As for the pgadmin user secrets, I hear you on the inconsistency. I wonder if PGO should accept user-defined passwords OR (if not present), create its own passwords/secrets. Let me talk to the team about that.)

[mzwettler2]

@benjaminjb

Sorry. I were not able to upgrade to 8.6 right now. I will try this in Q12025.

PGO should create its own secrets if they do not exist. Anyone can patch these secrets afterwards to get custom passwords. Automation is easier if PGO creates its own passwords.