i have issue with ml-pipeline pods, for my kubeflow 1.9.0 and latest i am using tag 1.9.0, please see why ml-pipelines pods are not up & running getting these logs #11089
ravi-gundavaram
started this conversation in
General
Replies: 1 comment
-
|
Hi @neerfri |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
ubuntu@ip-10-11-1-60:
/eio-mlops-dev/manifests$ kubectl logs ml-pipeline-9cd449fd6-gvhtr -n kubeflow/eio-mlops-dev/manifests$ kubectl get pods -n kubeflowI0811 18:25:14.884098 7 client_manager.go:170] Initializing client manager
I0811 18:25:14.884220 7 config.go:57] Config DBConfig.MySQLConfig.ExtraParams not specified, skipping
[mysql] 2024/08/11 18:25:14 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:14 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:14 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:15 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:15 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:15 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:16 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:16 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:16 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:17 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:17 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:17 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:18 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:18 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:18 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:21 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:21 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:21 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:27 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:27 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:27 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:34 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:34 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:34 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:43 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:43 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:43 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:57 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:57 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:25:57 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:26:22 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:26:22 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:26:22 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:26:37 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:26:37 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:26:37 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:27:24 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:27:24 packets.go:37: unexpected EOF
[mysql] 2024/08/11 18:27:24 packets.go:37: unexpected EOF
ubuntu@ip-10-11-1-60:
NAME READY STATUS RESTARTS AGE
admission-webhook-deployment-6dfbf7c8c6-pvmdd 1/1 Running 0 36h
cache-server-58f5d8c7d5-gmmc6 2/2 Running 169 (9m35s ago) 36h
centraldashboard-d6f49bb67-7tr82 2/2 Running 0 36h
jupyter-web-app-deployment-bc78b48f8-ksh4m 2/2 Running 0 36h
katib-controller-754877f9f-62hj8 1/1 Running 0 36h
katib-db-manager-78cf5894bd-fgtls 1/1 Running 0 31h
katib-mysql-8fc5ccdd4-jr44b 1/1 Running 0 32h
katib-ui-858f447bfb-zxd5q 2/2 Running 0 36h
kserve-controller-manager-b96c76496-gkjmn 2/2 Running 0 36h
kserve-models-web-app-5d7d5857df-fcncr 2/2 Running 0 36h
kubeflow-pipelines-profile-controller-7795c68cfd-mk64r 1/1 Running 0 36h
metacontroller-0 1/1 Running 0 36h
metadata-db-7fd57b5949-5pxrf 1/1 Running 0 30h
metadata-envoy-deployment-74d7589fbd-frfmf 1/1 Running 0 31h
metadata-grpc-deployment-c8c7f488c-f7f6f 2/2 Running 1 (31h ago) 31h
metadata-writer-747b476588-tnjfx 2/2 Running 0 30h
minio-64c6fd469d-xrmcb 2/2 Running 0 102m
ml-pipeline-9cd449fd6-gvhtr 1/2 CrashLoopBackOff 13 (50s ago) 49m
ml-pipeline-persistenceagent-5679776bbf-gt79l 2/2 Running 0 36h
ml-pipeline-scheduledworkflow-b6658c7c-b8tk7 2/2 Running 0 36h
ml-pipeline-ui-577d569cc8-kgzwp 2/2 Running 0 36h
ml-pipeline-viewer-crd-56d7584db6-f5wfs 2/2 Running 1 (36h ago) 36h
ml-pipeline-visualizationserver-5f57d6b9cf-jkhcs 2/2 Running 0 36h
model-registry-db-7d88d94586-pfjqp 1/1 Running 0 32h
model-registry-deployment-7fc8896c9c-4k4mt 3/3 Running 0 31h
mysql-577cfcd74f-pzw97 2/2 Running 0 102m
notebook-controller-deployment-5d664bc767-5t5rw 2/2 Running 1 (36h ago) 36h
profiles-deployment-5f69674595-5pmvh 3/3 Running 1 (36h ago) 36h
pvcviewer-controller-manager-5b8487cb6c-vq8hc 3/3 Running 0 36h
tensorboard-controller-deployment-59d6c5dc44-szrcx 3/3 Running 1 (36h ago) 36h
tensorboards-web-app-deployment-5585954f8f-cjjqf 2/2 Running 0 36h
training-operator-78f4df6758-pgt7b 1/1 Running 0 36h
volumes-web-app-deployment-756c7fb9c7-7nzdc 2/2 Running 0 36h
workflow-controller-7c9c86b578-757db 2/2 Running 1 (36h ago) 36h
Beta Was this translation helpful? Give feedback.
All reactions