Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: Failure on querying dependency graph #370

Open
jotelha opened this issue Nov 12, 2024 · 3 comments
Open

BUG: Failure on querying dependency graph #370

jotelha opened this issue Nov 12, 2024 · 3 comments
Assignees
@jotelha

Comments

@jotelha
Copy link
Member

jotelha commented Nov 12, 2024

2024-11-12 09:55:52,768:dtool_lookup_gui.utils.dependency_graph:ERROR: 500, message='Attempt to decode JSON with unexpected mimetype: text/html', url='https://simdata.vm.uni-freiburg.de/lookup/graph/uuids/0005ee33-5094-4bcf-aea6-2627cade781d?page=1&page_size=10&sort=uri'
@jotelha
Copy link
Member Author

jotelha commented Nov 12, 2024

This is a server-side issue,

<html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

@jotelha jotelha self-assigned this Nov 12, 2024
@jotelha
Copy link
Member Author

jotelha commented Nov 12, 2024

Server logs point to internal timeout:

dserver-default-ports-dserver-1  | [2024-11-12 09:01:27 +0000] [36] [CRITICAL] WORKER TIMEOUT (pid:2202)
dserver-default-ports-dserver-1  | DEBUG:pymongo.connection:{"clientId": {"$oid": "673318b7b55ba2a32494875c"}, "message": "Connection checked in", "serverHost": "mongodb", "serverPort": 27017, "driverConnectionId": 1}
dserver-default-ports-dserver-1  | DEBUG:pymongo.connection:{"clientId": {"$oid": "673318b7b55ba2a32494875c"}, "message": "Connection closed", "serverHost": "mongodb", "serverPort": 27017, "driverConnectionId": 1, "reason": "An error occurred while using the connection", "error": "error"}
dserver-default-ports-dserver-1  | [2024-11-12 09:01:27 +0000] [2202] [ERROR] Error handling request /lookup/graph/uuids/0005ee33-5094-4bcf-aea6-2627cade781d?page=1&page_size=10
dserver-default-ports-dserver-1  | Traceback (most recent call last):
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/gunicorn/workers/sync.py", line 134, in handle
dserver-default-ports-dserver-1  |     self.handle_request(listener, req, client, addr)
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/gunicorn/workers/sync.py", line 177, in handle_request
dserver-default-ports-dserver-1  |     respiter = self.wsgi(environ, resp.start_response)
dserver-default-ports-dserver-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/app/wsgi.py", line 53, in __call__
dserver-default-ports-dserver-1  |     return self._app(env, log_response)
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 2213, in __call__
dserver-default-ports-dserver-1  |     return self.wsgi_app(environ, start_response)
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 2190, in wsgi_app
dserver-default-ports-dserver-1  |     response = self.full_dispatch_request()
dserver-default-ports-dserver-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 1484, in full_dispatch_request
dserver-default-ports-dserver-1  |     rv = self.dispatch_request()
dserver-default-ports-dserver-1  |          ^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 1469, in dispatch_request
dserver-default-ports-dserver-1  |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/flask_smorest/response.py", line 89, in wrapper
dserver-default-ports-dserver-1  |     func(*args, **kwargs)
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/flask_smorest/pagination.py", line 200, in wrapper
dserver-default-ports-dserver-1  |     result, status, headers = unpack_tuple_response(func(*args, **kwargs))
dserver-default-ports-dserver-1  |                                                     ^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/flask_jwt_extended/view_decorators.py", line 170, in decorator
dserver-default-ports-dserver-1  |     return current_app.ensure_sync(fn)(*args, **kwargs)
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/dserver_dependency_graph_plugin/__init__.py", line 276, in lookup_dependency_graph_by_default_keys
dserver-default-ports-dserver-1  |     datasets = dependency_graph_by_user_and_uuid(username, uuid)
dserver-default-ports-dserver-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/dserver_dependency_graph_plugin/__init__.py", line 251, in dependency_graph_by_user_and_uuid
dserver-default-ports-dserver-1  |     cx = DependencyGraphExtension.db[current_app.config['MONGO_COLLECTION']].aggregate(mongo_aggregation)
dserver-default-ports-dserver-1  |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/collection.py", line 2958, in aggregate
dserver-default-ports-dserver-1  |     return self._aggregate(
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/_csot.py", line 119, in csot_wrapper
dserver-default-ports-dserver-1  |     return func(self, *args, **kwargs)
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/collection.py", line 2866, in _aggregate
dserver-default-ports-dserver-1  |     return self._database.client._retryable_read(
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/mongo_client.py", line 1863, in _retryable_read
dserver-default-ports-dserver-1  |     return self._retry_internal(
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/_csot.py", line 119, in csot_wrapper
dserver-default-ports-dserver-1  |     return func(self, *args, **kwargs)
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/mongo_client.py", line 1830, in _retry_internal
dserver-default-ports-dserver-1  |     ).run()
dserver-default-ports-dserver-1  |       ^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/mongo_client.py", line 2554, in run
dserver-default-ports-dserver-1  |     return self._read() if self._is_read else self._write()
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/mongo_client.py", line 2697, in _read
dserver-default-ports-dserver-1  |     return self._func(self._session, self._server, conn, read_pref)  # type: ignore
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/aggregation.py", line 164, in get_cursor
dserver-default-ports-dserver-1  |     result = conn.command(
dserver-default-ports-dserver-1  |              ^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/helpers.py", line 45, in inner
dserver-default-ports-dserver-1  |     return func(*args, **kwargs)
dserver-default-ports-dserver-1  |            ^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/pool.py", line 566, in command
dserver-default-ports-dserver-1  |     self._raise_connection_failure(error)
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/pool.py", line 538, in command
dserver-default-ports-dserver-1  |     return command(
dserver-default-ports-dserver-1  |            ^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/network.py", line 208, in command
dserver-default-ports-dserver-1  |     reply = receive_message(conn, request_id)
dserver-default-ports-dserver-1  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/network.py", line 320, in receive_message
dserver-default-ports-dserver-1  |     length, _, response_to, op_code = _UNPACK_HEADER(_receive_data_on_socket(conn, 16, deadline))
dserver-default-ports-dserver-1  |                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/network.py", line 390, in _receive_data_on_socket
dserver-default-ports-dserver-1  |     wait_for_read(conn, deadline)
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/synchronous/network.py", line 375, in wait_for_read
dserver-default-ports-dserver-1  |     readable = conn.socket_checker.select(sock, read=True, timeout=timeout)
dserver-default-ports-dserver-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/pymongo/socket_checker.py", line 66, in select
dserver-default-ports-dserver-1  |     res = self._poller.poll(timeout_)
dserver-default-ports-dserver-1  |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
dserver-default-ports-dserver-1  |   File "/usr/local/lib/python3.12/site-packages/gunicorn/workers/base.py", line 204, in handle_abort
dserver-default-ports-dserver-1  |     sys.exit(1)
dserver-default-ports-dserver-1  | SystemExit: 1

@jotelha
Copy link
Member Author

jotelha commented Nov 12, 2024 •
edited
Loading

The database logs point to the dserver app being to impatient, no failure here

dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:06:33.893+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:40974","uuid":{"uuid":{"$uuid":"5d3fa78f-c100-43a1-a216-ebaca72b3a19"}},"connectionId":1306,"connectionCount":21}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:06:33.893+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1306","msg":"client metadata","attr":{"remote":"172.29.0.4:40974","client":"conn1306","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:06:33.894+00:00"},"s":"I",  "c":"NETWORK",  "id":6788700, "ctx":"conn1306","msg":"Received first command on ingress connection since session start or auth handshake","attr":{"elapsedMillis":1}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.275+00:00"},"s":"I",  "c":"-",        "id":20883,   "ctx":"conn1297","msg":"Interrupted operation as its client disconnected","attr":{"opId":2636834}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.275+00:00"},"s":"I",  "c":"-",        "id":20883,   "ctx":"conn1299","msg":"Interrupted operation as its client disconnected","attr":{"opId":2638882}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.275+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1297","msg":"Connection ended","attr":{"remote":"172.29.0.4:38308","uuid":{"uuid":{"$uuid":"60740fb3-2912-4b42-9c30-8cc55860bd9c"}},"connectionId":1297,"connectionCount":20}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.280+00:00"},"s":"I",  "c":"-",        "id":20883,   "ctx":"conn1296","msg":"Interrupted operation as its client disconnected","attr":{"opId":2637858}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.280+00:00"},"s":"I",  "c":"-",        "id":20883,   "ctx":"conn1300","msg":"Interrupted operation as its client disconnected","attr":{"opId":2640930}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.280+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1299","msg":"Connection ended","attr":{"remote":"172.29.0.4:38312","uuid":{"uuid":{"$uuid":"4859a9ac-26df-46c4-ae8f-5ec6481b8711"}},"connectionId":1299,"connectionCount":19}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.280+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1296","msg":"Connection ended","attr":{"remote":"172.29.0.4:38306","uuid":{"uuid":{"$uuid":"2ad3f76e-6d73-44ce-aad2-26c586abf204"}},"connectionId":1296,"connectionCount":18}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.288+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1300","msg":"Connection ended","attr":{"remote":"172.29.0.4:38314","uuid":{"uuid":{"$uuid":"e4813ad6-6722-456b-87ab-cf56deb12ca1"}},"connectionId":1300,"connectionCount":17}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.345+00:00"},"s":"W",  "c":"NETWORK",  "id":4615610, "ctx":"conn1306","msg":"Failed to check socket connectivity","attr":{"error":{"code":6,"codeName":"HostUnreachable","errmsg":"peekASIOStream :: caused by :: Connection closed by peer"}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.345+00:00"},"s":"I",  "c":"-",        "id":20883,   "ctx":"conn1306","msg":"Interrupted operation as its client disconnected","attr":{"opId":2649094}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.345+00:00"},"s":"W",  "c":"QUERY",    "id":23799,   "ctx":"conn1306","msg":"Aggregate command executor error","attr":{"error":{"code":279,"codeName":"ClientDisconnect","errmsg":"operation was interrupted because a client disconnected"},"stats":{},"cmd":{"aggregate":"metadata","pipeline":[{"$match":{"$and":[{"base_uri":"s3://frct-simdata"},{"uuid":"0005ee33-5094-4bcf-aea6-2627cade781d"}]}},{"$graphLookup":{"from":"dep:2024-11-11T12:21:19.510741","startWith":"$uuid","connectFromField":"dependencies","connectToField":"uuid","as":"dependency_graph"}},{"$unwind":"$dependency_graph"},{"$replaceRoot":{"newRoot":"$dependency_graph"}},{"$lookup":{"from":"metadata","localField":"uuid","foreignField":"uuid","as":"dataset"}},{"$unwind":"$dataset"},{"$replaceRoot":{"newRoot":"$dataset"}},{"$match":{"base_uri":"s3://frct-simdata"}},{"$facet":{"key0":[{"$unwind":{"path":"$readme","preserveNullAndEmptyArrays":true}},{"$unwind":{"path":"$readme.derived_from","preserveNullAndEmptyArrays":true}},{"$unwind":{"path":"$readme.derived_from.uuid","preserveNullAndEmptyArrays":true}},{"$project":{"_id":false}},{"$addFields":{"derived_from":"$readme.derived_from.uuid"}}],"key1":[{"$unwind":{"path":"$annotations","preserveNullAndEmptyArrays":true}},{"$unwind":{"path":"$annotations.source_dataset_uuid","preserveNullAndEmptyArrays":true}},{"$project":{"_id":false}},{"$addFields":{"derived_from":"$annotations.source_dataset_uuid"}}]}},{"$project":{"nested":{"$concatArrays":["$key0","$key1"]}}},{"$unwind":{"path":"$nested","preserveNullAndEmptyArrays":true}},{"$group":{"_id":"$nested.uuid","derived_from":{"$push":"$nested.derived_from"},"nested":{"$mergeObjects":"$nested"}}},{"$addFields":{"nested.uuid":"$_id","nested.derived_from":"$derived_from"}},{"$replaceRoot":{"newRoot":"$nested"}},{"$sort":{"uuid":1}},{"$project":{"_id":false,"readme":false,"manifest":false,"annotations":false}}],"cursor":{},"lsid":{"id":{"$uuid":"9a895b20-9f6f-44b6-9933-dce38c8bc3a1"}},"$db":"dtool_info"}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.360+00:00"},"s":"W",  "c":"COMMAND",  "id":20526,   "ctx":"conn1306","msg":"Failed to gather storage statistics for slow operation","attr":{"opId":2649094,"error":"ClientDisconnect: operation was interrupted"}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.360+00:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn1306","msg":"Slow query","attr":{"type":"command","isFromUserConnection":true,"ns":"dtool_info.metadata","collectionType":"normal","queryShapeHash":"E7D1986E1686B8846CDAA4831FC10FAE168D928F4E154502547613B48E5E9FD1","command":{"aggregate":"metadata","pipeline":[{"$match":{"$and":[{"base_uri":"s3://frct-simdata"},{"uuid":"0005ee33-5094-4bcf-aea6-2627cade781d"}]}},{"$graphLookup":{"from":"dep:2024-11-11T12:21:19.510741","startWith":"$uuid","connectFromField":"dependencies","connectToField":"uuid","as":"dependency_graph"}},{"$unwind":"$dependency_graph"},{"$replaceRoot":{"newRoot":"$dependency_graph"}},{"$lookup":{"from":"metadata","localField":"uuid","foreignField":"uuid","as":"dataset"}},{"$unwind":"$dataset"},{"$replaceRoot":{"newRoot":"$dataset"}},{"$match":{"base_uri":"s3://frct-simdata"}},{"$facet":{"key0":[{"$unwind":{"path":"$readme","preserveNullAndEmptyArrays":true}},{"$unwind":{"path":"$readme.derived_from","preserveNullAndEmptyArrays":true}},{"$unwind":{"path":"$readme.derived_from.uuid","preserveNullAndEmptyArrays":true}},{"$project":{"_id":false}},{"$addFields":{"derived_from":"$readme.derived_from.uuid"}}],"key1":[{"$unwind":{"path":"$annotations","preserveNullAndEmptyArrays":true}},{"$unwind":{"path":"$annotations.source_dataset_uuid","preserveNullAndEmptyArrays":true}},{"$project":{"_id":false}},{"$addFields":{"derived_from":"$annotations.source_dataset_uuid"}}]}},{"$project":{"nested":{"$concatArrays":["$key0","$key1"]}}},{"$unwind":{"path":"$nested","preserveNullAndEmptyArrays":true}},{"$group":{"_id":"$nested.uuid","derived_from":{"$push":"$nested.derived_from"},"nested":{"$mergeObjects":"$nested"}}},{"$addFields":{"nested.uuid":"$_id","nested.derived_from":"$derived_from"}},{"$replaceRoot":{"newRoot":"$nested"}},{"$sort":{"uuid":1}},{"$project":{"_id":false,"readme":false,"manifest":false,"annotations":false}}],"cursor":{},"lsid":{"id":{"$uuid":"9a895b20-9f6f-44b6-9933-dce38c8bc3a1"}},"$db":"dtool_info"},"planSummary":"COLLSCAN","planningTimeMicros":162,"resolvedViews":[{"viewNamespace":"dtool_info.dep:2024-11-11T12:21:19.510741","dependencyChain":["dep:2024-11-11T12:21:19.510741","metadata"],"resolvedPipeline":[{"$facet":{"key0":[{"$unwind":{"path":"$annotations","preserveNullAndEmptyArrays":true}},{"$unwind":{"path":"$annotations.source_dataset_uuid","preserveNullAndEmptyArrays":true}},{"$project":{"_id":false,"uuid":true,"derived_from":"$annotations.source_dataset_uuid"}}],"key1":[{"$unwind":{"path":"$readme","preserveNullAndEmptyArrays":true}},{"$unwind":{"path":"$readme.derived_from","preserveNullAndEmptyArrays":true}},{"$unwind":{"path":"$readme.derived_from.uuid","preserveNullAndEmptyArrays":true}},{"$project":{"_id":false,"uuid":true,"derived_from":"$readme.derived_from.uuid"}}]}},{"$project":{"dependencies":{"$concatArrays":["$key0","$key1"]}}},{"$unwind":{"path":"$dependencies","preserveNullAndEmptyArrays":true}},{"$replaceRoot":{"newRoot":"$dependencies"}},{"$facet":{"derived_from":[{"$group":{"_id":"$uuid","dependencies":{"$push":"$derived_from"}}},{"$project":{"_id":false,"uuid":"$_id","dependencies":true}}],"derivative":[{"$group":{"_id":"$derived_from","dependencies":{"$push":"$uuid"}}},{"$project":{"_id":false,"uuid":"$_id","dependencies":true}}]}},{"$project":{"dependencies":{"$concatArrays":["$derived_from","$derivative"]}}},{"$unwind":{"path":"$dependencies","preserveNullAndEmptyArrays":true}},{"$replaceRoot":{"newRoot":"$dependencies"}},{"$match":{"uuid":{"$regex":"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[4][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}"}}},{"$group":{"_id":"$uuid","dependencies":{"$push":"$dependencies"}}},{"$project":{"_id":false,"uuid":"$_id","dependencies":true}},{"$unwind":{"path":"$dependencies","preserveNullAndEmptyArrays":true}},{"$unwind":{"path":"$dependencies","preserveNullAndEmptyArrays":true}},{"$group":{"_id":"$uuid","dependencies":{"$push":"$dependencies"}}},{"$project":{"_id":true,"uuid":"$_id","dependencies":true}}]}],"numYields":134,"queryHash":"7226AF80","planCacheKey":"064634E0","queryFramework":"classic","ok":0,"errMsg":"PlanExecutor error during aggregation :: caused by :: operation was interrupted because a client disconnected","errName":"ClientDisconnect","errCode":279,"reslen":201,"locks":{"Global":{"acquireCount":{"r":4846}}},"cpuNanos":27349770651,"remote":"172.29.0.4:40974","protocol":"op_msg","queues":{"ingress":{"admissions":1},"execution":{"admissions":4847}},"workingMillis":30409,"durationMillis":30409}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.382+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1306","msg":"Connection ended","attr":{"remote":"172.29.0.4:40974","uuid":{"uuid":{"$uuid":"5d3fa78f-c100-43a1-a216-ebaca72b3a19"}},"connectionId":1306,"connectionCount":16}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.557+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1304","msg":"Connection ended","attr":{"remote":"172.29.0.4:38322","uuid":{"uuid":{"$uuid":"b92e04bc-a7e3-4fc9-937d-d79156082187"}},"connectionId":1304,"connectionCount":15}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.557+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1301","msg":"Connection ended","attr":{"remote":"172.29.0.4:38316","uuid":{"uuid":{"$uuid":"ac51a677-d61e-49f4-b647-c7fd2180fabb"}},"connectionId":1301,"connectionCount":14}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.557+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1303","msg":"Connection ended","attr":{"remote":"172.29.0.4:38320","uuid":{"uuid":{"$uuid":"160720f3-d4ba-4b1c-85a1-fd607be9ddc9"}},"connectionId":1303,"connectionCount":13}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.557+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1302","msg":"Connection ended","attr":{"remote":"172.29.0.4:38318","uuid":{"uuid":{"$uuid":"9eb9e6c5-d6af-4dd2-a74c-092b2ab04abc"}},"connectionId":1302,"connectionCount":12}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.558+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1305","msg":"Connection ended","attr":{"remote":"172.29.0.4:38324","uuid":{"uuid":{"$uuid":"1287eae2-2f0d-4978-9172-903262c2cd01"}},"connectionId":1305,"connectionCount":11}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.588+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41226","uuid":{"uuid":{"$uuid":"64b83776-7ff5-4d72-b33f-ce777240dc98"}},"connectionId":1307,"connectionCount":12}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.589+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1307","msg":"client metadata","attr":{"remote":"172.29.0.4:41226","client":"conn1307","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.591+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41228","uuid":{"uuid":{"$uuid":"9bd1a7df-1935-4d13-a846-62c5a368851c"}},"connectionId":1308,"connectionCount":13}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.591+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41230","uuid":{"uuid":{"$uuid":"c87b8a0d-99b9-4c2c-af35-c6f3aa7852c7"}},"connectionId":1309,"connectionCount":14}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.591+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1308","msg":"client metadata","attr":{"remote":"172.29.0.4:41228","client":"conn1308","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.591+00:00"},"s":"I",  "c":"NETWORK",  "id":6788700, "ctx":"conn1308","msg":"Received first command on ingress connection since session start or auth handshake","attr":{"elapsedMillis":0}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.596+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41232","uuid":{"uuid":{"$uuid":"bde81de2-5247-4e02-b6cf-475d60290f67"}},"connectionId":1310,"connectionCount":15}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.597+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41234","uuid":{"uuid":{"$uuid":"4c107f6c-2e05-478c-ab99-4ad5c25c1e83"}},"connectionId":1311,"connectionCount":16}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.597+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1309","msg":"client metadata","attr":{"remote":"172.29.0.4:41230","client":"conn1309","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.597+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41236","uuid":{"uuid":{"$uuid":"8806ace6-177e-4388-a339-0e2ab45e0082"}},"connectionId":1312,"connectionCount":17}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.597+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1310","msg":"client metadata","attr":{"remote":"172.29.0.4:41232","client":"conn1310","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.598+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1312","msg":"client metadata","attr":{"remote":"172.29.0.4:41236","client":"conn1312","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.598+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1311","msg":"client metadata","attr":{"remote":"172.29.0.4:41234","client":"conn1311","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.601+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41238","uuid":{"uuid":{"$uuid":"d26a2913-844a-4942-906e-883f4604f5a6"}},"connectionId":1313,"connectionCount":18}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.601+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41240","uuid":{"uuid":{"$uuid":"53f4eda0-a7dd-4308-8e8c-c377c68af402"}},"connectionId":1314,"connectionCount":19}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.601+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41242","uuid":{"uuid":{"$uuid":"69bebb24-dfc0-49de-bf1c-cb377aa5522c"}},"connectionId":1315,"connectionCount":20}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.602+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41244","uuid":{"uuid":{"$uuid":"d2aac818-9a2a-4b2e-96cf-704a70fc0389"}},"connectionId":1316,"connectionCount":21}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.602+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1313","msg":"client metadata","attr":{"remote":"172.29.0.4:41238","client":"conn1313","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.602+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1314","msg":"client metadata","attr":{"remote":"172.29.0.4:41240","client":"conn1314","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.602+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1316","msg":"client metadata","attr":{"remote":"172.29.0.4:41244","client":"conn1316","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.603+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1315","msg":"client metadata","attr":{"remote":"172.29.0.4:41242","client":"conn1315","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.604+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41246","uuid":{"uuid":{"$uuid":"b10d2314-544e-4f18-b616-dcff81c2988d"}},"connectionId":1317,"connectionCount":22}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.605+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41248","uuid":{"uuid":{"$uuid":"c43ded37-3c63-443a-8867-ff0b8ad26a97"}},"connectionId":1318,"connectionCount":23}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.605+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1308","msg":"Connection ended","attr":{"remote":"172.29.0.4:41228","uuid":{"uuid":{"$uuid":"9bd1a7df-1935-4d13-a846-62c5a368851c"}},"connectionId":1308,"connectionCount":22}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.605+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1317","msg":"client metadata","attr":{"remote":"172.29.0.4:41246","client":"conn1317","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.606+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1318","msg":"client metadata","attr":{"remote":"172.29.0.4:41248","client":"conn1318","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.608+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41250","uuid":{"uuid":{"$uuid":"7816fc24-68c3-42bb-86bf-7d577c8ea617"}},"connectionId":1319,"connectionCount":23}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.608+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1319","msg":"client metadata","attr":{"remote":"172.29.0.4:41250","client":"conn1319","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.608+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.29.0.4:41252","uuid":{"uuid":{"$uuid":"52c2ce69-caca-406e-b6c2-c72be8f84262"}},"connectionId":1320,"connectionCount":24}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.609+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn1320","msg":"client metadata","attr":{"remote":"172.29.0.4:41252","client":"conn1320","negotiatedCompressors":[],"doc":{"driver":{"name":"PyMongo|c","version":"4.10.1"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"4.18.0-240.22.1.el8_3.x86_64"},"platform":"CPython 3.12.4.final.0","env":{"container":{"runtime":"docker"}}}}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:05.609+00:00"},"s":"I",  "c":"NETWORK",  "id":6788700, "ctx":"conn1319","msg":"Received first command on ingress connection since session start or auth handshake","attr":{"elapsedMillis":0}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:06.091+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1309","msg":"Connection ended","attr":{"remote":"172.29.0.4:41230","uuid":{"uuid":{"$uuid":"c87b8a0d-99b9-4c2c-af35-c6f3aa7852c7"}},"connectionId":1309,"connectionCount":23}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:06.091+00:00"},"s":"I",  "c":"-",        "id":20883,   "ctx":"conn1307","msg":"Interrupted operation as its client disconnected","attr":{"opId":2652161}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:06.092+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1307","msg":"Connection ended","attr":{"remote":"172.29.0.4:41226","uuid":{"uuid":{"$uuid":"64b83776-7ff5-4d72-b33f-ce777240dc98"}},"connectionId":1307,"connectionCount":22}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:06.101+00:00"},"s":"I",  "c":"-",        "id":20883,   "ctx":"conn1310","msg":"Interrupted operation as its client disconnected","attr":{"opId":2655233}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:06.101+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1310","msg":"Connection ended","attr":{"remote":"172.29.0.4:41232","uuid":{"uuid":{"$uuid":"bde81de2-5247-4e02-b6cf-475d60290f67"}},"connectionId":1310,"connectionCount":21}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:06.101+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn1315","msg":"Connection ended","attr":{"remote":"172.29.0.4:41242","uuid":{"uuid":{"$uuid":"69bebb24-dfc0-49de-bf1c-cb377aa5522c"}},"connectionId":1315,"connectionCount":20}}
dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:19.945+00:00"},"s":"I",  "c":"WTCHKPT",  "id":22430,   "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":{"ts_sec":1731402439,"ts_usec":945667,"thread":"1:0x7f4c55d136c0","session_name":"WT_SESSION.checkpoint","category":"WT_VERB_CHECKPOINT_PROGRESS","category_id":7,"verbose_level":"DEBUG_1","verbose_level_id":1,"msg":"saving checkpoint snapshot min: 54237, snapshot max: 54237 snapshot count: 0, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0) base write gen: 1"}}}

See line

dserver-default-ports-mongodb-1  | {"t":{"$date":"2024-11-12T09:07:04.345+00:00"},"s":"W",  "c":"QUERY",    "id":23799,   "ctx":"conn1306","msg":"Aggregate command executor error","attr":{"error":{"code":279,"codeName":"ClientDisconnect","errmsg":"operation was interrupted because a client disconnected"},"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jotelha jotelha

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jotelha