Skip to content
This repository has been archived by the owner on Nov 4, 2024. It is now read-only.

Stop going through artifact documents when querying CouchDB for several runs #263

Merged
merged 6 commits into from
Aug 15, 2024

Conversation

eamansour
Copy link
Member

@eamansour eamansour commented Aug 14, 2024

Why?

See galasa-dev/projectmanagement#1921

Currently, for each run found during a galasactl runs get or equivalent operation, the associated artifact document is being retrieved and each of the artifacts within that document are being processed. So the more runs there are to list, the more artifacts there are to go through as well - here's a snippet of the RAS logs during a runs get:

[notice] 2024-08-08T12:49:58.177687Z ... POST /galasa_run/_find 200 ok 25
[notice] 2024-08-08T12:49:58.184608Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65a07b6e 200 ok 5
[notice] 2024-08-08T12:49:58.188547Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65a8c5dc 200 ok 2
[notice] 2024-08-08T12:49:58.192407Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65936c88 200 ok 3
[notice] 2024-08-08T12:49:58.197171Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65a06e0b 200 ok 3
[notice] 2024-08-08T12:49:58.202133Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65a77d59 200 ok 3
[notice] 2024-08-08T12:49:58.205531Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65ab3aca 200 ok 2
[notice] 2024-08-08T12:49:58.209981Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65ad981e 200 ok 2
[notice] 2024-08-08T12:49:58.221534Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c6593ee5d 200 ok 10
[notice] 2024-08-08T12:49:58.227187Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65946fd1 200 ok 2
[notice] 2024-08-08T12:49:58.230659Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c659fbe9f 200 ok 2
[notice] 2024-08-08T12:49:58.233558Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65a6c73d 200 ok 2
[notice] 2024-08-08T12:49:58.237007Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65ab2bcc 200 ok 2
[notice] 2024-08-08T12:49:58.279214Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c6594dd08 200 ok 40
[notice] 2024-08-08T12:49:58.289546Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65956716 200 ok 4
[notice] 2024-08-08T12:49:58.293587Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c659606a9 200 ok 2
[notice] 2024-08-08T12:49:58.296797Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c659726de 200 ok 2
[notice] 2024-08-08T12:49:58.309766Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65974d8e 200 ok 11
[notice] 2024-08-08T12:49:58.315103Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c6597e7d6 200 ok 2
[notice] 2024-08-08T12:49:58.319121Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c6598461d 200 ok 2
[notice] 2024-08-08T12:49:58.322468Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c6598631a 200 ok 2
[notice] 2024-08-08T12:49:58.325641Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65a476cf 200 ok 2
[notice] 2024-08-08T12:49:58.329678Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c659a66c8 200 ok 3
[notice] 2024-08-08T12:49:58.333793Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65a51d43 200 ok 2
[notice] 2024-08-08T12:49:58.337465Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c65a9a2ed 200 ok 2
[notice] 2024-08-08T12:49:58.342782Z ... GET /galasa_artifacts/0c931ef9697cf407cb63326c659c102b 200 ok 2
...

This PR stops the above from happening. The RAS logs look like this now:

[notice] 2024-08-15T08:46:06.669785Z ... POST /galasa_run/_find 200 ok 9
[notice] 2024-08-15T08:46:06.673487Z ... POST /galasa_run/_find 200 ok 1
...

Changes

  • Updated the CouchDB RAS extension to stop going through artifacts when querying for runs using search criteria
  • Upped the maximum number of query results returned by CouchDB when searching for runs from 25 (default) to 100, to reduce the number of queries sent to CouchDB
  • Added initial unit tests for the getRuns method in the CouchdbDirectoryService class

@galasa-team
Copy link

Build failed, see http://localhost:8001/api/v1/namespaces/tekton-pipelines/services/tekton-dashboard:http/proxy/#/namespaces/galasa-build/pipelineruns/repo-extensions-pr-263-8zt7f for details. If you are unable to do so, please contact a member of the Galasa team.

@galasa-team
Copy link

Build failed, see http://localhost:8001/api/v1/namespaces/tekton-pipelines/services/tekton-dashboard:http/proxy/#/namespaces/galasa-build/pipelineruns/repo-extensions-pr-263-s47q2 for details. If you are unable to do so, please contact a member of the Galasa team.

Signed-off-by: Eamonn Mansour <[email protected]>
@galasa-team
Copy link

Build successful

@eamansour eamansour requested a review from techcobweb August 15, 2024 08:48
@galasa-team
Copy link

Build successful

@eamansour eamansour merged commit b1ea642 into main Aug 15, 2024
6 checks passed
@eamansour eamansour deleted the iss1921-runs-get-performance branch August 15, 2024 09:25
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants