Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add load test to CI #688

Open
jisantuc opened this issue Apr 22, 2021 · 0 comments
Open

Add load test to CI #688

jisantuc opened this issue Apr 22, 2021 · 0 comments

Comments

@jisantuc
Copy link
Contributor

jisantuc commented Apr 22, 2021

Improvement

We'd like to ensure that changes we make don't dramatically reduce Franklin's performance. One way to do that would be to bring up a service, fill it with data, then run some queries against it in CI, with some service-level objectives like "search with a datetime filter should never take longer than 750ms". A list of SLOs will be added to this issue before any work begins.

Notes + Context

Our go-to for load testing at this point is k6.io. k6.io expects a node.js script. As far as I know the only JS-compatible client currently available is purescript-stac. My proposal in this issue is to seed data into an API running with transactions with purescript-stac, return the ids of the created entities, and use those ids to create requests for k6.

This load test will initially focus on GETs of different kinds -- collections by id, collection lists, collection items by id, collection items lists, and searches. That is, the performance of seeding the data won't be tested. This could be part of the test at some point, but not now. In particular, k6 cycling back through the request list causes some problems with uniqueness constraints for item and collection IDs that I don't think it's worth it to solve as a first cut at performance testing. It will also exclude tile routes, since:

  • creating items with tileable raster assets requires having a stable reference to a public COG and adding logic to the test to figure out sensible requests over that COG
  • vector tile route methods for collections aren't currently in the client and I'd like to restrict this to performance testing only

Pending a successful proof of concept with the GETs mentioned above, those other tests can be added at a later date.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant