You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We'd like to ensure that changes we make don't dramatically reduce Franklin's performance. One way to do that would be to bring up a service, fill it with data, then run some queries against it in CI, with some service-level objectives like "search with a datetime filter should never take longer than 750ms". A list of SLOs will be added to this issue before any work begins.
Notes + Context
Our go-to for load testing at this point is k6.io. k6.io expects a node.js script. As far as I know the only JS-compatible client currently available is purescript-stac. My proposal in this issue is to seed data into an API running with transactions with purescript-stac, return the ids of the created entities, and use those ids to create requests for k6.
This load test will initially focus on GETs of different kinds -- collections by id, collection lists, collection items by id, collection items lists, and searches. That is, the performance of seeding the data won't be tested. This could be part of the test at some point, but not now. In particular, k6 cycling back through the request list causes some problems with uniqueness constraints for item and collection IDs that I don't think it's worth it to solve as a first cut at performance testing. It will also exclude tile routes, since:
creating items with tileable raster assets requires having a stable reference to a public COG and adding logic to the test to figure out sensible requests over that COG
vector tile route methods for collections aren't currently in the client and I'd like to restrict this to performance testing only
Pending a successful proof of concept with the GETs mentioned above, those other tests can be added at a later date.
The text was updated successfully, but these errors were encountered:
Improvement
We'd like to ensure that changes we make don't dramatically reduce Franklin's performance. One way to do that would be to bring up a service, fill it with data, then run some queries against it in CI, with some service-level objectives like "search with a datetime filter should never take longer than 750ms". A list of SLOs will be added to this issue before any work begins.
Notes + Context
Our go-to for load testing at this point is k6.io. k6.io expects a node.js script. As far as I know the only JS-compatible client currently available is
purescript-stac
. My proposal in this issue is to seed data into an API running with transactions withpurescript-stac
, return the ids of the created entities, and use those ids to create requests for k6.This load test will initially focus on GETs of different kinds -- collections by id, collection lists, collection items by id, collection items lists, and searches. That is, the performance of seeding the data won't be tested. This could be part of the test at some point, but not now. In particular, k6 cycling back through the request list causes some problems with uniqueness constraints for item and collection IDs that I don't think it's worth it to solve as a first cut at performance testing. It will also exclude tile routes, since:
Pending a successful proof of concept with the GETs mentioned above, those other tests can be added at a later date.
The text was updated successfully, but these errors were encountered: