Speeding the testsuite up

[jvoisin]

Currently, the testsuite is spending most of its time hashing the same password over and over in func (s *Storage) CreateUser(userCreationRequest *model.UserCreationRequest) (*model.User, error).

This isn't critical, but I think it's always good to not waste cpu cycles on useless things if we can avoid it. I'm opening a conversation instead of a pull-request, since I'm a bit unsure about the best way to improve the situation.

Here are some suggestions:

1. Running all tests under the same user, dropping its data between every tests. This would improve things a bit, but would likely mean that we can't parallelize the tests anymore.
2. Add a CreateTestUser method, taking an already-hashed password.
3. Talk directly to the database to create the usernames instead of using CreateUser.
4. Have a global cost variable, set to DefaultCost by default, and overridden by tests to MinCost.

What would be the least worst way to go?

[jvoisin]

Another way to speed the CI up would be to not build/test debian packages. The github action responsible for this has pull_request: branches: [ main ] in it, meaning that the action will be triggered for every pull-request done against the main branch. Same remark for RPM.

I think it should only be enabled for tags.

[fguillot]

Currently, the testsuite is spending most of its time hashing the same password over and over in func (s *Storage) CreateUser(userCreationRequest *model.UserCreationRequest) (*model.User, error).

This isn't critical, but I think it's always good to not waste cpu cycles on useless things if we can avoid it. I'm opening a conversation instead of a pull-request, since I'm a bit unsure about the best way to improve the situation.

Here are some suggestions:

1. Running all tests under the same user, dropping its data between every tests. This would improve things a bit, but would likely mean that we can't parallelize the tests anymore.
2. Add a CreateTestUser method, taking an already-hashed password.
3. Talk directly to the database to create the usernames instead of using CreateUser.
4. Have a global cost variable, set to DefaultCost by default, and overridden by tests to MinCost.

What would be the least worst way to go?

I did some refactoring in PR #2523, and there is still room for improvement. Solution 1) seems to be the most reasonable. To be honest, the API integration tests don't bother me that much. The job takes about 1 minute to run on GitHub Actions.

[jvoisin]

Unfortunately, I haven't found a nice way to have a global setUp/tearDown mechanism to create/trash the single test user without resorting to gross hacks :/

It seems that the way to go is subtests, but I think it's a bit cumbersome.

[jvoisin]

There are some neat suggestions here: https://gajus.com/blog/setting-up-postgre-sql-for-running-integration-tests

[fguillot]

Another way to speed the CI up would be to not build/test debian packages.
I think it should only be enabled for tags.

This job takes time, but it has been configured for a reason. For example, I was able to detect a compilation issue when building the packages on armhf architecture when Go 1.22 was released (see #2346). If this pipeline was triggered only for tags, I would have released a broken release of Miniflux.

One possible way to work around this is to run this workflow once a day instead of each pull-request.

[jvoisin]

I think running the testsuite on various arch would be enough. What's taking the most time is running apt update/apt install and all the docker dance.

[fguillot]

The issue mentioned above would not have been detected if only the test suite was run for each architecture.

[jvoisin]

How so? You said there as a "a compilation issue when building the packages on armhf architecture", so go build ./... && go test ./... should catch it, no?