ods-buildah build completes successfully without Aqua being installed correctly

[henninggross]

In a new ods-pipeline installation I did for a project the pipeline runs were failing in the package-image step with the following error:

aqua scan: start cmd: fork/exec /usr/local/bin/aquasec: exec format error

As there was an issue with aqua somehow I checked the logs of the successful ods-buildah build an noticed the following:

< HTTP/2 200
< date: Fri, 02 Dec 2022 07:56:48 GMT
< content-type: application/json; charset=utf-8
< server: openresty/1.19.9.1
< strict-transport-security: max-age=31536000; includeSubDomains
< content-security-policy: img-src *;
< x-frame-options: SAMEORIGIN
< x-content-type-options: nosniff
< referrer-policy: no-referrer-when-downgrade
< permissions-policy: midi=()
< docker-distribution-api-version: registry/2.0
< www-authenticate: Basic realm="Aqua Download"
< x-xss-protection: 1; mode=block
<
{ [0 bytes data]
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Connection #0 to host download.aquasec.com left intact
Aqua scanner version:
Aqua scanner installation completed!

compared to a successful installation of aqua in a different project:

< HTTP/2 200
< date: Fri, 02 Dec 2022 11:13:46 GMT
< content-type: binary/octet-stream
< content-length: 86966209
< server: openresty/1.19.9.1
< strict-transport-security: max-age=31536000; includeSubDomains
< content-security-policy: img-src *;
< x-frame-options: SAMEORIGIN
< x-content-type-options: nosniff
< referrer-policy: no-referrer-when-downgrade
< permissions-policy: midi=()
< docker-distribution-api-version: registry/2.0
< www-authenticate: Basic realm="Aqua Download"
< x-xss-protection: 1; mode=block
< x-amz-id-2: 4oEy/bQm6T2zQ340wvv1LSUpGFO62wVA2+eINWZUn9hJTth2oWTsbkbghbr9xqxk6w5NGcEwF1k=
< x-amz-request-id: THTJPZ41JJT8VXX5
< last-modified: Mon, 28 Feb 2022 16:53:49 GMT
< etag: "3b35ac02dae988dd031bd712a64e6398-11"
< accept-ranges: bytes
<
{ [15612 bytes data]
0 82.9M 0 495k 0 0 558k 0 0:02:32 --:--:-- 0:02:32 557k 25 82.9M 25 21.3M 0 0 11.4M 0 0:00:07 0:00:01 0:00:06 11.3M 52 82.9M 52 43.7M 0 0 15.2M 0 0:00:05 0:00:02 0:00:03 15.2M 80 82.9M 80 66.3M 0 0 17.2M 0 0:00:04 0:00:03 0:00:01 17.2M100 82.9M 100 82.9M 0 0 18.3M 0 0:00:04 0:00:04 --:--:-- 18.3M
* Connection #0 to host download.aquasec.com left intact
Aqua scanner version:
scannercli version 5.3.22055, compiled Feb 24 2022 16:53:20
Aqua scanner installation completed!

It is obvious that something is incorrectly setup leading to the aqua scanner not being downloaded, but no matter the reason why the aqua installation failed, the build should not complete successfully if the aqua scanner wasn't download and installed correctly.

[michaelsauter]

Interesting, this happened some time ago as well, I think there is something wrong with how the Aqua scanner is provided.

There is an aqua-gate param (see https://github.com/opendevstack/ods-pipeline/blob/master/docs/tasks/ods-package-image.adoc). Unless this is set to true, the build won't fail. In your specific case it should distinguish though which kind of error it is and fail if aqua cannot be executed at all. BTW, which version are you using? 0.7?

Finally, just for context: the download of the Aqua scanner will likely be moved to happen in the task instead of being downloaded in the wrapper image, see #621.

[henninggross]

Sorry for not clarifying, yes, 0.7.0 was used.

good to know wrt. #621. Then I will remove Aqua in this project for now.

[michaelsauter]

@henninggross Simply rebuilding the image should fix your issue and you can use Aqua. #621 is not removing Aqua, it is just changing how it is installed.

[henninggross]

I tried rebuilding three times but that did not fix the issue. Actually today I tried a fourth time and Aqua was installed successfully.

[michaelsauter]

I am going to reopen this because the issue that the tasks succeeds without the scanner being involved is still there.

[michaelsauter]

Closing now that #621 is merged.