Verifier: Add IBM Secure Execution driver framework #345

huoqifeng · 2024-03-06T03:20:58Z

Fixes: #342
This is kbs side code and related with PR: confidential-containers/guest-components#492
Depends on:

Add a new TEE type: SE (IBM Secure Execution) virtee/kbs-types#26

The IBM SE Remote Attestation flow:

The verifier generate the encrypted attestation-request based on hkd, CA, signing_key, a measurement key and a nonce, the encrypted data is protected by a symmetric attestation request protection key, which is encrypted using the Host-key document
Verifier sends the request to attester
Firmware on the Attester's system decrypts the request via private host-key and calculates the evidence based on the encrpted part of the request (Measurement key + nonce)
Attester send the evidence to verifier
Verifier recalculates the evidence based on the Configuration UID, Additional data, user-data, guest image hashes, and nonce. (its a HMAC-SHA512 with the measurement key as secret)
if both HMACs, the one from the Firmware and the calculated one from the verifier match -> attestation success

kbs/src/api/src/attestation/mod.rs

huoqifeng · 2024-03-15T07:06:18Z

@Xynnn007 @fitzthum I'd like start the review process of the API change to support IBM SE. May you please help on that?

Xynnn007 · 2024-03-15T08:39:21Z

Oh. I am still worried about the breakage for the PR. As it couple the KBS logic and AS logic to prepare an attreq.bin. After some reading of https://www.ibm.com/docs/en/linuxonibm/pdf/l130se03.pdf, I think of some different design ways for this and I must ignore something.

Way 1:

Con: We need to embed the host cert key chain (host key doc, CA cert, IBM_sign_key_cert) into the guest image

The logic of attester side:

After receiving challenge, generate an arp.key by HKDF(challenge || host_key_doc || CA cert || IBM_sign_key_cert)
Generate a report using the arp.key and other materials using pvattest

The logic of verifier side:

Use the same host key doc, CA cert, IBM_sign_key_cert and challenge to generate a same arp.key, and verify with pvattest verify -i attresp.bin --arpk arp.key --hdr hdr.bin

Rationality:
Looking at the entire attestation protocol of pvattest, the most critical thing is this arp.key, which is used for anti-replay, similar to the Nonce of other architectures, so we combine platform information host_key_doc || intermediate CA certificate || IBM sign key cert together with challenge. The strong binding information is used as the derivation factor of this key to ensure that the attester cannot use an instance on another platform to forge evidence that can pass verification - because the arp.key of the verifier side must be based on the certificate of the target environment.

BTW, if SE guest could support initdata (either though hostdata mechanism or vTPM), we could inject those documents when launching the guest without embed them inside the guest.

Way 2: Add a `get_challenge` to the `verifier` API

This is the current implementation.

Con: get_challenge() might mean nothing to other platforms.

This means every time KBS receives a RCAR request, it should first call the backend AS to call get_challenge().

This hints that CoCoAS should also support 4-pass protocol besides current 2-pass protocol, s.t. two ways to use CoCoAS

get_challenge() and then evaluate()
directly evaluate()

cc @jialez0

Way 3: Integrite IBM SE verifier logic inside KBS, and attester logic in KBS-client

Con: No se plugin will be support in attester and verifier crate, and thus not be support by AS. KBS should store arp.keys and se_guest.hdrs

This is based on a different attestation protocol of SE. We can introduce some conditional judgement for SE on the both KBS and KBS client side.

Way 4: Integrite IBM SE verifier logic inside KBS but attester logic in `attester` crate

Con: KBS should store arp.keys and se_guest.hdrs.

On the attester side, we can reuse report_data/challenge field for SE to present the host key doc, CA cert, IBM_sign_key_cert and attreq.bin bundle and generate an attresp.bin

On the verifier side, KBS will implement the generation of attreq.bin, and the verification of attresp.bin.

huoqifeng · 2024-03-19T00:36:06Z

Oh. I am still worried about the breakage for the PR. As it couple the KBS logic and AS logic to prepare an attreq.bin. After some reading of https://www.ibm.com/docs/en/linuxonibm/pdf/l130se03.pdf, I think of some different design ways for this and I must ignore something.

Way 1:

Con: We need to embed the host cert key chain (host key doc, CA cert, IBM_sign_key_cert) into the guest image

The logic of attester side:
1. After receiving `challenge`, generate an `arp.key` by `HKDF(challenge || host_key_doc || CA cert || IBM_sign_key_cert)`

2. Generate a report using the `arp.key` and other materials using `pvattest`
The logic of verifier side:
1. Use the same `host key doc`, `CA cert`, `IBM_sign_key_cert` and `challenge` to generate a same `arp.key`, and verify with `pvattest verify -i attresp.bin --arpk arp.key --hdr hdr.bin`
Rationality: Looking at the entire attestation protocol of pvattest, the most critical thing is this arp.key, which is used for anti-replay, similar to the Nonce of other architectures, so we combine platform information host_key_doc || intermediate CA certificate || IBM sign key cert together with challenge. The strong binding information is used as the derivation factor of this key to ensure that the attester cannot use an instance on another platform to forge evidence that can pass verification - because the arp.key of the verifier side must be based on the certificate of the target environment.

BTW, if SE guest could support initdata (either though hostdata mechanism or vTPM), we could inject those documents when launching the guest without embed them inside the guest.

Way 2: Add a get_challenge to the verifier API

This is the current implementation.

Con: get_challenge() might mean nothing to other platforms.

This means every time KBS receives a RCAR request, it should first call the backend AS to call get_challenge().

This hints that CoCoAS should also support 4-pass protocol besides current 2-pass protocol, s.t. two ways to use CoCoAS
1. `get_challenge()` and then [evaluate()](https://github.com/confidential-containers/trustee/blob/main/attestation-service/protos/attestation.proto#L92)

2. directly [evaluate()](https://github.com/confidential-containers/trustee/blob/main/attestation-service/protos/attestation.proto#L92)
cc @jialez0

Way 3: Integrite IBM SE verifier logic inside KBS, and attester logic in KBS-client

Con: No se plugin will be support in attester and verifier crate, and thus not be support by AS. KBS should store arp.keys and se_guest.hdrs

This is based on a different attestation protocol of SE. We can introduce some conditional judgement for SE on the both KBS and KBS client side.

Way 4: Integrite IBM SE verifier logic inside KBS but attester logic in attester crate

Con: KBS should store arp.keys and se_guest.hdrs.

On the attester side, we can reuse report_data/challenge field for SE to present the host key doc, CA cert, IBM_sign_key_cert and attreq.bin bundle and generate an attresp.bin

On the verifier side, KBS will implement the generation of attreq.bin, and the verification of attresp.bin.

Thanks @Xynnn007 for good suggestions, I think way 2 approach is used in this PR, generate_challenge_extra_params is added in verifier with a default impplementation. I'm happy to chnage the function name. CC @jialez0

huoqifeng · 2024-03-19T00:47:11Z

@Xynnn007 @jialez0 we also want to add the extra-params in Attestation like https://github.com/huoqifeng/kbs-types/blob/271ef4bac54035fb25fa44b24e5bbca844cd057c/src/lib.rs#L74-L75, how do you think?

fitzthum · 2024-03-20T19:21:58Z

This is a tricky one. If we get this extra challenge information from the AS, does that mean that the AS will have to maintain some per-connection state?

huoqifeng · 2024-03-21T00:59:26Z

This is a tricky one. If we get this extra challenge information from the AS, does that mean that the AS will have to maintain some per-connection state?

@fitzthum No need keep the state. For IBM SE, the attester will embed the original encrypted attestation request in evidence also, the verifier will extract the request first and then perform the verifying.

huoqifeng · 2024-03-27T09:41:51Z

@fitzthum @Xynnn007 CCC-Attestation/meetings#32, slides 9~18 describes the details of IBM SE attestation and also answers why we have attestation-request needed. The record is coming soon.

mkulke

lgtm.

I'll suggest to wait for the attester module to be merged first and then bump the kbs/tools/client guest-component dependency revision for e2e tests

We are encountering an issue on s390x when compiling AS with `all-verifier`. The error message is as follows: ``` error: failed to run custom build command for `tss-esapi-sys v0.5.0` ``` A platform-specific verifier (e.g., `se-verifier`) is used here. (confidential-containers#345) Although we can easily configure the verifier using `--features`, this approach lacks flexibility when the crate is selectively called from outside (e.g., kbs) based on `target_arch`. The optimal solution would be to open up room for configuring the verifier at a `dependencies` level rather than a `features` level. This commit aims to remove `all-verifier` from the default feature set and configure it differently for s390x. Signed-off-by: Hyounggyu Choi <[email protected]>

Xynnn007

Thanks. lgtm. Let's get this merged after confidential-containers/guest-components#492

attestation-service/verifier/Cargo.toml

attestation-service/verifier/src/lib.rs

attestation-service/verifier/src/se/mod.rs

attestation-service/verifier/src/se/ibmse.rs

huoqifeng · 2024-06-04T03:15:39Z

@mkulke @Xynnn007 thank you very much for reviewing the PR and gave so many useful suggestions! thanks for your approval.

kbs/src/api/src/http/attest.rs

* Verifier: Add IBM Secure Execution verifier driver framework Signed-off-by: Qi Feng Huo <[email protected]> * kbs/attestation: support attestation service to generate Challenge Related to confidential-containers#162 .1 This commit allows the Challenge step in RCAR handshake to be generated due to backend attestation service. For typical attestation services, a nonce will be generated. This commit also covers IBM SE + CoCoAS case. In this case, CoCoAS must be accessed to get a challenge which is specificly used by IBM SE. Signed-off-by: Xynnn007 <[email protected]> --------- Signed-off-by: Qi Feng Huo <[email protected]> Signed-off-by: Xynnn007 <[email protected]> Co-authored-by: Xynnn007 <[email protected]>

huoqifeng · 2024-06-04T13:23:10Z

Squashed commits.

Xynnn007

lgtm

attestation-service/README.md

attestation-service/verifier/src/se/ibmse.rs

…ents Signed-off-by: Qi Feng Huo <[email protected]>

attestation-service/docs/parsed_claims.md

attestation-service/verifier/Cargo.toml

attestation-service/verifier/src/lib.rs

attestation-service/verifier/src/se/ibmse.rs

fitzthum

Looks pretty good. A few comments.

attestation-service/verifier/src/se/mod.rs

kbs/src/api/src/attestation/coco/grpc.rs

kbs/src/api/src/attestation/mod.rs

attestation-service/verifier/src/se/ibmse.rs

…ents Signed-off-by: Qi Feng Huo <[email protected]>

fitzthum

LGTM

I do want to emphasize that since this verifier does not check the report data, it is not secure. Hopefully that will be addressed in a follow-up. I think this PR is fine for getting all the support code in.

huoqifeng · 2024-06-11T00:09:42Z

LGTM

I do want to emphasize that since this verifier does not check the report data, it is not secure. Hopefully that will be addressed in a follow-up. I think this PR is fine for getting all the support code in.

Thank you! @fitzthum , yes, the checking algorithm checks the evidence from TEE guest and re-calculated value from verifier, which also based on same SE OS Image. We'll add additional check in a follow up PR.

Xynnn007 · 2024-06-11T01:32:36Z

confidential-containers/guest-components#492 was merged. we should update the rev in https://github.com/confidential-containers/trustee/blob/main/kbs/tools/client/Cargo.toml#L21

Signed-off-by: Qi Feng Huo <[email protected]>

We are encountering an issue on s390x when compiling AS with `all-verifier`. The error message is as follows: ``` error: failed to run custom build command for `tss-esapi-sys v0.5.0` ``` A platform-specific verifier (e.g., `se-verifier`) is used here. (confidential-containers#345) Although we can easily configure the verifier using `--features`, this approach lacks flexibility when the crate is selectively called from outside (e.g., kbs) based on `target_arch`. The optimal solution would be to open up room for configuring the verifier at a `dependencies` level rather than a `features` level. This commit aims to remove `all-verifier` from the default feature set and configure it differently for s390x. Signed-off-by: Hyounggyu Choi <[email protected]>

We are encountering an issue on s390x when compiling AS with `all-verifier`. The error message is as follows: ``` error: failed to run custom build command for `tss-esapi-sys v0.5.0` ``` A platform-specific verifier (e.g., `se-verifier`) is used here. (#345) Although we can easily configure the verifier using `--features`, this approach lacks flexibility when the crate is selectively called from outside (e.g., kbs) based on `target_arch`. The optimal solution would be to open up room for configuring the verifier at a `dependencies` level rather than a `features` level. This commit aims to remove `all-verifier` from the default feature set and configure it differently for s390x. Signed-off-by: Hyounggyu Choi <[email protected]>

We are encountering an issue on s390x when compiling AS with `all-verifier`. The error message is as follows: ``` error: failed to run custom build command for `tss-esapi-sys v0.5.0` ``` A platform-specific verifier (e.g., `se-verifier`) is used here. (confidential-containers#345) Although we can easily configure the verifier using `--features`, this approach lacks flexibility when the crate is selectively called from outside (e.g., kbs) based on `target_arch`. The optimal solution would be to open up room for configuring the verifier at a `dependencies` level rather than a `features` level. This commit aims to remove `all-verifier` from the default feature set and configure it differently for s390x. Signed-off-by: Hyounggyu Choi <[email protected]> Signed-off-by: Pawel Proskurnicki <[email protected]>

huoqifeng requested a review from sameo as a code owner March 6, 2024 03:20

huoqifeng changed the title ~~Verifier: Add IBM Secure Execution verifier driver framework~~ [WIP] Verifier: Add IBM Secure Execution verifier driver framework Mar 6, 2024

huoqifeng mentioned this pull request Mar 6, 2024

aa/attester: add IBM Secure Execution driver framework confidential-containers/guest-components#492

Merged

1 task

Xynnn007 reviewed Mar 6, 2024

View reviewed changes

kbs/src/api/src/attestation/mod.rs Outdated Show resolved Hide resolved

huoqifeng force-pushed the s390x-verifier branch 3 times, most recently from c723db8 to 52f0b09 Compare March 7, 2024 01:15

huoqifeng marked this pull request as draft March 7, 2024 06:49

huoqifeng changed the title ~~[WIP] Verifier: Add IBM Secure Execution verifier driver framework~~ [WIP] Verifier: Add IBM Secure Execution driver framework Mar 7, 2024

huoqifeng force-pushed the s390x-verifier branch 8 times, most recently from 47fbb1a to abdf68f Compare March 8, 2024 02:42

huoqifeng force-pushed the s390x-verifier branch 2 times, most recently from 231177c to c0b4061 Compare March 15, 2024 06:09

huoqifeng changed the title ~~[WIP] Verifier: Add IBM Secure Execution driver framework~~ Verifier: Add IBM Secure Execution driver framework Mar 15, 2024

huoqifeng marked this pull request as ready for review March 21, 2024 01:00

huoqifeng mentioned this pull request Mar 21, 2024

Attestation: add extra parameters virtee/kbs-types#38

Closed

huoqifeng force-pushed the s390x-verifier branch from be396b5 to 417404b Compare March 22, 2024 08:42

mkulke approved these changes May 31, 2024

View reviewed changes

huoqifeng force-pushed the s390x-verifier branch from 3903850 to 89c0d65 Compare June 3, 2024 14:09

Xynnn007 mentioned this pull request Jun 4, 2024

prep for: let AS to initiate nonce/challenge #163

Closed

Xynnn007 approved these changes Jun 4, 2024

View reviewed changes

mythi reviewed Jun 4, 2024

View reviewed changes

kbs/src/api/src/http/attest.rs Outdated Show resolved Hide resolved

huoqifeng force-pushed the s390x-verifier branch from 48c2c81 to b1e14d1 Compare June 4, 2024 13:12

Xynnn007 approved these changes Jun 5, 2024

View reviewed changes

attestation-service/README.md Outdated Show resolved Hide resolved

Xynnn007 reviewed Jun 5, 2024

View reviewed changes

attestation-service/verifier/src/se/ibmse.rs Outdated Show resolved Hide resolved

huoqifeng force-pushed the s390x-verifier branch 2 times, most recently from 5651d3e to af3d0b5 Compare June 5, 2024 05:05

Verifier: Add IBM Secure Execution verifier driver framework fix comm…

567fbe3

…ents Signed-off-by: Qi Feng Huo <[email protected]>

huoqifeng force-pushed the s390x-verifier branch from af3d0b5 to 567fbe3 Compare June 5, 2024 05:22

steffen-eiden reviewed Jun 6, 2024

View reviewed changes

fitzthum reviewed Jun 6, 2024

View reviewed changes

huoqifeng force-pushed the s390x-verifier branch from d503b63 to 3d555dc Compare June 7, 2024 13:49

Verifier: Add IBM Secure Execution verifier driver framework fix comm…

f89bf21

…ents Signed-off-by: Qi Feng Huo <[email protected]>

huoqifeng force-pushed the s390x-verifier branch from 3d555dc to f89bf21 Compare June 7, 2024 23:38

fitzthum approved these changes Jun 10, 2024

View reviewed changes

Verifier: Add IBM Secure Execution and update guest-components rev

2bab3b3

Signed-off-by: Qi Feng Huo <[email protected]>

Xynnn007 merged commit 8867039 into confidential-containers:main Jun 11, 2024
17 checks passed

BbolroC mentioned this pull request Jun 28, 2024

Enable the k8s attestation tests for IBM Secure Execution (SE) kata-containers/kata-containers#9933

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Verifier: Add IBM Secure Execution driver framework #345

Verifier: Add IBM Secure Execution driver framework #345

huoqifeng commented Mar 6, 2024 •

edited

Loading

huoqifeng commented Mar 15, 2024

Xynnn007 commented Mar 15, 2024 •

edited

Loading

huoqifeng commented Mar 19, 2024

Way 1:

Way 2: Add a `get_challenge` to the `verifier` API

Way 3: Integrite IBM SE verifier logic inside KBS, and attester logic in KBS-client

Way 4: Integrite IBM SE verifier logic inside KBS but attester logic in `attester` crate

huoqifeng commented Mar 19, 2024

fitzthum commented Mar 20, 2024

huoqifeng commented Mar 21, 2024

huoqifeng commented Mar 27, 2024

mkulke left a comment

Xynnn007 left a comment

huoqifeng commented Jun 4, 2024

huoqifeng commented Jun 4, 2024

Xynnn007 left a comment

fitzthum left a comment

fitzthum left a comment

huoqifeng commented Jun 11, 2024

Xynnn007 commented Jun 11, 2024

Verifier: Add IBM Secure Execution driver framework #345

Verifier: Add IBM Secure Execution driver framework #345

Conversation

huoqifeng commented Mar 6, 2024 • edited Loading

The IBM SE Remote Attestation flow:

huoqifeng commented Mar 15, 2024

Xynnn007 commented Mar 15, 2024 • edited Loading

Way 1:

Way 2: Add a get_challenge to the verifier API

Way 3: Integrite IBM SE verifier logic inside KBS, and attester logic in KBS-client

Way 4: Integrite IBM SE verifier logic inside KBS but attester logic in attester crate

huoqifeng commented Mar 19, 2024

Way 1:

Way 2: Add a get_challenge to the verifier API

Way 3: Integrite IBM SE verifier logic inside KBS, and attester logic in KBS-client

Way 4: Integrite IBM SE verifier logic inside KBS but attester logic in attester crate

huoqifeng commented Mar 19, 2024

fitzthum commented Mar 20, 2024

huoqifeng commented Mar 21, 2024

huoqifeng commented Mar 27, 2024

mkulke left a comment

Choose a reason for hiding this comment

Xynnn007 left a comment

Choose a reason for hiding this comment

huoqifeng commented Jun 4, 2024

huoqifeng commented Jun 4, 2024

Xynnn007 left a comment

Choose a reason for hiding this comment

fitzthum left a comment

Choose a reason for hiding this comment

fitzthum left a comment

Choose a reason for hiding this comment

huoqifeng commented Jun 11, 2024

Xynnn007 commented Jun 11, 2024

huoqifeng commented Mar 6, 2024 •

edited

Loading

Xynnn007 commented Mar 15, 2024 •

edited

Loading

Way 2: Add a `get_challenge` to the `verifier` API

Way 4: Integrite IBM SE verifier logic inside KBS but attester logic in `attester` crate

Way 2: Add a `get_challenge` to the `verifier` API

Way 4: Integrite IBM SE verifier logic inside KBS but attester logic in `attester` crate