Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Replace deprecated autorest SDK with azidentity #1904

Open
wants to merge 14 commits into
base: dev
Choose a base branch
from

Conversation

shahramk64
Copy link
Contributor

@shahramk64 shahramk64 commented Oct 30, 2024

Description

What this PR does / why we need it:

Azure authentication currently uses go-autorest SDK. It's now deprecated and using MSAL and azidentity SDKs are now recommended. This PR replaces the deprecated SDK with azidentity.

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):

Fixes #1630

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Helm Chart Change (any edit/addition/update that is necessary for changes merged to the main branch)
  • This change requires a documentation update

How Has This Been Tested?

This tutorial was followed to test authenticating AKS with AKV and retrieval of certificates.
1- KeyManagementProvider scenario: After deploying the following kmp resource using kubectl apply -f kmp_config.yaml, the Ratify pod log shows successful retrieval of the certificate.

kmp_config.yaml:

apiVersion: config.ratify.deislabs.io/v1beta1
kind: KeyManagementProvider
metadata:
  name: keymanagementprovider-akv
spec:
  type: azurekeyvault
  parameters:
    vaultURI: https://skal21kv.vault.azure.net/
    keys:
      - name: cosign
        version: 16c798cd4be049908bdcd841d20c19cd
    certificates:
      - name: wabbit-networks-io
        version: 3d6a30dfb52645ebb307d99e900c10ad
    tenantID: 72f988bf-86f1-41af-91ab-2d7cd011db47
    clientID: 6f77181a-6ca6-404d-accc-703b3ae18d03

Ratify logs:

time=2024-11-05T04:27:27.618019657Z level=info msg=provider.cloudEnv.KeyVaultEndpoint https://vault.azure.net/, provider.vaultURI https://skal21kv.vault.azure.net/ component-type=keyManagementProvider go.version=go1.22.8
time=2024-11-05T04:27:27.618039936Z level=info msg=kvEndpoint https://skal21kv.vault.azure.net component-type=keyManagementProvider go.version=go1.22.8
time=2024-11-05T04:27:27.618466847Z level=info msg=azsecrets kvclient created successfully component-type=keyManagementProvider go.version=go1.22.8
time=2024-11-05T04:27:27.618525541Z level=info msg=fetching secret from key vault, certName wabbit-networks-io, certVersion %!v(MISSING) component-type=keyManagementProvider go.version=go1.22.8
time=2024-11-05T04:27:27.901644022Z level=info msg=1 certificate(s) & 1 key(s) fetched for key management provider keymanagementprovider-akv

It was also verified by the following command:
kubectl describe KeyManagementProvider keymanagementprovider-akv

Name:         keymanagementprovider-akv
Namespace:    
Labels:       <none>
Annotations:  <none>
API Version:  config.ratify.deislabs.io/v1beta1
Kind:         KeyManagementProvider
Metadata:
  Creation Timestamp:  2024-11-05T04:27:27Z
  Generation:          1
  Resource Version:    1802478
  UID:                 889e589a-f696-4410-bd01-73bfa8a330b3
Spec:
  Parameters:
    Certificates:
      Name:     wabbit-networks-io
      Version:  3d6a30dfb52645ebb307d99e900c10ad
    Client ID:  6f77181a-6ca6-404d-accc-703b3ae18d03
    Keys:
      Name:          cosign
      Version:       16c798cd4be049908bdcd841d20c19cd
    Tenant ID:       72f988bf-86f1-41af-91ab-2d7cd011db47
    Vault URI:       https://skal21kv.vault.azure.net/
  Refresh Interval:  
  Type:              azurekeyvault
Status:
  Issuccess:        true
  Lastfetchedtime:  2024-11-05T04:27:27Z
  Properties:
    Certificates:
      Last Refreshed:  2024-11-05T04:27:27Z
      Name:            wabbit-networks-io
      Version:         3d6a30dfb52645ebb307d99e900c10ad
    Keys:
      Last Refreshed:  2024-11-05T04:27:27Z
      Name:            cosign
      Version:         16c798cd4be049908bdcd841d20c19cd
Events:                <none>

2- CertificateStore scenario: After deploying the following certificate store resource using kubectl apply -f cs_config.yaml, the Ratify pod log shows successful retrieval of the certificate.

cs_config.yaml:

apiVersion: config.ratify.deislabs.io/v1beta1
kind: CertificateStore
metadata:
  name: certstore-akv
spec:
  provider: azurekeyvault
  parameters:
    vaultURI: https://skal21kv.vault.azure.net/
    certificates: |
      array:
        - |
          certificateName: wabbit-networks-io
          certificateVersion: 3d6a30dfb52645ebb307d99e900c10ad
    tenantID: 72f988bf-86f1-41af-91ab-2d7cd011db47
    clientID: 6f77181a-6ca6-404d-accc-703b3ae18d03

Ratify logs:

time=2024-11-11T10:26:40.87690862Z level=info msg=reconciling certificate store 'gatekeeper-system/certstore-akv'
time=2024-11-11T10:26:41.589902887Z level=info msg=1 certificates fetched for certificate store gatekeeper-system/certstore-akv

It was also verified by the following command:
kubectl describe CertificateStore certstore-akv

Name:         certstore-akv
Namespace:    gatekeeper-system
Labels:       <none>
Annotations:  <none>
API Version:  config.ratify.deislabs.io/v1beta1
Kind:         CertificateStore
Metadata:
  Creation Timestamp:  2024-11-11T10:26:40Z
  Generation:          1
  Resource Version:    4036265
  UID:                 a9f89508-e7a0-45a4-855f-b1e7ec0ee13e
Spec:
  Parameters:
    Certificates:  array:
  - |
    certificateName: wabbit-networks-io
    certificateVersion: 3d6a30dfb52645ebb307d99e900c10ad

    Client ID:  6f77181a-6ca6-404d-accc-703b3ae18d03
    Tenant ID:  72f988bf-86f1-41af-91ab-2d7cd011db47
    Vault URI:  https://skal21kv.vault.azure.net/
  Provider:     azurekeyvault
Status:
  Issuccess:        true
  Lastfetchedtime:  2024-11-11T10:26:40Z
  Properties:
    Certificates:
      Certificate Name:  wabbit-networks-io
      Last Refreshed:    2024-11-11T10:26:41Z
      Version:           3d6a30dfb52645ebb307d99e900c10ad
Events:                  <none>

Checklist:

  • Does the affected code have corresponding tests?
  • Are the changes documented, not just with inline documentation, but also with conceptual documentation such as an overview of a new feature, or task-based documentation like a tutorial? Consider if this change should be announced on your project blog.
  • Does this introduce breaking changes that would require an announcement or bumping the major version?
  • Do all new files have appropriate license header?

Post Merge Requirements

  • MAINTAINERS: manually trigger the "Publish Package" workflow after merging any PR that indicates Helm Chart Change

Copy link

codecov bot commented Oct 30, 2024

Codecov Report

Attention: Patch coverage is 74.13793% with 30 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...kg/keymanagementprovider/azurekeyvault/provider.go 76.08% 17 Missing and 5 partials ⚠️
pkg/certificateprovider/azurekeyvault/provider.go 66.66% 8 Missing ⚠️
Files with missing lines Coverage Δ
pkg/certificateprovider/azurekeyvault/auth.go 65.21% <ø> (+31.12%) ⬆️
pkg/keymanagementprovider/azurekeyvault/auth.go 65.21% <ø> (+8.39%) ⬆️
pkg/certificateprovider/azurekeyvault/provider.go 62.56% <66.66%> (+4.45%) ⬆️
...kg/keymanagementprovider/azurekeyvault/provider.go 82.66% <76.08%> (-2.19%) ⬇️

... and 28 files with indirect coverage changes

@shahramk64 shahramk64 changed the title Replace deprecated autorest SDK with azidentity chore: Replace deprecated autorest SDK with azidentity Oct 30, 2024
@shahramk64 shahramk64 force-pushed the skalantari/Refactor-Azure-authentication-support-to-use-azidentity branch 6 times, most recently from 6e5c9bd to e8c85c4 Compare November 4, 2024 04:01
@shahramk64 shahramk64 marked this pull request as ready for review November 4, 2024 04:08
@shahramk64 shahramk64 force-pushed the skalantari/Refactor-Azure-authentication-support-to-use-azidentity branch from e8c85c4 to 2ba81a3 Compare November 4, 2024 04:09
Copy link
Collaborator

@susanshi susanshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @shahramk64 , looks great overall. Would you be able to include all automated and manual validation in the PR description? our automated test may have switched to using kmp, we might need to manually validate certificateprovider path to ensure this provider continues to be functional.

@shahramk64
Copy link
Contributor Author

Manual Validation test failed. I'll continue working on this PR and will update you once it's fixed
cc: @susanshi @akashsinghal @binbin-li

@shahramk64 shahramk64 force-pushed the skalantari/Refactor-Azure-authentication-support-to-use-azidentity branch from 8d9b220 to db0f2a9 Compare November 5, 2024 05:04
@shahramk64
Copy link
Contributor Author

@susanshi @binbin-li @akashsinghal
I fixed a bug, manual testing succeeded and I updated the PR description to include the testing evidence.

Copy link
Collaborator

@susanshi susanshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thankyou @shahramk64 ! code change looks good to me. I have two ask:

  • thank you for providing the validation result for kmp CR, since this change also updated certificate provider CR, would you be able to validate certificateprovider CR and paste the result?
  • would you be to review @duffney 's PR to help understand if there will be any updates required to resolve any merge conflict.

pkg/certificateprovider/azurekeyvault/provider.go Outdated Show resolved Hide resolved
pkg/certificateprovider/azurekeyvault/provider.go Outdated Show resolved Hide resolved
pkg/certificateprovider/azurekeyvault/provider.go Outdated Show resolved Hide resolved
pkg/certificateprovider/azurekeyvault/provider.go Outdated Show resolved Hide resolved
pkg/certificateprovider/azurekeyvault/provider.go Outdated Show resolved Hide resolved
pkg/keymanagementprovider/azurekeyvault/provider.go Outdated Show resolved Hide resolved
@@ -207,30 +210,42 @@ func parseAzureEnvironment(cloudName string) (*azure.Environment, error) {
return &env, err
}

func initializeKvClient(ctx context.Context, keyVaultEndpoint, tenantID, clientID string) (*kv.BaseClient, error) {
kvClient := kv.New()
func initializeKvClient(ctx context.Context, keyVaultEndpoint, tenantID, clientID string, credProvider azcore.TokenCredential) (*azsecrets.Client, error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct me if I'm wrong, but I didn't see any caller pass in a non-nil credprovider.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct. It only is non-nill in unit tests for mocking purposes.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I feel it's not the best way to add this nil credProvider for testing purpose, but may need some refactoring on how we invoke initKVClient function. It's out of the scope of this PR, could you add some TODO comments so that we can address it later.

pkg/keymanagementprovider/azurekeyvault/provider.go Outdated Show resolved Hide resolved
@@ -207,30 +210,42 @@ func parseAzureEnvironment(cloudName string) (*azure.Environment, error) {
return &env, err
}

func initializeKvClient(ctx context.Context, keyVaultEndpoint, tenantID, clientID string) (*kv.BaseClient, error) {
kvClient := kv.New()
func initializeKvClient(ctx context.Context, keyVaultEndpoint, tenantID, clientID string, credProvider azcore.TokenCredential) (*azsecrets.Client, error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I feel it's not the best way to add this nil credProvider for testing purpose, but may need some refactoring on how we invoke initKVClient function. It's out of the scope of this PR, could you add some TODO comments so that we can address it later.

@@ -79,11 +79,6 @@ func (s *akvCertProvider) GetCertificates(ctx context.Context, attrib map[string
return nil, nil, re.ErrorCodeConfigInvalid.NewError(re.CertProvider, providerName, re.AKVLink, nil, "clientID is not set", re.HideStackTrace)
}

azureCloudEnv, err := parseAzureEnvironment(cloudName)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we not need cloud env information now? Is this because the new SDK can infer cloud type based on the URI?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find any usecases for the couldName variable, other than extracting azureenv which is removed here because the new SDK only needs the vault's URI when creating a client. So I removed it.

if err != nil {
return nil, nil, fmt.Errorf("failed to get secret objectName:%s, objectVersion:%s, error: %w", keyVaultCert.CertificateName, keyVaultCert.CertificateVersion, err)
}
secretBundle := secretResponse.SecretBundle
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nil check required for secretReponse?

Copy link
Contributor Author

@shahramk64 shahramk64 Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go newbie question:
Is this the right way to compare the secretResponse struct with nil:

if reflect.DeepEqual(secretResponse, azsecrets.GetSecretResponse{}) {
	return nil, nil, fmt.Errorf("failed to get secret objectName:%s, objectVersion:%s, secret response is nil", keyVaultCert.Name, keyVaultCert.Version)
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is if secretResponse == nil working in this case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@binbin-li doesn't seem like it:

invalid operation: secretResponse == nil (mismatched types azsecrets.GetSecretResponse and untyped nil)compiler[MismatchedTypes](https://pkg.go.dev/golang.org/x/tools/internal/typesinternal#MismatchedTypes)

webKey.Kty = kv.EC
case kv.RSAHSM:
webKey.Kty = kv.RSA
case azkeys.JSONWebKeyTypeECHSM:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have we verified the new values for each case exact the existing values? Just want to double check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is how the new consts are defined in azkeys:

const (
	JSONWebKeyTypeEC JSONWebKeyType = "EC"
	JSONWebKeyTypeECHSM JSONWebKeyType = "EC-HSM"
	JSONWebKeyTypeOct JSONWebKeyType = "oct"
	JSONWebKeyTypeOctHSM JSONWebKeyType = "oct-HSM"
	JSONWebKeyTypeRSA JSONWebKeyType = "RSA"
	JSONWebKeyTypeRSAHSM JSONWebKeyType = "RSA-HSM"
)

And here is how they were defined in keyvault (kv):

const (
	EC = "EC"
	ECHSM = "EC-HSM"
	Oct = "oct"
	RSA = "RSA"
	RSAHSM = "RSA-HSM"
)

I replicated the same two conditions ( both in the condition and the body of the if statements). Is that whatever you are concerned about or something else?

@shahramk64 shahramk64 force-pushed the skalantari/Refactor-Azure-authentication-support-to-use-azidentity branch 4 times, most recently from af4254c to 8666a69 Compare November 11, 2024 09:22
Signed-off-by: Shahram Kalantari <[email protected]>
Signed-off-by: Shahram Kalantari <[email protected]>
Signed-off-by: Shahram Kalantari <[email protected]>
Signed-off-by: Shahram Kalantari <[email protected]>
Signed-off-by: Shahram Kalantari <[email protected]>
Signed-off-by: Shahram Kalantari <[email protected]>
Signed-off-by: Shahram Kalantari <[email protected]>
Signed-off-by: Shahram Kalantari <[email protected]>
@shahramk64 shahramk64 force-pushed the skalantari/Refactor-Azure-authentication-support-to-use-azidentity branch from 8666a69 to cbabb9c Compare November 11, 2024 10:15
@shahramk64
Copy link
Contributor Author

@duffney Can you please have a look at this PR and see if the merge makes sense to you?
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor Azure authentication support to use azidentity
7 participants