Skip to content

Commit

Permalink
feat(controller): irsa support (#2235)
Browse files Browse the repository at this point in the history 
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Kent Rancourt <[email protected]>
Co-authored-by: Kent Rancourt <[email protected]>
  • Loading branch information
@blakepettersson @krancour
blakepettersson and krancour authored Jul 15, 2024
1 parent 9101f2d commit f1583c1
Show file tree
Hide file tree
Showing 7 changed files with 82 additions and 47 deletions.
1 change: 1 addition & 0 deletions charts/kargo/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@ the Kargo controller is running.
| `controller.annotations` | Annotations to add to the api resources. Merges with `global.annotations`, allowing you to override or add to the global annotations. | `{}` |
| `controller.podLabels` | Optional labels to add to pods. Merges with `global.podLabels`, allowing you to override or add to the global labels. | `{}` |
| `controller.podAnnotations` | Optional annotations to add to pods. Merges with `global.podAnnotations`, allowing you to override or add to the global annotations. | `{}` |
| `controller.serviceAccount.iamRole` | Specifies the ARN of an AWS IAM role to be used by the controller in an IRSA-enabled EKS cluster. | `""` |
| `controller.globalCredentials.namespaces` | List of namespaces to look for shared credentials. | `[]` |
| `controller.gitClient.name` | Specifies the name of the Kargo controller (used when authoring Git commits). | `Kargo Render` |
| `controller.gitClient.email` | Specifies the email of the Kargo controller (used when authoring Git commits). | `[email protected]` |
Expand Down
4 changes: 4 additions & 0 deletions charts/kargo/templates/controller/service-account.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,8 @@ metadata:
labels:
{{- include "kargo.labels" . | nindent 4 }}
{{- include "kargo.controller.labels" . | nindent 4 }}
{{- if .Values.controller.serviceAccount.iamRole }}
annotations:
eks.amazonaws.com/role-arn: {{ .Values.controller.serviceAccount.iamRole }}
{{- end }}
{{- end }}
5 changes: 5 additions & 0 deletions charts/kargo/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,11 @@ controller:
## @param controller.podAnnotations Optional annotations to add to pods. Merges with `global.podAnnotations`, allowing you to override or add to the global annotations.
podAnnotations: {}

## All settings relating to the service account for the controller
serviceAccount:
## @param controller.serviceAccount.iamRole Specifies the ARN of an AWS IAM role to be used by the controller in an IRSA-enabled EKS cluster.
iamRole: ""

## All settings relating to shared credentials (used across multiple kargo projects)
globalCredentials:
## @param controller.globalCredentials.namespaces List of namespaces to look for shared credentials.
Expand Down
65 changes: 43 additions & 22 deletions docs/docs/30-how-to-guides/20-managing-credentials.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -378,54 +378,75 @@ to the required ECR repositories.

:::caution
This method of authentication is a "lowest common denominator" approach that
will work regardless of where Kargo is deployed. i.e. If running Kargo outside
of EKS, this method will still work.
will work regardless of where Kargo is deployed. i.e. if running Kargo outside EKS, this method will still work.

If running Kargo within EKS, you may wish to consider using EKS Pod Identity
If running Kargo within EKS, you may wish to either consider using EKS Pod Identity or IRSA
instead.
:::

#### EKS Pod Identity
#### EKS Pod Identity or IAM Roles for Service Accounts (IRSA)

If Kargo locates no `Secret` resources matching a repository URL, and if Kargo
is deployed within an EKS cluster, it will attempt to use
If Kargo locates no `Secret` resources matching a repository URL and is deployed
within an EKS cluster, it will attempt to use
[EKS Pod Identity](https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html)
to authenticate, but this relies upon some external setup. Leveraging this
option eliminates the need to store credentials in a `Secret` resource.
or
[IAM Roles for Service Accounts (IRSA)](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)
to authenticate. Both of these rely upon some external setup. Leveraging either
eliminates the need to store ECR credentials in a `Secret` resource.

First, follow
Follow
[this overview](https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html#pod-id-setup-overview)
to set up EKS Pod Identity in your EKS cluster and assign an IAM role to the
to set up EKS Pod Identity in your EKS cluster or
[this one](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)
to set up IRSA. For either, you will assign an IAM role to the
`kargo-controller` `ServiceAccount` within the `Namespace` to which Kargo is (or
will be) installed.

:::note
To use IRSA, you will additionally need to specify the
[ARN](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html) of
the controller's IAM role as the value of the
`controller.serviceAccount.iamRole` setting in Kargo's Helm chart. Refer to
[the advanced section of the installation guide](./10-installing-kargo.md#advanced-installation)
for more details.
:::


At this point, an IAM role will be associated with the Kargo _controller_,
however, that controller acts on behalf of multiple Kargo projects, each of
which may require access to _different_ ECR repositories. To account for this,
when Kargo attempts to access an ECR repository on behalf of a specific project,
it will first attempt to
[assume an IAM role](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html)
specific to that project. The name of the role it attempts to assume will _always_
be of the form `kargo-project-<project name>`. It is this role that should be
granted read-only access to applicable ECR repositories.
specific to that project. The name of the role it attempts to assume will
_always_ be of the form `kargo-project-<project name>`. It is this role that
should be granted read-only access to applicable ECR repositories.

:::info
The name of the IAM role associated with each Kargo project is deliberately
not configurable to prevent project admins from attempting to coerce Kargo into
assuming arbitrary IAM roles.
:::

Once Kargo is able to assume the appropriate IAM role for a given project, it
will follow a process similar to that described in the previous section to
obtain a token that is valid for 12 hours and cached for 10.

:::caution
Following the principle of least privilege, the IAM role associated with the
`kargo-controller` `ServiceAccount` should be limited only to the ability to
assume project-specific IAM roles. Project-specific IAM roles should be limited
only to read-only access to the applicable ECR repositories.
For optimal adherence to the principle of least permissions, the IAM role
associated with the `kargo-controller` `ServiceAccount` should be limited only
to the ability to assume project-specific IAM roles. Project-specific IAM roles
should be limited only to read-only access to applicable ECR repositories.
:::

:::info
If the Kargo controller is unable to assume a project-specific IAM role, it will
fall back to using its own IAM role directly. For organizations without strict
tenancy requirements, this can eliminate the need to manage a large number of
project-specific IAM roles. While useful, this approach is not strictly
recommended.
:::

Once Kargo is able to gain necessary permissions to access an ECR repository,
it will follow a process similar to that described in the previous section to
obtain a token that is valid for 12 hours and cached for 10.

### Google Artifact Registry

The authentication options described in this section are applicable only to
Expand Down Expand Up @@ -551,7 +572,7 @@ Azure Container Registry directly supports long-lived credentials.

It is possible to
[create tokens with repository-scoped permissions](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-repository-scoped-permissions),
with or without an expiration date. These tokens can be be stored in the
with or without an expiration date. These tokens can be stored in the
`username` and `password` fields of a `Secret` resource as described
[in the first section](#credentials-as-kubernetes-secret-resources) of this
document.
Expand Down
2 changes: 1 addition & 1 deletion internal/credentials/kubernetes/database.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ func NewDatabase(
credentialHelpers := []credentials.Helper{
basic.SecretToCreds,
ecr.NewAccessKeyCredentialHelper(),
ecr.NewPodIdentityCredentialHelper(ctx),
ecr.NewManagedIdentityCredentialHelper(ctx),
gar.NewServiceAccountKeyCredentialHelper(),
gar.NewWorkloadIdentityFederationCredentialHelper(ctx),
github.NewAppCredentialHelper(),
Expand Down
28 changes: 16 additions & 12 deletions ...redentials/kubernetes/ecr/pod_identity.go → ...rnetes/ecr/managed_identity_credential.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import (
"github.com/akuity/kargo/internal/logging"
)

type podIdentityCredentialHelper struct {
type managedIdentityCredentialHelper struct {
awsAccountID string

tokenCache *cache.Cache
Expand All @@ -37,27 +37,31 @@ type podIdentityCredentialHelper struct {
) (string, error)
}

// NewPodIdentityCredentialHelper returns an implementation of
// NewManagedIdentityCredentialHelper returns an implementation of
// credentials.Helper that utilizes a cache to avoid unnecessary calls to AWS.
func NewPodIdentityCredentialHelper(ctx context.Context) credentials.Helper {
func NewManagedIdentityCredentialHelper(ctx context.Context) credentials.Helper {
logger := logging.LoggerFromContext(ctx)
var awsAccountID string
if os.Getenv("AWS_CONTAINER_CREDENTIALS_FULL_URI") == "" {
logger.Info("AWS_CONTAINER_CREDENTIALS_FULL_URI not set; assuming EKS Pod Identity is not in use")
if os.Getenv("AWS_CONTAINER_CREDENTIALS_FULL_URI") != "" {
logger.Info("EKS Pod Identity appears to be in use")
} else if os.Getenv("AWS_ROLE_ARN") != "" && os.Getenv("AWS_WEB_IDENTITY_TOKEN_FILE") != "" {
logger.Info("AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN set; assuming IRSA is being used")
} else {
logger.Info("Neither AWS_CONTAINER_CREDENTIALS_FULL_URI nor AWS_WEB_IDENTITY_TOKEN_FILE " +
"and AWS_ROLE_ARN are set; assuming neither EKS Pod Identity nor IRSA are in use")
return nil
}
logger.Info("EKS Pod Identity appears to be in use")
cfg, err := config.LoadDefaultConfig(ctx)
if err != nil {
logger.Error(
err, "error loading AWS config; EKS Pod Identity integration will be disabled",
err, "error loading AWS config; AWS credentials integration will be disabled",
)
} else {
stsSvc := sts.NewFromConfig(cfg)
res, err := stsSvc.GetCallerIdentity(ctx, &sts.GetCallerIdentityInput{})
if err != nil {
logger.Error(
err, "error getting caller identity; EKS Pod Identity integration will be disabled",
err, "error getting caller identity; AWS credentials integration will be disabled",
)
} else {
logger.Debug(
Expand All @@ -67,7 +71,7 @@ func NewPodIdentityCredentialHelper(ctx context.Context) credentials.Helper {
awsAccountID = *res.Account
}
}
p := &podIdentityCredentialHelper{
p := &managedIdentityCredentialHelper{
awsAccountID: awsAccountID,
tokenCache: cache.New(
// Tokens live for 12 hours. We'll hang on to them for 10.
Expand All @@ -79,7 +83,7 @@ func NewPodIdentityCredentialHelper(ctx context.Context) credentials.Helper {
return p.getCredentials
}

func (p *podIdentityCredentialHelper) getCredentials(
func (p *managedIdentityCredentialHelper) getCredentials(
ctx context.Context,
project string,
credType credentials.Type,
Expand Down Expand Up @@ -129,7 +133,7 @@ func (p *podIdentityCredentialHelper) getCredentials(
return decodeAuthToken(encodedToken)
}

func (p *podIdentityCredentialHelper) tokenCacheKey(region, project string) string {
func (p *managedIdentityCredentialHelper) tokenCacheKey(region, project string) string {
return fmt.Sprintf(
"%x",
sha256.Sum256([]byte(
Expand All @@ -141,7 +145,7 @@ func (p *podIdentityCredentialHelper) tokenCacheKey(region, project string) stri
// getAuthToken returns an ECR authorization token obtained by assuming a
// project-specific IAM role and using that to obtain a short-lived ECR access
// token.
func (p *podIdentityCredentialHelper) getAuthToken(
func (p *managedIdentityCredentialHelper) getAuthToken(
ctx context.Context,
region string,
project string,
Expand Down
Loading

0 comments on commit f1583c1

Please sign in to comment.