Design of oras-credential-go

[Wwwsylvia]

Design of oras-credential-go

Goals

1. The library should be able to read credentials from Docker configuration file and external credentials store such as the native keychain of the operating system.
2. The library should be able to save credentials in Docker configuration file and external credentials store.
3. The library should be able to remove the specified credentials from external credential stores or Docker configuration file.
4. The library should be easy to integrate with Go projects that uses oras-go v2.
5. The library should be able to replace the following credential modules:
   • oras: https://github.com/oras-project/oras/tree/main/internal/credential
   • oras-go v1: https://github.com/oras-project/oras-go/tree/v1/pkg/auth/docker
   • notation: https://github.com/notaryproject/notation/tree/main/pkg/auth

Notes: The following is an example of Docker configuration file.

{
    "auths": {
        "registry1.example.com": {
            "auth": "base64_encode(username:password)",
        },
        "registry2.example.com": {
            "identitytoken": "identity_token"
        },
        "registry3.example.com": {
            "registrytoken": "registry_token"
        },
        "registry4.example.com": {}
    },
    "credsStore": "desktop",
    "credHelpers": {
        "registry.example.com": "registryhelper",
        "awesomereg.example.org": "hip-star",
        "unicorn.example.io": "vcbait"
    }
}

Non-Goals

1. The library will not support configuration formats other than the Docker configuration file.
2. The library will not handle encryption of credentials.

Challenges

1. Depending on the version of the Docker CLI installed on the target machine, the format of the Docker configuration file may be different. The library should ensure that no config field is lost when saving credentials to the Docker configuration file.
2. Some clients like Notation need to read plain-text credentials from Docker configuration file, but do not want to save credentials in plain-text in the configuration file.

Proposal

Solution to challenges

1. To ensure that no config field is lost when saving credentials to the Docker configuration file, we can first unmarshal the json file into a json object instead of a fixed struct when parsing the configuration file. And then we can make some changes to the auths field of the json object, and marshal the updated json object back to the file. That way we can keep all the unknown fields in the configuration file.
2. Provide an option to allow users to disable saving credentials in plain-text in configuration files. If the option is set, reading credentials from configuration files will be allowed but saving will result in no operation.

Interfaces

We can define a basic interface for reading, saving and removing credentials as follows.

package credentials

// Store is the interface that any credentials store must implement.
type Store interface {
	// Put saves credentials into the store
	Put(ctx context.Context, serverAddress string, cred auth.Credential) error
	// Delete removes credentials from the store for the given server
	Delete(ctx context.Context, serverAddress string) error
	// Get retrieves credentials from the store for the given server
	Get(ctx context.Context, serverAddress string) (auth.Credential, error)
}

The auth.Credentials refers to Credential defined in the auth package of oras-go v2.

File Store

Based on the interface, we can further implement a FileStore for managing credentials stored in the Docker configuration file.

package credentials

// FileStore implements a credentials store using the docker configuration file
// to keep the credentials in plain-text.
type FileStore struct {
	configPath  string
	DisableSave bool
}

// NewFileStore creates a new file credentials store.
func NewFileStore(configPath string) Store {
	return &FileStore{
		configPath: configPath,
	}
}

// Put saves credentials into the store
func (fs *FileStore) Put(ctx context.Context, serverAddress string, cred auth.Credential) error {
	panic("not implemented") // TODO: Implement
}

// Delete removes credentials from the store for the given server
func (fs *FileStore) Delete(ctx context.Context, serverAddress string) error {
	panic("not implemented") // TODO: Implement
}

// Get retrieves credentials from the store for the given server
func (fs *FileStore) Get(ctx context.Context, serverAddress string) (auth.Credential, error) {
	panic("not implemented") // TODO: Implement
}

Native Store

Besides, we can also implement a NativeStore for managing credentials using a native credential store or credential helpers.

package credentials

const remoteCredentialsPrefix = "docker-credential-"

// NativeStore implements a credentials store
// using native keychain to keep credentials secure.
type NativeStore struct {
	programFunc client.ProgramFunc
}

// NewNativeStore creates a new native store that uses a remote helper program to
// manage credentials.
func NewNativeStore(helperSuffix string) Store {
	return &NativeStore{
		programFunc: client.NewShellProgramFunc(remoteCredentialsPrefix + helperSuffix),
	}
}

// Put saves credentials into the store
func (ns *NativeStore) Put(ctx context.Context, serverAddress string, cred auth.Credential) error {
	panic("not implemented") // TODO: Implement
}

// Delete removes credentials from the store for the given server
func (ns *NativeStore) Delete(ctx context.Context, serverAddress string) error {
	panic("not implemented") // TODO: Implement
}

// Get retrieves credentials from the store for the given server
func (ns *NativeStore) Get(ctx context.Context, serverAddress string) (auth.Credential, error) {
	panic("not implemented") // TODO: Implement
}

The client.ProgramFunc refers to the ProgramFunc defined in the package client of docker-credential-helper.

Utility Methods

We can provide some common utility methods for convenience. The method names can be determined later.

NewStore()

This method is to return a new credential store based on the settings in the configuration file.
The method should look for the credential store for a given server address in the order of credential helper, credential store and configuration file.
The method should provide an option AllowPlainText to allow users to specify whether to save credentials in plain-text. If the native store is not available, when the option is set to false (default value), NewStore().Save() will return an error; when the option is set to true, NewStore().Save() will save the credential in plain-text in the configuration file.

package credentials

// StoreOptions provides options for NewStore.
type StoreOptions struct {
	// AllowPlainText allows saving credentials in plain text in configuration file.
	AllowPlainText bool
}

// NewStore returns a new store from the settings in the configuration
// file.
func NewStore(configPath string, opts StoreOptions) Store {
	panic("not implemented") // TODO: Implement
}

NewStoreFromDocker()

This method is to return a store from the default docker config file.

package credentials

// NewStoreFromDocker returns a store from the default docker config file.
func NewStoreFromDocker(opts StoreOptions) Store {
	panic("not implemented") // TODO: Implement
}

NewStoreWithFallbacks()

This method is to return a new store based on the given stores. The second and the subsequent stores will be used as fallbacks for the first store.

package credentials

// NewStoreWithFallbacks returns a new store based on the given stores.
// The second and the subsequent stores will be used as fallbacks for the first store.
func NewStoreWithFallbacks(store Store, fallbacks ...Store) Store {
	panic("not implemented") // TODO: Implement
}

Login()

This method is to log a registry in.

package credentials

func Login(ctx context.Context, store Store, registry remote.Registry, cred auth.Credential) error {
	panic("not implemented") // TODO: Implement
}

Logout()

This method is to log a registry out.

package credentials

func Logout(ctx context.Context, store Store, registryName string) error {
	panic("not implemented") // TODO: Implement
}

Credential()

This method is to return a Credential function that can be used by auth.Client of oras-go v2.

package credentials

func Credential(store Store) func(context.Context, string) (auth.Credential, error) {
	panic("not implemented") // TODO: Implement
}

Additional Requirements

1. The library should support legacy auth config keys (See Credential library support for legacy auth config keys #1)
2. The library should support DockerHub-specific domain redirection rules (See @qweeah's comments):
   • Redirect credential GETs for docker.io to registry-1.docker.io
   • Redirect credential PUTs for registry-1.docker.io to https://index.docker.io/v1/

References

• Project proposal: SDK for `oras login` oras-go#413
• Design requirements #2
• Docker Credentails Store
• docker/cli/config

[shizhMSFT]

Overall LGTM.

A few suggestions:

• FileStore requires minor changes. We can discuss that offline or comment in future PRs.
• NewStore would be better than the function name GetConfiguredStore.
• Those examples in the Usage section is very common. We should make them available in oras-credentials-go as exported methods
  • credentials.Login() to login a registry
  • credentials.Logout() to logout a registry
  • credentials.Credentials(store Store) func(context.Context, string) (auth.Credential, error) to be used by oras-go more easily
• Should we add ctx context.Context to the Store interface?
• Should we drop the dependency of docker-credential-helper?
  • os/exec.CommandContext is preferred over os/exec.Command
  • There is no need to use execabs as the security patch is applied to os/exec since golang 1.19 (see release note)

[Wwwsylvia]

Should we add ctx context.Context to the Store interface?

Yes, will add.

Should we drop the dependency of docker-credential-helper?

I think we can start with docker-credential-helper in early versions, and gradually remove the dependency in the future.

[patrickzheng200]

Got a question regarding:

type GetStoreOptions struct {
    // Disable saving credentials in plain text in configuration file.
    DisablePlainTextSave bool
}

Shall we make DisablePlainTextSave default to true? something like:

type GetStoreOptions struct {
    // Enable saving credentials in plain text in configuration file.
    PlainTextSave bool
}

[Wwwsylvia]

I think most clients would want to save the credential in plain text, except for notation?

[imjasonh]

How will credentials be stored in non-plaintext when this option is set? Using credsStore?

As a total outside bystander, I think I'd prefer to use a library that has secure defaults wherever possible.

[Wwwsylvia]

@patrickzheng200 @imjasonh Thank you for your inputs! I've renamed the option to PlainTextSave and the updated the doc. Please see the "Challenges" and "NewStore()" sections.
My thinking is that, when the native store is not available, if the option is set to false (default value), calling NewStore().Save() will result in no operation; if the option is set to true, calling NewStore().Save() will save the credential in plain text in the configuration file.
Do you think this make sense? Or do you have any ideas about ensuring the security?

[shizhMSFT]

Secure by default is a good practice.

PlainTextSave is a good attempt. How about AllowPlaintext?

If we don't allow plaintext and there is no native store, we should return error on save.

[imjasonh]

Is it a goal or non-goal of this design to support Podman's auth config as well, which uses a file of the same structure, but at a different location (${XDG_RUNTIME_DIR}/containers/auth.json)

We've found some Podman users do alias docker=podman (as suggested in some Podman docs) then are surprised when docker login (which is podman login) writes the config file to a different location, and third-party tools (like crane, notation, etc.) don't know to look for them there. In go-containerregistry we fallback to look for Podman's auth to support this case.

I think it's fine to say this is not something you want to support, I just wonder whether the design should take it into account and explicitly state it as out-of-scope.

[Wwwsylvia]

Or do you mean we should have a default store reading some built-in paths, and add the Podman's path into the built-in paths?

[imjasonh]

Yeah, I think there's a few layered questions basically:

1. is there a Store implementation that can read from a specified file? (yes)
2. is there a Store implementation that can read from a list of files until one is found?
3. is there a default Store implementation that can read from a standard set of locations, including the Podman location?

[qweeah]

+1 on default to read auth configuration from a standard set of locations.

[Wwwsylvia]

Thanks for the suggestions!

[shizhMSFT]

podman support is not the goal of the first milestone of this project. We need to do further investigation to support podman in future milestones.

[TerryHowe]

Yeh, the path to the file needs to be flexible. Helm was using a different path for the config.json file at least at some point, not sure if they still are.

It would be nice if certs (for auth and validation) were handled in the credentials files as well, but maybe future scope.

PR to review would be easier IMO

[imjasonh]

My suggestion might actually be to allow multiple config file paths, and search them in order. In go-containerregistry we look for registry creds in $HOME/.docker/config.json, then $DOCKER_CONFIG/.docker/config.json, then $XDG_RUNTIME_DIR/containers/auth.json until we find a config file, and then fall back to anonymous requests if none of those are found.

You can see what we do here: https://github.com/google/go-containerregistry/blob/46488f7da0c9176882691e8a42e242191f4128ca/pkg/authn/keychain.go#L65 if that helps

We also rely on github.com/docker/cli/cli/config to standardize parsing the file, which I would recommend.

If you're interested we could even unify this implementation so that there's one fewer credential config search implementations in the world, up to you.

[TerryHowe]

I wasn't around when this project was conceived, but I would think part of the goal of this project would be to get rid of docker dependencies. I could be totally off though.

[Wwwsylvia]

Agreed with @imjasonh that we should support multiple config file paths. I didn't mention this in the doc, but I've just added it now in the "NewNStore()" section.
@TerryHowe I think one of the initial goals is to remove the dependency on docker CLI. @shizhMSFT can confirm.

[imjasonh]

I think it's an okay decision either way to decide to rely or not rely on docker/cli to parse the file. I will say there's one nice thing about depending on Docker's implementation, which is that if they add new features then you might be able to pick them up "for free".

For example, cfg.GetAuthConfig takes a registry server URL and returns the auth config for it, which handles credHelpers, plaintext auths, and interactions with the configured credentialStore, if any. If Docker adds a fourth way to configure creds and supports it in GetAuthConfig, callers relying on that logic will get that behavior right away. Users who are using docker push will also be able to oras push using the same auth. If you implement your own, there will be some delay between oras push having the same behavior as docker push.

There may also be subtle behavior differences (for example, different preference ordering between the three ways to provide auth) that might be frustrating to users and difficult to debug.

I still think it's fine to want to drop the dependency on docker/cli, but there are at least trade-offs worth noting.

[shizhMSFT]

Yes, dropping the dependency on docker CLI is one goal of this project.

[TerryHowe]

I think the verbs could have more agreement. I think this set would be harder to remember then basics like Write/Read/Delete

[Wwwsylvia]

Yeah these names might not be very ideal but I just wanted them to be somewhat consistent with docker-credential-helpers.

[TerryHowe]

Makes sense, but I wonder if this is a good time to break a bad pattern

[shizhMSFT]

How about Put / Get / Delete?

[Wwwsylvia]

Put / Get / Delete sounds good to me.

[Wwwsylvia]

@shizhMSFT @imjasonh @TerryHowe @patrickzheng200 Thanks everyone for your helpful inputs! I've just updated the doc and will submit corresponding PRs later.

[qweeah]

5. The library should be able to replace the following credential modules:
   • oras: https://github.com/oras-project/oras/tree/main/internal/credential

oras has Dockerhub-specific domain redirection rules:

1. if a credential request is sent to docker.io, redirect to registry-1.docker.io
2. if a credential is saved to registry-1.docker.io, save it to https://index.docker.io/v1/

Will 1 and 2 be included in oras-credential-go?

[shizhMSFT]

@Wwwsylvia They must be included.

[Wwwsylvia]

Thanks for the inputs. Yes we will include that.

[shizhMSFT]

It seems serverAddress is not unnecessary for NewStore() as Store.Get() takes the serverAddress

func NewStore(configPath, serverAddress string, opts StoreOptions) Store

Also, instead of NewNStore,

func NewNStore(configPaths []string, opts StoreOptions) Store

you may want to have something like

func MultiStore(store ...Store) Store

Again, you may want

func NewDockerStore(opts StoreOptions) Store

to cover NewClientWithDockerFallback from oras-go v1.

[shizhMSFT]

There is a strange behavior on NewStore:

store := NewStore("config.json", "registry.example.com")
cred, err := store.Get(ctx, "foobar.example.com")

The above code works but should not.

[shizhMSFT]

The problem of NewNStore is that it cannot handle other input source such as DOCKER_CONFIG.

[shizhMSFT]

The name of MultiStore comes from io.MultiReader and io.MultiWriter. The behavior of MultiStore can follow docker.NewClient() from oras-go v1.

[shizhMSFT]

Maybe FallbackStore is a better name for MultiStore.

[Wwwsylvia]

Maybe StoreWithFallbacks?

[Wwwsylvia]

Updated per comments.