refactor: status, metadata and content handlers for `manifest index` commands #1509

wangxiaoxuan273 · 2024-09-24T08:55:43Z

What this PR does / why we need it:
Implements output handlers for manifest index commands.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1503

Please check the following list:

Does the affected code have corresponding tests, e.g. unit test, E2E test?
Does this change require a documentation update?
Does this introduce breaking changes that would require an announcement or bumping the major version?
Do all new files have an appropriate license header?

Signed-off-by: Xiaoxuan Wang <[email protected]>

codecov · 2024-09-24T08:58:28Z

Codecov Report

Attention: Patch coverage is 63.10160% with 69 lines in your changes missing coverage. Please review.

Project coverage is 84.86%. Comparing base (304a89f) to head (3a2fb99).

Files with missing lines	Patch %	Lines
cmd/oras/root/manifest/index/update.go	42.85%	12 Missing and 12 partials ⚠️
cmd/oras/root/manifest/index/create.go	39.13%	7 Missing and 7 partials ⚠️
cmd/oras/internal/display/status/discard.go	25.00%	12 Missing ⚠️
cmd/oras/internal/display/metadata/discard.go	42.85%	8 Missing ⚠️
cmd/oras/internal/display/status/text.go	75.00%	6 Missing ⚠️
...nal/display/metadata/text/manifest_index_update.go	85.71%	2 Missing and 1 partial ⚠️
...md/oras/internal/display/content/manifest_index.go	87.50%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1509      +/-   ##
==========================================
- Coverage   86.01%   84.86%   -1.15%     
==========================================
  Files         118      120       +2     
  Lines        4225     4348     +123     
==========================================
+ Hits         3634     3690      +56     
- Misses        353      399      +46     
- Partials      238      259      +21

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Xiaoxuan Wang <[email protected]>

cmd/oras/internal/display/content/manifest_index.go

Signed-off-by: Xiaoxuan Wang <[email protected]>

cmd/oras/root/manifest/index/create.go

cmd/oras/internal/display/content/manifest_index.go

cmd/oras/internal/display/metadata/text/manifest_index_create.go

cmd/oras/internal/display/metadata/discard.go

cmd/oras/internal/display/metadata/interface.go

cmd/oras/internal/display/status/interface.go

qweeah · 2024-09-26T04:58:32Z

cmd/oras/internal/display/status/interface.go

+}
+
+// ManifestIndexUpdateHandler handles status output for manifest index update command.
+type ManifestIndexUpdateHandler interface {


Contains duplicated methods ManifestIndexCreateHandler as below

OnIndexFetching(indexRef string) error OnIndexFetched(indexRef string, digest digest.Digest) error

You can create a new interface and use it in both interfaces.

Created an interface ManifestReferenceFetchHandler and both index create and index update handlers contain it.

cmd/oras/internal/display/handler.go

qweeah · 2024-09-26T06:20:45Z

cmd/oras/root/manifest/index/update.go

+			// ignore --pretty when output to a file
+			if opts.outputPath != "" && opts.outputPath != "-" {
+				opts.Pretty.Pretty = false
+			}


Can be merged into the handler construction.

If this is a general option thing, maybe a method in otpions?

If this is a general option thing, maybe a method in otpions?

@TerryHowe We can move it to content handler to avoid code repeat.

TerryHowe · 2024-09-27T17:50:46Z

cmd/oras/internal/display/content/manifest_index.go

+// OnContentCreated is called after index content is created.
+func (h *manifestIndexCreate) OnContentCreated(manifest []byte) error {
+	out := h.stdout
+	if h.outputPath != "-" && h.outputPath != "" {


if you changed outputPath to "" for "-" this logic would look less odd.

Did you mean to put "" before "-"? Changed the if condition to if outputPath != "" && outputPath != "-"

TerryHowe · 2024-09-27T17:52:08Z

cmd/oras/internal/display/handler.go

+}
+
+// NewManifestIndexUpdateHandler returns status, metadata and content handlers for index update command.
+func NewManifestIndexUpdateHandler(outputPath string, printer *output.Printer, pretty bool) (


should there just be a NewManivestIndexHandler method for both for now and someone can add the other in the future if needed?

Currently we need both create and update handlers.

wangxiaoxuan273 · 2024-10-09T07:00:29Z

cmd/oras/internal/display/status/text.go

+	tmich := TextManifestIndexCreateHandler{
+		printer: printer,
+	}
+	return &tmich


The current naming style is aligned with the rest of the file.

wangxiaoxuan273 · 2024-10-09T07:00:36Z

cmd/oras/internal/display/status/text.go

+	miuh := TextManifestIndexUpdateHandler{
+		printer: printer,
+	}
+	return &miuh


The current naming style is aligned with the rest of the file.

cmd/oras/root/manifest/index/create.go

TerryHowe · 2024-09-27T18:11:21Z

cmd/oras/root/manifest/index/create.go

@@ -188,7 +197,8 @@ func getPlatform(ctx context.Context, target oras.ReadOnlyTarget, manifestBytes
 	return &platform, nil
 }

-func pushIndex(ctx context.Context, target oras.Target, desc ocispec.Descriptor, content []byte, ref string, extraRefs []string, path string, printer *output.Printer) error {
+func pushIndex(ctx context.Context, onIndexPushed func(path string) error, onTagged func(desc ocispec.Descriptor, tag string) error,


This is a private method here, seems like it would make more sense for it to take handlers than function pointers.

Another thing, it seems like an extremely long list of params like something is not right here.

Since both create and update use this method and index create handlers and index update handlers have different types, the function argument can't take handler argument.

Yeah this is an extremely long list of parameters, but I can't find another way :(

TerryHowe · 2024-09-27T18:13:44Z

cmd/oras/root/manifest/index/update.go

+			// ignore --pretty when output to a file
+			if opts.outputPath != "" && opts.outputPath != "-" {
+				opts.Pretty.Pretty = false
+			}


If this is a general option thing, maybe a method in otpions?

Signed-off-by: Xiaoxuan Wang <[email protected]>

wangxiaoxuan273 · 2024-10-09T07:00:29Z

cmd/oras/internal/display/status/text.go

+	tmich := TextManifestIndexCreateHandler{
+		printer: printer,
+	}
+	return &tmich


The current naming style is aligned with the rest of the file.

wangxiaoxuan273 · 2024-10-09T07:00:36Z

cmd/oras/internal/display/status/text.go

+	miuh := TextManifestIndexUpdateHandler{
+		printer: printer,
+	}
+	return &miuh


The current naming style is aligned with the rest of the file.

Signed-off-by: Xiaoxuan Wang <[email protected]>

cmd/oras/internal/display/metadata/text/manifest_index.go

cmd/oras/root/manifest/index/update.go

qweeah · 2024-10-11T06:12:59Z

cmd/oras/internal/display/metadata/text/manifest_index.go

+
+// ManifestIndexUpdateHandler handles text metadata output for index update events.
+type ManifestIndexUpdateHandler struct {
+	printer *output.Printer


Can you help separate ManifestIndexUpdateHandler and ManifestIndexCreateHandler into manifest_index_update.go and manifest_index_create.go?

qweeah · 2024-10-11T06:15:16Z

cmd/oras/internal/display/metadata/interface.go

+	OnManifestRemoved(digest digest.Digest) error
+	OnManifestAdded(manifestRef string, digest digest.Digest) error
+	OnIndexMerged(indexRef string, digest digest.Digest) error
 }


Change digest to descriptor since it will be used for formatted output in the future.

The parameter of OnManifestRemoved(digest digest.Digest) error must be a digest, since the descriptor is unknown. Changed the parameter of the other two functions as descriptors.

qweeah · 2024-10-11T06:19:53Z

cmd/oras/internal/display/handler.go

+	status.ManifestIndexCreateHandler,
+	metadata.ManifestIndexCreateHandler,
+	content.ManifestIndexCreateHandler,
+	error) {


When will an error be returned?

good catch. Removed the error return.

qweeah · 2024-10-11T06:20:56Z

cmd/oras/internal/display/status/utils.go

@@ -52,12 +52,6 @@ const (
 const (
 	IndexPromptFetching = "Fetching "


Extra space, same for fetched.

Suggested change

IndexPromptFetching = "Fetching "

IndexPromptFetching = "Fetching"

We are not doing status output indentation align anymore?

Yes, there are still aligning work but there is trailing space for both Fetching and Fetched. Should remove one space from both prompts.

qweeah · 2024-10-11T06:36:33Z

cmd/oras/root/manifest/index/create.go

+func pushIndex(ctx context.Context, onIndexPushed func(path string) error, onTagged func(desc ocispec.Descriptor, tag string) error,
+	target oras.Target, desc ocispec.Descriptor, content []byte, ref string, extraRefs []string, path string) error {


Try using handlers like:

Suggested change

func pushIndex(ctx context.Context, onIndexPushed func(path string) error, onTagged func(desc ocispec.Descriptor, tag string) error,

target oras.Target, desc ocispec.Descriptor, content []byte, ref string, extraRefs []string, path string) error {

func pushIndex(ctx context.Context, pmh metadata.ManifestPackHandler, th metadata.TaggedHandler,target oras.Target, desc ocispec.Descriptor, content []byte, ref string, extraRefs []string, path string) error {

shizhMSFT · 2024-10-11T06:39:58Z

cmd/oras/internal/display/content/discard.go

 // NewDiscardHandler returns a new discard handler.
-func NewDiscardHandler() ManifestFetchHandler {
+func NewDiscardHandler() discardHandler {


Should we export discardHandler just like status.DiscardHandler?

shizhMSFT · 2024-10-11T06:52:19Z

cmd/oras/internal/display/content/manifest_index.go

+		if err != nil {
+			return fmt.Errorf("failed to open %q: %w", h.outputPath, err)
+		}
+		defer f.Close()


We need to create an issue to track handling errors returned by Close() in all write paths.

shizhMSFT · 2024-10-11T06:54:35Z

cmd/oras/internal/display/content/manifest_index.go

+	// ignore --pretty when output to a file
+	if outputPath != "" && outputPath != "-" {
+		pretty = false
+	}


It is different from NewManifestFetchHandler. Is NewManifestFetchHandler wrong? /cc @qweeah

NewManifestFetchHandler is used only once by oras manifest fetch and such code snippet is in command code

oras/cmd/oras/root/manifest/fetch.go

Lines 80 to 83 in dcef719

// ignore --pretty when output to a file

case opts.outputPath != "" && opts.outputPath != "-":

opts.Pretty.Pretty = false

}

I think we need to remove it and change the flag reset to NewManifestFetchHandler too.

shizhMSFT · 2024-10-11T07:01:40Z

cmd/oras/internal/display/metadata/discard.go

+import (
+	"github.com/opencontainers/go-digest"
+	ocispec "github.com/opencontainers/image-spec/specs-go/v1"
+)

 type discard struct{}


Do you want to export it just like other discard handlers?

shizhMSFT · 2024-10-11T07:04:43Z

cmd/oras/internal/display/metadata/interface.go

+type manifestPackHandler interface {
+	OnIndexPacked(desc ocispec.Descriptor) error
+	OnIndexPushed(path string) error
+	OnCompleted(desc ocispec.Descriptor) error
+}


Do you want to merge this to ManifestIndexCreateHandler?

shizhMSFT · 2024-10-11T07:06:16Z

cmd/oras/internal/display/status/interface.go

+// referenceFetchHandler handles status output for reference fetch events.
+type referenceFetchHandler interface {
+	OnFetching(manifestRef string) error
+	OnFetched(manifestRef string, desc ocispec.Descriptor) error
+}


Do you want to merge it to ManifestIndexCreateHandler?

shizhMSFT · 2024-10-11T07:07:51Z

cmd/oras/internal/display/status/utils.go

 	IndexPromptFetching = "Fetching "
 	IndexPromptFetched  = "Fetched  "


Why those 2 are left?

Signed-off-by: Xiaoxuan Wang <[email protected]>

Xiaoxuan Wang added 5 commits September 24, 2024 10:39

create handler

182c8f0

Signed-off-by: Xiaoxuan Wang <[email protected]>

compiles

495ab7e

Signed-off-by: Xiaoxuan Wang <[email protected]>

align update output with tag

d81fd04

Signed-off-by: Xiaoxuan Wang <[email protected]>

align update output with digest

f807af8

Signed-off-by: Xiaoxuan Wang <[email protected]>

refinement

9134a8f

Signed-off-by: Xiaoxuan Wang <[email protected]>

wangxiaoxuan273 changed the title ~~Output output~~ refactor: status, metadata and content handlers for manifest index commands Sep 24, 2024

Xiaoxuan Wang added 3 commits September 24, 2024 17:04

added error checks

5104cc9

Signed-off-by: Xiaoxuan Wang <[email protected]>

content handlers

a9b1164

Signed-off-by: Xiaoxuan Wang <[email protected]>

refinement

4d144be

Signed-off-by: Xiaoxuan Wang <[email protected]>

wangxiaoxuan273 marked this pull request as ready for review September 25, 2024 05:57

wangxiaoxuan273 requested review from sajayantony, shizhMSFT, SteveLasker, qweeah and TerryHowe as code owners September 25, 2024 05:57

qweeah reviewed Sep 25, 2024

View reviewed changes

cmd/oras/internal/display/content/manifest_index.go Show resolved Hide resolved

fix pretty with file output

4d18845

Signed-off-by: Xiaoxuan Wang <[email protected]>

qweeah requested changes Sep 26, 2024

View reviewed changes

qweeah reviewed Sep 26, 2024

View reviewed changes

qweeah requested changes Sep 26, 2024

View reviewed changes

TerryHowe reviewed Sep 27, 2024

View reviewed changes

discard parameter naming

8ea9b14

Signed-off-by: Xiaoxuan Wang <[email protected]>

wangxiaoxuan273 commented Oct 9, 2024

View reviewed changes

Xiaoxuan Wang added 5 commits October 9, 2024 15:01

resolved comments

16cf5ab

Signed-off-by: Xiaoxuan Wang <[email protected]>

manifest ref fetch handler

c56fdec

Signed-off-by: Xiaoxuan Wang <[email protected]>

reset pretty in handler construction

49091a2

Signed-off-by: Xiaoxuan Wang <[email protected]>

resolved comments

66249a8

Signed-off-by: Xiaoxuan Wang <[email protected]>

Merge branch 'main' into output-output

6e3b09a

wangxiaoxuan273 commented Oct 11, 2024

View reviewed changes

cmd/oras/internal/display/metadata/text/manifest_index.go Outdated Show resolved Hide resolved

wangxiaoxuan273 commented Oct 11, 2024

View reviewed changes

cmd/oras/internal/display/metadata/text/manifest_index.go Outdated Show resolved Hide resolved

wangxiaoxuan273 commented Oct 11, 2024

View reviewed changes

cmd/oras/root/manifest/index/update.go Show resolved Hide resolved

qweeah requested changes Oct 11, 2024

View reviewed changes

shizhMSFT reviewed Oct 14, 2024

View reviewed changes

Xiaoxuan Wang added 2 commits October 14, 2024 14:55

resolved billy's comment

851701d

Signed-off-by: Xiaoxuan Wang <[email protected]>

Merge branch 'main' into output-output

3a2fb99

wangxiaoxuan273 force-pushed the output-output branch from 4daf7d6 to 3a2fb99 Compare October 14, 2024 07:09

qweeah mentioned this pull request Oct 14, 2024

fix: update --add should add plain descriptor #1513

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: status, metadata and content handlers for `manifest index` commands #1509

refactor: status, metadata and content handlers for `manifest index` commands #1509

wangxiaoxuan273 commented Sep 24, 2024

codecov bot commented Sep 24, 2024 •

edited

Loading

qweeah Sep 26, 2024

wangxiaoxuan273 Oct 9, 2024

qweeah Sep 26, 2024

TerryHowe Sep 27, 2024

wangxiaoxuan273 Oct 10, 2024

TerryHowe Sep 27, 2024

wangxiaoxuan273 Oct 10, 2024

TerryHowe Sep 27, 2024

wangxiaoxuan273 Oct 10, 2024

wangxiaoxuan273 Oct 9, 2024

wangxiaoxuan273 Oct 9, 2024

TerryHowe Sep 27, 2024

wangxiaoxuan273 Oct 10, 2024

TerryHowe Sep 27, 2024

wangxiaoxuan273 Oct 9, 2024

wangxiaoxuan273 Oct 9, 2024

qweeah Oct 11, 2024

qweeah Oct 11, 2024

wangxiaoxuan273 Oct 14, 2024

qweeah Oct 11, 2024

wangxiaoxuan273 Oct 14, 2024

qweeah Oct 11, 2024

wangxiaoxuan273 Oct 14, 2024

qweeah Oct 14, 2024

qweeah Oct 11, 2024

shizhMSFT Oct 11, 2024

shizhMSFT Oct 11, 2024

shizhMSFT Oct 11, 2024

qweeah Oct 14, 2024

shizhMSFT Oct 11, 2024

shizhMSFT Oct 11, 2024

shizhMSFT Oct 11, 2024

shizhMSFT Oct 11, 2024

		@@ -52,12 +52,6 @@ const (
		const (
		IndexPromptFetching = "Fetching "

	IndexPromptFetching = "Fetching "
	IndexPromptFetching = "Fetching"

		func pushIndex(ctx context.Context, onIndexPushed func(path string) error, onTagged func(desc ocispec.Descriptor, tag string) error,
		target oras.Target, desc ocispec.Descriptor, content []byte, ref string, extraRefs []string, path string) error {

	func pushIndex(ctx context.Context, onIndexPushed func(path string) error, onTagged func(desc ocispec.Descriptor, tag string) error,
	target oras.Target, desc ocispec.Descriptor, content []byte, ref string, extraRefs []string, path string) error {
	func pushIndex(ctx context.Context, pmh metadata.ManifestPackHandler, th metadata.TaggedHandler,target oras.Target, desc ocispec.Descriptor, content []byte, ref string, extraRefs []string, path string) error {

	// ignore --pretty when output to a file
	case opts.outputPath != "" && opts.outputPath != "-":
	opts.Pretty.Pretty = false
	}

		IndexPromptFetching = "Fetching "
		IndexPromptFetched = "Fetched "

refactor: status, metadata and content handlers for manifest index commands #1509

Are you sure you want to change the base?

refactor: status, metadata and content handlers for manifest index commands #1509

Conversation

wangxiaoxuan273 commented Sep 24, 2024

codecov bot commented Sep 24, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refactor: status, metadata and content handlers for `manifest index` commands #1509

refactor: status, metadata and content handlers for `manifest index` commands #1509

codecov bot commented Sep 24, 2024 •

edited

Loading