PodLogsFollower #104

otaviof · 2022-03-25T12:23:43Z

Changes

Refactoring the PodFollower into PodLogsFollower, a component focused on the pod lifecycle, capable of both streaming live logs and read logs from an existing pod
shp buildrun logs waits a few seconds for the pod, instead of error out immediately
Refactoring BuildRun inspections into a single component
Introducing FakeParams to simplify unit-testing
Refactoring Params to avoid cyclic-dependencies
Minor documentation updates

Fixes #103

Submitter Checklist

Includes tests if functionality changed/was added
Includes docs if changes are user-facing
Set a kind label on this PR
Release notes block has been filled in, or marked NONE

Release Notes

Able to both stream and read all logs from a existing BuildRun, therefore the --follow/-F flag has been removed from "shp buildrun logs"

otaviof · 2022-03-28T12:33:44Z

/assign @gabemontero @HeavyWombat

docs/shp_buildrun_logs.md

gabemontero

OK the changes are substantial enough @otaviof that I may just have to check out the branch and trace some end to end flows to get my bearings. The git file diffs, even with the different commits, are a bit hard for me to follow. That said, if my hunches are right, I like what you have done.

Before I go down that path, a couple of high level questions that might avoid more churn on my part:

am I correct in thinking that -F was removed because you simply "follow by default" and the logic figures out if the pod is finished before the watch apparatus is fully set up. If so, highlighting that in the release note and PR description would be very helpful.
per Can't tail buildruns. (buildrun create, buildrun logs) #80 (comment) can you construct the command sequence now used to follow logs after running shp buildrun create -n build --buildref-name foo .... (the answer here might confirm what I just said in (1)).

thanks

gabemontero · 2022-03-28T20:57:19Z

pkg/shp/cmd/buildrun/logs.go

-		if len(pods.Items) == 0 {
-			fmt.Fprintf(ioStreams.ErrOut, "no builder pod found for BuildRun %q\n", c.name)
-			return false, nil
+		c.podLogsFollower, err = follower.NewPodLogsFollowerFromParams(ctx, p, pw, ioStreams)


ok so I see some form of log following still occurring

Now from what I recall Params are set from the cli flags. so I'm still not sure how this reconciles with removing -F

Please consider my latest comment. Thanks for pointing that out!

openshift-ci · 2022-03-31T06:28:50Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please ask for approval from gabemontero after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

- Refactoring the former `PodFollower` to become `PodLogsFollower`, focused on the pod lifecycle, also able to both stream live logs or read all logs from a existing pod - Refactoring `BuildRun` inspections into a single component - Introducing `FakeParams` to simplify unit-testing and refactoring `Params` to avoid cycle dependencies

otaviof · 2022-04-01T12:19:17Z

Before I go down that path, a couple of high level questions that might avoid more churn on my part:

1. am I correct in thinking that `-F` was removed because you simply "follow by default" and the logic figures out if the pod is finished before the watch apparatus is fully set up.  If so, highlighting that in the release note and PR description would be very helpful.

I need to go back on this @gabemontero. Testing the behavior on main branch, it shows a different outcome during on using the --follow flag.

I traced back the GetPodLogs() usage, and consolidated it the new PodLogsFollower, with the same behavior we see today on main branch.

Please consider the latest changes.

SaschaSchwarze0

Nice, this cleans up some things like the global follow instance.

SaschaSchwarze0 · 2022-04-02T13:27:03Z

pkg/shp/cmd/buildrun/inspect_buildrun.go

+	case br != nil && !br.HasStarted():
+		fmt.Fprintf(ioStreams.ErrOut, "BuildRun %q has been marked as failed.\n", name)


Pseude code:

Suggested change

case br != nil && !br.HasStarted():

fmt.Fprintf(ioStreams.ErrOut, "BuildRun %q has been marked as failed.\n", name)

case br != nil && !br.HasStarted():

fmt.Fprintf(ioStreams.ErrOut, "BuildRun %q has failed to start because of %q.\n", name, br.Status.Conditions[type=Succeeded].Message)

SaschaSchwarze0 · 2022-04-02T13:32:22Z

pkg/shp/follower/follower.go

+
+	p.log("Pod %q in %q state, starting up log tail\n", pod.GetName(), corev1.PodRunning)
+
+	time.Sleep(3 * time.Second)


I assume we wait here so that the user can read the message that was just printed? Is worth a comment, I think.

If possible, I would like to remove the sleep here. One can always scroll up to see the beginning. An artificial wait feels not right as it may slow down a response/result I would like to see.

SaschaSchwarze0 · 2022-04-02T13:38:21Z

pkg/shp/follower/follower.go

+	logTail        *tail.Tail      // helper to tail container logs
+	logTailStarted map[string]bool // containers with logTail started
+
+	onlyOnce bool // retrieve the pod logs only once


This irritated me a little bit. If I understand it correctly, then this PodLogsFollower is capable to retrieve logs with and without following which is controlled by this flag. If that's the case, what about

renaming PodLogsFollower to PodLogsRetriever

renaming onlyOnce to follow

Alternatively, we could also just change onlyOnce to noFollow as a simple solution (but then the struct name would be confusing).

HeavyWombat · 2022-04-04T07:54:06Z

docs/testing.md


 ```sh
-make build test-e2e
+make test-e2e 


Nit: trailing space

Suggested change

make test-e2e

make test-e2e

HeavyWombat · 2022-04-04T11:26:34Z

pkg/shp/cmd/build/delete.go

-func (c *DeleteCommand) Complete(params *params.Params, io *genericclioptions.IOStreams, args []string) error {
+func (c *DeleteCommand) Complete(p params.Interface, io *genericclioptions.IOStreams, args []string) error {
+	if len(args) != 1 {
+		return errors.New("build name is not informed")


Not sure here, could be that I am missing something. My thought was just if it is != to one, it could be that the user provided multiple build names thinking this could be supported. Therefore, the error message would be or could be misleading. We could consider using the Cobra setting to define the expected number of arguments.

HeavyWombat · 2022-04-04T11:28:43Z

pkg/shp/cmd/build/run.go

-		br.GetName(),
-	)}
-	_, err = r.follower.Start(listOpts)
+	buildNameLabel := fmt.Sprintf("%s=%s", buildv1alpha1.LabelBuild, r.buildName)


We could do it now, or keep it for later, but with embedded BuildSpecs (aka one-off builds) we will have TaskRuns with only the buildrun label, not the build label since there is no build.

HeavyWombat · 2022-04-04T11:33:57Z

pkg/shp/cmd/build/upload.go

-	// buildNameAnnotation label to identify the Build name.
-	buildNameAnnotation = "build.shipwright.io/name"
-	// buildRunNameAnnotation label to identify the BuildRun name.
-	buildRunNameAnnotation = "buildrun.shipwright.io/name"


HeavyWombat · 2022-04-04T11:53:14Z

pkg/shp/follower/follower.go

+
+	p.log("Pod %q in %q state, starting up log tail\n", pod.GetName(), corev1.PodRunning)
+
+	time.Sleep(3 * time.Second)


If possible, I would like to remove the sleep here. One can always scroll up to see the beginning. An artificial wait feels not right as it may slow down a response/result I would like to see.

HeavyWombat · 2022-04-04T12:28:25Z

pkg/shp/reactor/pod_watcher_test.go

 	})

 	pw.Start(metav1.ListOptions{})
-	g.Expect(called).To(BeTrue())
+	g.Expect(called > 0).To(BeTrue())


Nit: Gomega

Suggested change

g.Expect(called > 0).To(BeTrue())

g.Expect(called).To(BeNumerically(">", 0))

SaschaSchwarze0 · 2022-04-04T13:15:06Z

/hold

Making sure this does not go into v0.10 release.

openshift-ci · 2022-04-09T22:18:49Z

@otaviof: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

otaviof added the kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. label Mar 25, 2022

openshift-ci bot added the release-note label Mar 25, 2022

openshift-ci bot requested review from HeavyWombat and imjasonh March 25, 2022 12:23

Vendoring in dependencies

020a90f

otaviof force-pushed the pod-logs-follower branch 7 times, most recently from 15a0c76 to 30ab1c9 Compare March 27, 2022 16:54

openshift-ci bot assigned gabemontero and HeavyWombat Mar 28, 2022

gabemontero reviewed Mar 28, 2022

View reviewed changes

docs/shp_buildrun_logs.md Outdated Show resolved Hide resolved

gabemontero reviewed Mar 28, 2022

View reviewed changes

otaviof added 2 commits March 31, 2022 08:27

E2E Documentation

3f6f647

Renaming Follower

537e493

otaviof force-pushed the pod-logs-follower branch from 30ab1c9 to 74293a1 Compare March 31, 2022 06:28

otaviof force-pushed the pod-logs-follower branch from 74293a1 to 8567f1f Compare March 31, 2022 06:51

otaviof force-pushed the pod-logs-follower branch from 8567f1f to 53f0df2 Compare March 31, 2022 06:56

otaviof changed the title ~~PodLogsFollower and Params Refactoring~~ PodLogsFollower Mar 31, 2022

SaschaSchwarze0 requested changes Apr 2, 2022

View reviewed changes

HeavyWombat reviewed Apr 4, 2022

View reviewed changes

SaschaSchwarze0 added this to the release-v0.10.0 milestone Apr 4, 2022

openshift-ci bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Apr 4, 2022

This was referenced Apr 19, 2022

Flaky Unit Test Results #103

Closed

fix flakes in log following related unit tests #112

Merged

SaschaSchwarze0 modified the milestones: release-v0.10.0, release-v0.11.0 Jun 1, 2022

SaschaSchwarze0 modified the milestones: release-v0.11.0, release-v0.12.0 Sep 8, 2022

SaschaSchwarze0 modified the milestones: release-v0.12.0, release-v0.13.0 Oct 26, 2023

SaschaSchwarze0 removed this from the release-v0.13.0 milestone Aug 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PodLogsFollower #104

PodLogsFollower #104

otaviof commented Mar 25, 2022 •

edited

Loading

otaviof commented Mar 28, 2022

gabemontero left a comment

gabemontero Mar 28, 2022

otaviof Apr 1, 2022

openshift-ci bot commented Mar 31, 2022

otaviof commented Apr 1, 2022

SaschaSchwarze0 left a comment

SaschaSchwarze0 Apr 2, 2022

SaschaSchwarze0 Apr 2, 2022

HeavyWombat Apr 4, 2022

SaschaSchwarze0 Apr 2, 2022 •

edited

Loading

HeavyWombat Apr 4, 2022

HeavyWombat Apr 4, 2022

HeavyWombat Apr 4, 2022

HeavyWombat Apr 4, 2022

HeavyWombat Apr 4, 2022

HeavyWombat Apr 4, 2022

SaschaSchwarze0 commented Apr 4, 2022

openshift-ci bot commented Apr 9, 2022

		case br != nil && !br.HasStarted():
		fmt.Fprintf(ioStreams.ErrOut, "BuildRun %q has been marked as failed.\n", name)


		p.log("Pod %q in %q state, starting up log tail\n", pod.GetName(), corev1.PodRunning)

		time.Sleep(3 * time.Second)

	g.Expect(called > 0).To(BeTrue())
	g.Expect(called).To(BeNumerically(">", 0))

PodLogsFollower #104

Are you sure you want to change the base?

PodLogsFollower #104

Conversation

otaviof commented Mar 25, 2022 • edited Loading

Changes

Submitter Checklist

Release Notes

otaviof commented Mar 28, 2022

gabemontero left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-ci bot commented Mar 31, 2022

otaviof commented Apr 1, 2022

SaschaSchwarze0 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SaschaSchwarze0 Apr 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SaschaSchwarze0 commented Apr 4, 2022

openshift-ci bot commented Apr 9, 2022

otaviof commented Mar 25, 2022 •

edited

Loading

SaschaSchwarze0 Apr 2, 2022 •

edited

Loading