ci: add fluentd basic e2e tests #71
Conversation
ningziwen
commented
Sep 16, 2023
•
ningziwen
commented
- breaking out the 'log generation' piece out of the aws log test file into the main test file as func sendTestLogByContainerd so that it can be reused for every driver (which seems like a good idea)
- add a fluentd test that does validation
- adds fluentd to the GitHub action (note this launches a docker containerd with the fluentd server)
- adds fluentd test to makefile
ningziwen
force-pushed
the
fluentd
branch
6 times, most recently
from
9c41202
to
7ea617e
Compare
|
So looking at this PR. From what I can gather the majority of the work is as follows:
|
@sbuckfelder Thanks. Added to the description. |
ningziwen
force-pushed
the
fluentd
branch
3 times, most recently
from
2bc7f0e
to
755ec8d
Compare
Signed-off-by: Ziwen Ning <[email protected]>
| gomega.Expect(err).ShouldNot(gomega.HaveOccurred()) | ||
| gomega.Expect(code).Should(gomega.Equal(uint32(0))) | ||
| func validateTestLogsInAwslogs(client *cloudwatchlogs.Client, logGroupName string, logStreamName string, testLog string) { | ||
| cwOutput, err := client.GetLogEvents(context.TODO(), &cloudwatchlogs.GetLogEventsInput{ |
There was a problem hiding this comment.
Consider passing in a Ginkgo SpecContext so that this function can time out when the test times out. Same for cleanupAwslogs.
Probably not 100% necessary.
There was a problem hiding this comment.
Looking into the documentation. https://onsi.github.io/ginkgo/#spec-subjects-it
SpecContext is added immediately after ginkgo.It() such as
ginkgo.It("should send logs to fluentd log driver", func(ctx SpecContext) {}
It is usually used for long-running code so may not need for our case.
| return err | ||
| } | ||
| if strings.HasPrefix(info.Name(), "data.") && strings.HasSuffix(info.Name(), ".log") && info.Name() != "data.log" { | ||
| fileName = path |
There was a problem hiding this comment.
nit: you could add a return after this assignment if you only expect one file to match the if statement's condition
There was a problem hiding this comment.
Is there any difference of whether to return early if only one matched file is always expected?
| // 2023-09-16T22:54:11+00:00 123456789012 {"source":"stdout","log":"test-e2e-log","container_id":"123456789012","container_name":"test-container"} | ||
| // 2023-09-16T22:54:30+00:00 123456789012 {"container_id":"123456789012","container_name":"test-container","source":"stdout","log":"test-e2e-log"} | ||
| // 2023-09-16T22:56:17+00:00 123456789012 {"container_id":"123456789012","container_name":"test-container","source":"stdout","log":"test-e2e-log"} | ||
| // The following steps retrieves the "log" field of the third string parsed by tab of the last line to validate the tests sent. |
There was a problem hiding this comment.
I'm not clear as to why we only care about the last line's log when all 3 have the "log": "test-e2e-log" data. Could just be because I'm not familiar with this. Could you either add more explanation to the comment or just respond to this comment for my own knowledge?
There was a problem hiding this comment.
The log events are following time sequence. Each e2e test case only sends one log event. For now there is only one case so it doesn't matter. In the future, if there are multiple test cases, assuming they are running in serial, getting the last line could match the log the current test case just sent.
