Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add stiching option for the payload #86

Open
kortemik opened this issue Aug 1, 2023 · 2 comments
Open

add stiching option for the payload #86

kortemik opened this issue Aug 1, 2023 · 2 comments

Comments

@kortemik
Copy link
Member

kortemik commented Aug 1, 2023

container runtime (dockerd at least) may split long (>32K?) lines, split line does not include a newline character at the end of the "log" value:

here:

{"log":"01.08.2023 11:47:25.018 [INFO] [com.teragrep.k8s_01.KubernetesLogReader] [main] Found 1 monitored logfile definitions, reading them with maximum of 2 threads each.\n","stream":"stdout","time":"2023-08-01T11:47:25.018743252Z"}
ch.\n

if this is missing the lines should be merged.

it may require support from https://github.com/teragrep/rlo_13 because k8s_01 is stateless in that sense.

@StrongestNumber9
Copy link
Contributor

I am not sure if rlo_13 can detect this as it requires processing from k8s_01 to read the log, figure out if it ends in newline and then stitch it with the next one. But what if the log timestamps mismatch on these two events as that is fairly precise? How are the events actually combined?

@StrongestNumber9
Copy link
Contributor

This most likely requires rlo_13 to have a plugin support where the component using it controls whether event is finished or not

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants