Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Omitting client.Stop() prevents logs being pushed to loki server #15

Open
lfonosol opened this issue Apr 12, 2024 · 3 comments
Open

Omitting client.Stop() prevents logs being pushed to loki server #15

lfonosol opened this issue Apr 12, 2024 · 3 comments

Comments

@lfonosol
Copy link

Hello

I managed to get logs pushed to my loki grafana setup (with some trial and error) and I was wondering if it is necessary to call client.Stop() in order for the logs to be pushed to the loki server?

`func SetDefaultLogger() {
url := "http://localhost:3100/loki/api/v1/push"

config, lokiErr := loki.NewDefaultConfig(url)
if lokiErr != nil {
	panic(lokiErr)
}

client, lokiErr := loki.New(config)
if lokiErr != nil {
	panic(lokiErr)
}

// Is this necessary?
defer client.Stop()

option := slogloki.Option{Level: slog.LevelDebug, Client: client}
handler := option.NewLokiHandler()
logger := slog.New(handler).With(slog.String("job", "audit-log"))

slog.SetDefault(logger)

log.Print("log.Print")
logger.Debug("logger.Debug")
logger.Info("logger.Info")
logger.Warn("logger.Warn")
logger.Error("logger.Error")

}`

If I remove the line defer client.Stop() I don't see the messages in Grafana.

The problem for me is that my application will hang if the client is stopped and I try and call the default logger elsewhere in the code.

Is there a way I can set this default logger without having to call client.Stop()

@lfonosol lfonosol changed the title Behaviour of the client Omitting client.Stop() prevents logs being pushed to loki server Apr 12, 2024
@lfonosol
Copy link
Author

lfonosol commented Apr 12, 2024

I cloned the branch and created the following test

func TestNewLokiHandler_WhenDebugLevelNil_UsesDefaultLevel(t *testing.T) {

	config, _ := getDefaultConfig()
	client, _ := loki.New(config)

	option := Option{
		Level:  nil,
		Client: client,
	}
	handler := option.NewLokiHandler()

	// Omitting this causes leak
	client.Stop()
	assert.NotNil(t, handler)
}

func getDefaultConfig() (loki.Config, error) {
	return loki.NewDefaultConfig("http://localhost:3100/loki/api/v1/push")
}

When I omit the Client.Stop() line I get the following output:

goleak: Errors on successful test run: found unexpected goroutines:
[Goroutine 26 in state select, with github.com/grafana/loki-client-go/loki.(*Client).run on top of the stack:
goroutine 26 [select]:
github.com/grafana/loki-client-go/loki.(*Client).run(0xc00022f7c0)

Is this behaviour expected?

@samber
Copy link
Owner

samber commented Apr 12, 2024

If i remember well, the loki client put logs into a buffer and send a batch regularly.

Calling client.Stop() before program closes allows you to flush the buffer and perform a graceful stop.

If you remove this line, some logs might be dropped on exit.

@lfonosol
Copy link
Author

at the moment none of the messages are being pushed. I wonder if my buffer is set too large somehow?

I will investigate this further and post updates here in case others run into similar problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants