Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Offer unlimited log buffer size when scanning handler logs #159

Merged
merged 1 commit into from
Aug 26, 2024

Conversation

alexellis
Copy link
Member

@alexellis alexellis commented Aug 26, 2024

Description

Offer unlimited log buffer size when scanning handler logs

Motivation and Context

Some users have complained that they cannot log lines longer than 16KB. A 16KB log line seems excessive, but this has been requested at least 2-3 times over the years. For example: #100

To enable the feature, set log_buffer_size to -1. This is not a default, because it is not as efficient as using a pre- determined buffer size.

How Has This Been Tested?

Tested by running curl with --data-binary and sending a 6.7M Go binary into a function with the 16KB maximum log line size set. That produced the error, when the size was set to -1, the output was written, albeit quite slowly.

The error before the changes:

Screenshot 2024-08-26 at 16 47 15

Sample function which prints any input HTTP body to stderr

package function

import (
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"
)

type Res struct {
	Body string `json:"body"`
}

func Handle(w http.ResponseWriter, r *http.Request) {
	var input []byte

	if r.Body != nil {
		defer r.Body.Close()

		body, _ := io.ReadAll(r.Body)

		input = body
	}

	jres, err := json.Marshal(Res{Body: string(input)})
	if err != nil {
		http.Error(w, err.Error(), http.StatusInternalServerError)
		return
	}

	log.Printf("%s", string(jres))

	w.WriteHeader(http.StatusOK)
	w.Write([]byte(fmt.Sprintf("Body: %s", string(input))))
}

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I've read the CONTRIBUTION guide
  • I have signed-off my commits with git commit -s
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Some users have complained that they cannot log lines longer
than 16KB. A 16KB log line seems excessive, but this has been
requested at least 2-3 times over the years.

To enable the feature, set log_buffer_size to -1. This is not
a default, because it is not as efficient as using a pre-
determined buffer size.

Tested by running curl with --data-binary and sending a 6.7M
Go binary into a function with the 16KB maximum log line size
set. That produced the error, when the size was set to -1, the
output was written, albeit quite slowly.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <[email protected]>
@alexellis alexellis merged commit f9aec5c into master Aug 26, 2024
2 checks passed
@alexellis alexellis deleted the unbuffered_log_size branch August 26, 2024 16:24
alexellis added a commit to openfaas/templates that referenced this pull request Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant