Skip to content
This repository has been archived by the owner on Mar 11, 2020. It is now read-only.

Memory leak when parse large request #25

Open
lfyzjck opened this issue Dec 17, 2014 · 0 comments
Open

Memory leak when parse large request #25

lfyzjck opened this issue Dec 17, 2014 · 0 comments

Comments

@lfyzjck
Copy link

lfyzjck commented Dec 17, 2014

func readArgument(r *bufio.Reader) ([]byte, error) {

    line, err := r.ReadString('\n')
    if err != nil {
        return nil, malformed("$<argumentLength>", line)
    }
    var argSize int
    if _, err := fmt.Sscanf(line, "$%d\r", &argSize); err != nil {
        return nil, malformed("$<argumentSize>", line)
    }

    // I think int is safe here as the max length of request
    // should be less then max int value?
    data, err := ioutil.ReadAll(io.LimitReader(r, int64(argSize)))
    if err != nil {
        return nil, err
    }

    if len(data) != argSize {
        return nil, malformedLength(argSize, len(data))
    }

    // Now check for trailing CR
    if b, err := r.ReadByte(); err != nil || b != '\r' {
        return nil, malformedMissingCRLF()
    }

    // And LF
    if b, err := r.ReadByte(); err != nil || b != '\n' {
        return nil, malformedMissingCRLF()
    }

    return data, nil
}

ioutil.ReadAll will allocate at least 512bytes memery even if we don't need it, when prase a large request (for instance, MGET key1 key2 ... keyn), the memory costs will increase fast ( n * 512bytes).

Force gc will trigger every 2 min, and gc is also a goroutine. if memory increase very fast, even reach the phyiscal memory, gc will not invoke immediately.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant