Commit may fail when container write many small files (failed to create diff tar stream: failed to copy: archive/tar: write too long)

[abel-von]

Description

Currently we got an error occasionally in our product environment, when we commit a container to an image.

failed to create diff: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount260055902: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount4247854495: failed to write compressed diff: failed to create diff tar stream: failed to copy: archive/tar: write too long: unknown

It's more likely to occur when writing a large number of small files in a container. I think this maybe related to buffer io, that the file content is changed during tar write it. There is a very low probability that the problem recurs.

func (fw *regFileWriter) Write(b []byte) (n int, err error) {
	overwrite := int64(len(b)) > fw.nb
	if overwrite {
		b = b[:fw.nb]
	}
	if len(b) > 0 {
		n, err = fw.w.Write(b)
		fw.nb -= int64(n)
	}
	switch {
	case err != nil:
		return n, err
	case overwrite:
		return n, ErrWriteTooLong
	default:
		return n, nil
	}
}

Steps to reproduce the issue

1. start a container to write many small files
2. nerdctl commit this container
3. repeat it thousands of times.

Describe the results you received and expected

If this is related to the buffer io, I think maybe the only way to avoid this is to add a sleep after Pause so that kernel can flush all data into disk.

What version of nerdctl are you using?

nerdctl version 2.0.1-2-g1f812253

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

Host information

No response