Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

agent: retry mount on ENOMEM #788

Merged
merged 1 commit into from
May 23, 2024
Merged

Commits on May 22, 2024

  1. agent: retry mount on ENOMEM

    There is a race between firecracker-containerd replacing the stub drive
    with the actual drive and mounting this drive. When the disk is replaced
    the kernel will schedule asynchronous work in virtblk_config_changed. In
    the meantime firecracker-containerd can proceed and already send a mount
    command to the agent running in the guest. This mount operation will,
    however, fail because the guest kernel still sees the stub drive with
    only 512 bytes in size. The resulting error code is a ENOMEM in this
    case. This commit therefore adds this as an retryable error code to
    accommodate for this situation.
    
    The issue can be reproduced when an artificial msleep(1000) is added in
    virtblk_config_changed_work. This produced the following error:
    
      error="failed to get stub drive for task \"test\": failed to mount
      drive inside vm: failed to mount newly patched drive: rpc error: code
      = Unknown desc = non-retryable failure mounting drive from
      \"/dev/vdb\" to \"/container/test/rootfs\": cannot allocate memory"
    
    Signed-off-by: Maximilian Heyne <[email protected]>
    heynemax committed May 22, 2024
    Configuration menu
    Copy the full SHA
    99b03b3 View commit details
    Browse the repository at this point in the history