Add support for vectorized read with libc::readv

[shamb0]

Fixes #4048

Add POSIX readv() System Call Support to MIRI

This PR implements emulation support for libc::readv(), a POSIX vectored I/O operation that allows reading data into multiple buffers with a single system call. This enhancement addresses issue #4048.

[RalfJung]

I'll assume this should not be reviewed yet since it is marked as a draft. I just noticed this one small thing. ;)

I'd recommend to make a first PR with just readv. Smaller PRs are always easier to handle. Please write @rustbot ready and remove the "draft" status once you want a review.

[RalfJung]

Please don't use as casts. Why are you casting here anyway?

[shamb0]

Hi @RalfJung,

Thank you for pointing this out—you're absolutely right that the casting is unnecessary. I appreciate your feedback.

As you noted, the PR is still in draft status, and there are some cleanups I plan to address. I shared it at this stage primarily to seek input, particularly regarding the issue we previously discussed here: Debugging Scalar Size Mismatch Error in Rust Miri.

Thank you again for reviewing and sharing your thoughts! :)

[RalfJung]

As you noted, the PR is still in draft status, and there are some cleanups I plan to address. I shared it at this stage primarily to seek input, particularly regarding the issue we previously discussed here: Debugging Scalar Size Mismatch Error in Rust Miri.

Ah, that was not clear at all. Please note things like that in the PR description.

Anyway judging from Zulip it seems you made progress there. I'm going to wait until you ask for further input on this PR. :)

[shamb0]

Hi @RalfJung,

I've completed the initial implementation of readv() support and would appreciate your review. The changes focus on three main areas:

1. Core Implementation

   • Added readv() emulation in src/shims/unix/fd.rs
   • Designed to comply with POSIX vectored I/O specifications

2. Safety Enhancement

   • Introduced new deref_pointer_and_offset_vectored() function in src/helpers.rs
   • Implements specialized array bounds checking for vectored I/O operations

3. Comprehensive Testing

   • Created test suite in tests/pass-dep/libc/libc-fs.rs covering:
     • Large buffer operations with pattern verification (64KB total)
     • Partial reads and EOF handling
     • Error condition validation
   • All tests pass with MIRIFLAGS=-Zmiri-disable-isolation

Looking forward to your feedback, particularly on the new array bounds checking approach and test coverage strategy. :)

[shamb0]

@rustbot ready

[RalfJung]

This is a very specific operation for a particular Linux/POSIX API. I don't think we need a global helper for this, so please move it into the only file where it is actually used.

It's also completely unclear to me why you need a complicated new check here. readv just iterates the array and does a read call for each element, right?

[shamb0]

• Thank you for your review comment regarding the placement and necessity of the deref_pointer_and_offset_vectored function. I'd like to explain the reasoning behind this implementation and agree with your suggestion about its placement.

• The implementation of readv() requires handling a base pointer to an array of iovec structures (iov_ptr: &OpTy<'tcx>) along with a count (iovcnt: i32). When reconstructing array elements from this base pointer, I encountered limitations with the existing deref_pointer_and_offset() API, specifically triggering assertions that weren't suitable for our array-based access pattern.

• You can find the complete MIRI backtrace demonstrating these limitations here: https://gist.github.com/shamb0/01491a8367c78404769b3e210ec860e0

• While the boundary checking logic may appear complex, it's necessary for safely handling array reconstruction in this context. However, I agree that this functionality is specific to the readv implementation and doesn't need to be a global helper. I've moved the function to src/shims/unix/fd.rs where it's being used, making its purpose and context clearer.

[RalfJung]

I would suggest you just call deref_pointer_as on the iov_ptr. The key is to construct the right pointee type: make it an array of libc::iovec pointee type, with length iovcnt. You do this once before the loop.

Then inside the loop, you can use project_index to access the n-th element of the array.

[RalfJung]

This seems to be always None, so please remove the argument.

[shamb0]

• I’ve removed the offset parameter from the readv function signature as requested.

• Initially, I had included it to align with the parameter structure of the existing read and write functions, given their similarity to file operations. Let me know if there’s anything else you’d like adjusted!"

[RalfJung]

offset will be required in the future for preadv, but let's go one step at a time :)

[RalfJung]

Is this justified by the docs? Also, what does the "or invalid" mean here?

[shamb0]

• I've updated the documentation to clarify the behavior of readv when iovcnt is 0. The updated documentation now explicitly covers:

  1. The function's return value semantics based on the POSIX specification
  2. The meaning of various error conditions and their corresponding return codes
  3. Special cases like empty buffer lists (iovcnt = 0)

[RalfJung]

I wasn't asking about your documentation, I was asking about the official documentation for readv. Often in cases like this we quite a sentence from the docs, and link to the docs, to explain why particular corner cases are handled the way they are.

[RalfJung]

Please remove the debug tracing from the PR for review.

[shamb0]

Done

[RalfJung]

I don't understand why you are allocating new storage here. What I would expect is that you call read once for each element of the vector, and directly put the results into the user-provided buffer. Yes that means we are using more than one syscall, but we don't really care about such details here -- the only thing that matters is the end-to-end behavior.

[RalfJung]

Ah it seems the docs say that the entire read must be atomic. Please fix the comment to explain why you are allocating a buffer here; currently you say something about error conditions but it's not clear what this means and it misses the main point.

[shamb0]

I've cleaned up the code and removed the problematic logic that attempted to handle reads individually.

The new implementation will focus on ensuring atomicity by performing a single read operation into a contiguous buffer, as required by POSIX readv() semantics.

[RalfJung]

You can't use a loop of reads, the read must be atomic. I thought that's why you are allocating a new buffer.

[shamb0]

I've cleaned up the code and removed the problematic logic that attempted to handle reads individually.

The new implementation will focus on ensuring atomicity by performing a single read operation into a contiguous buffer, as required by POSIX readv() semantics.

[RalfJung]

This unfortunately does not work if read blocks. The code after read will run too early, before the read actually completed.

Implementing this properly for blocking read is very complicated and will require some non-trivial refactoring of the file description trait.

[shamb0]

Thank you for highlighting the critical issue regarding blocking reads and concurrent access. This was an oversight in my initial implementation that could have led to race conditions.

I've refined the implementation, with a focus on atomic operations. The key improvement is the introduction of a new read_buffer interface in the FileDescription trait:

impl FileDescription for FileHandle {
    fn read_buffer<'tcx>(
        &self,
        self_ref: &FileDescriptionRef,
        communicate_allowed: bool,
        buf: &mut [u8],
        dest: &MPlaceTy<'tcx>,
        ecx: &mut MiriInterpCx<'tcx>,
    ) -> InterpResult<'tcx> {
        // Implementation details
    }
}

The revised strategy for readv() now follows these steps:

1. Allocates an intermediate byte buffer for the entire read operation
2. Performs a single atomic read operation into this buffer
3. Distributes the data from the intermediate buffer to the user's scattered iovec buffers

This approach ensures thread safety while maintaining POSIX compliance for atomic vectored I/O operations. I would appreciate your thoughts on this implementation and welcome any suggestions for further improvements.

[RalfJung]

This is in fact pretty much exactly how FileDescription::read used to look, but we changed it to support blocking reads. Adding it back will require all FDs to implement read twice, which we should avoid.

The problem with read is that when it returns, the read may not actually have completed yet. The fix for that is known as "continuation passing style": we need to equip read with a callback that it will invoke when the read was actually completed. That way, all the logic for taking the data from the big buffer and distributing it over the vectored buffers can be done at the right time, and we don't have to duplicate so much of the logic.

For this we should do a bit of preparation, in a separate PR: we currently have the UnblockCallback type for callbacks to be invoked on unblocking; we will have to generalize this to a general MachineCallback so that we can also use it in read. UnblockCallback has two methods, unblock and timeout, but in general we want only one method. The MachineCallback type should be generic over the argument type of that method. Thread unblocking can then use an argument type that indicates whether this is a successful unblock or a timeout.

So I would suggest a series of 3 PRs:

1. Generalize UnblockCallback to MachineCallback (will require fixes everywhere UnblockCallback is currently used, and adjustments to the callback! macro)
2. Equip FileDescription::read with a callback that is invoked after completion of the read.
3. Add readv, using that callback.

[RalfJung]

You have CI failures. Please make sure CI is green before asking for review; that avoids reviewers having to spend time on things that CI can check automatically. If you don't understand why CI fails, ask for help.

[shamb0]

You have CI failures. Please make sure CI is green before asking for review; that avoids reviewers having to spend time on things that CI can check automatically. If you don't understand why CI fails, ask for help.

You're absolutely correct about the importance of ensuring all CI checks pass before requesting a review. Moving forward, I will:

• Wait for CI builds to successfully complete across all supported HOST_TARGET configurations.
• Use rustbot ready only after confirming a green CI status.

[RalfJung]

That CI failure indicates that you need to rebase your branch over the latest Miri master branch, and then run ./miri toolchain. After that, a ./miri check should reproduce the issue on your machine so that you can fix it.

[shamb0]

Hi @RalfJung,

Thank you for the detailed design proposal outlining the three-phase implementation approach. Converting UnblockCallback to a more generic MachineCallback and improving read operation handling through continuation passing style is a robust solution.

While the current PR has addressed previous review comments with passing CI builds, your new design direction offers a better architectural path. I'd appreciate your guidance on whether to:

1. Start fresh with a new PR series implementing the three-phase approach, or
2. Evolve this PR to incorporate the suggested architectural changes

Looking forward to your direction on the preferred approach.

[shamb0]

@rustbot ready

[RalfJung]

I would suggest you convert this PR to a draft, and start working on the first and second PR outlined in my plan. Once they both landed, this PR can serve as the basis for step 3.

[shamb0]

I would suggest you convert this PR to a draft, and start working on the first and second PR outlined in my plan. Once they both landed, this PR can serve as the basis for step 3.

Sounds Good, Thankyou :)