Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Utility Function to Ensure Exact Number of Bytes Read from File #5929

Open
FergoTheGreat opened this issue Sep 10, 2024 · 2 comments
Open
Labels
feedback welcome We want community's feedback on this issue or PR

Comments

@FergoTheGreat
Copy link

Is your feature request related to a problem? Please describe.

Currently, Deno provides Deno.readFile(), which reads an entire file into a buffer, and Deno.FsFile.read(), which reads a portion of a file into a buffer. However, Deno.FsFile.read() may return before reading the full requested number of bytes. This creates a challenge when attempting to reliably read exactly a specified number of bytes, requiring users to write custom logic to ensure they handle partial reads correctly.

Describe the solution you'd like

I propose adding a utility function to the standard library that abstracts away the complexity of handling partial reads, ensuring that the desired number of bytes is read into a buffer. This function might behave similarly to the following code:

export async function read(file: Deno.FsFile, buffer: Uint8Array): Promise<Uint8Array> {
    let offset = 0
    while (offset < buffer.byteLength) {
        const bytesRead = await file.read(buffer.subarray(offset))
        if (bytesRead == null) break
        offset += bytesRead
    }
    return buffer.subarray(0, offset)
}

This function would ensure that the buffer is filled (if possible) or gracefully handle end-of-file conditions without requiring developers to write custom loops.

Describe alternatives you've considered

Alternatives include manually implementing this logic in every project, as illustrated above. However, this approach leads to code duplication and potential bugs across codebases.

@BlackAsLight
Copy link
Contributor

You can achieve the same goal by using a ReadableStreamBYOBReader and specifying a min amount. This way the entire buffer will be guaranteed to be filled, unless the stream ends.

const reader = (await Deno.open('path')).readable.getReader({ mode: 'byob' })

const { done, value } = await reader.read(new Uint8Array(100), { min: 100 })

// Remember to `reader.releaseLock()` or `reader.cancel()` when finished so the resource can be cleaned up.

@iuioiua iuioiua added the feedback welcome We want community's feedback on this issue or PR label Sep 10, 2024
@FergoTheGreat
Copy link
Author

FergoTheGreat commented Sep 11, 2024

Yes, but ReadableStreamBYOBReader doesn't support seeking so it's not relevant if you want to perform random access reads from a file without reading the whole thing into a buffer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feedback welcome We want community's feedback on this issue or PR
Projects
None yet
Development

No branches or pull requests

3 participants