-
Notifications
You must be signed in to change notification settings - Fork 513
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
119 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,119 @@ | ||
- Proposal Name: `write_returns_metadata` | ||
- Start Date: 2025-01-16 | ||
- RFC PR: [apache/opendal#0000](https://github.com/apache/opendal/pull/0000) | ||
- Tracking Issue: [apache/opendal#0000](https://github.com/apache/opendal/issues/0000) | ||
|
||
# Summary | ||
|
||
Enhance write operations by returning metadata after successful writes. | ||
|
||
# Motivation | ||
|
||
Currently, write operations (`write`, `write_with`, `writer`, `writer_with`) only return `Result<()>` or `Result<Writer>`. | ||
Users who need metadata after writing (like `ETag` or `version_id`) must make an additional `stat()` call. This is inefficient | ||
and can lead to race conditions if the file is modified between the write and stat operations. | ||
|
||
Many storage services (like S3, GCS, Azure Blob) return metadata in their write responses. We should expose this information | ||
to users directly after write operations. | ||
|
||
# Guide-level explanation | ||
|
||
The write operations will be enhanced to return metadata: | ||
|
||
```rust | ||
// Before | ||
op.write("path/to/file", data).await?; | ||
let meta = op.stat("path/to/file").await?; | ||
if Some(etag) = meta.etag() { | ||
println!("File ETag: {}", etag); | ||
} | ||
|
||
// After | ||
let meta = op.write("path/to/file", data).await?; | ||
if Some(etag) = meta.etag() { | ||
println!("File ETag: {}", etag); | ||
} | ||
``` | ||
|
||
For writer operations: | ||
|
||
```rust | ||
// Before | ||
let mut writer = op.writer("path/to/file").await?; | ||
writer.write(data).await?; | ||
writer.close().await?; | ||
let meta = op.stat("path/to/file").await?; | ||
if Some(etag) = meta.etag() { | ||
println!("File ETag: {}", etag); | ||
} | ||
|
||
// After | ||
let mut writer = op.writer("path/to/file").await?; | ||
writer.write(data).await?; | ||
let meta = writer.close().await?; | ||
if Some(etag) = meta.etag() { | ||
println!("File ETag: {}", etag); | ||
} | ||
``` | ||
|
||
The behavior remains unchanged if users don't need the metadata - they can simply ignore the return value. | ||
|
||
# Reference-level explanation | ||
|
||
## Changes to `Operator` API | ||
|
||
The following functions will be modified to return `Result<Metadata>` instead of `Result<()>`: | ||
|
||
- `write()` | ||
- `write_with()` | ||
|
||
The `writer()` and `writer_with()` return types remain unchanged as they return `Result<Writer>`. | ||
|
||
## Changes to struct `types::write::writer::Writer` | ||
|
||
The `Writer` struct will be modified to return `Result<Metadata>` instead of `Result<()>` for the `close()` function. | ||
|
||
## Changes to trait `oio::Write` and trait `oio::MultipartWrite` | ||
|
||
The `Write` trait will be modified to return `Result<Metadata>` instead of `Result<()>` for the `close()` function. | ||
|
||
The `MultipartWrite` trait will be modified to return `Result<Metadata>` instead of `Result<()>` for the `complete_part()` | ||
and `write_once` function. | ||
|
||
## Implementation Details | ||
|
||
For services that return metadata in their write responses: | ||
- The metadata will be captured from the service response | ||
- All available fields (etag, version_id, etc.) will be populated | ||
|
||
For services that don't return metadata in write responses: | ||
- A default metadata object will be returned | ||
|
||
|
||
# Drawbacks | ||
|
||
- Minor breaking change for users who explicitly type the return value of write operations | ||
- Additional complexity in the Writer implementation | ||
|
||
# Rationale and alternatives | ||
|
||
- Provides a clean, consistent API | ||
- Maintains backward compatibility for users who ignore the return value | ||
- Improves performance by avoiding additional stat calls when possible | ||
|
||
# Prior art | ||
|
||
Similar patterns exist in other storage SDKs: | ||
|
||
- `object_store` crate returns metadata in `PutResult` after calling `put_opts` | ||
- AWS SDK returns metadata in `PutObjectOutput` | ||
- Azure SDK returns `UploadFileResponse` after uploads | ||
|
||
# Unresolved questions | ||
|
||
- None | ||
|
||
|
||
# Future possibilities | ||
|
||
- None |