Support for custom object metadata

[damooo]

Many cloud backends supports attaching custom metadata as key-value pairs to an object. That is a great way to attach some metadata to an object in concurrency-safe transactional way without much fuss at application layer. It would be great if opendal propagates that support.

See also:

• s3: Using object metadata
• gcs: Object metadata
• etc.

These all services have different custom header coonventions for custom kv metadata. We can normalize at opendal args layer, and propagate as per convention.

[damooo]

Most modern filesystems like ext2, ext3, ext4, NTFS, zfs, etc support extended file attributes. Rust has popular xattr to handle them in os/fs agnostic way.

[damooo]

And also hdfs: https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/ExtendedAttributes.html

[Xuanwo]

Great idea! This is exactly what I would like to incorporate into OpenDAL.

[jokester]

I'm not sure this can be possible for all services, but even partial support would help much.

An important use of S3 or GCS is to host static files for web, it just feels incomplete when we can't set a contentType for a new file.

[Xuanwo]

it just feels incomplete when we can't set a contentType for a new file.

This feature has already been supported. You can use the following code to set content type for a file:

let bs = b"hello, world!".to_vec();
let _ = op
    .write_with("path/to/file", bs)
    .content_type("text/plain")
    .await?;

And this issue is mainly about custom object metadata, for example, x-amz-meta-abc for s3.

[jokester]

This feature has already been supported

Thanks & sorry I didn't know that. I only checked python bindings.

I will look deeper first, starting from whether that's available in python 👍🏽

[Xuanwo]

Tracked at #4842, closing this feature request now.