Support for direct access to S3 resources

[sahuguet]

Assuming this is not implemented yet (I could not find it in the doc), I would like to reference s3 resources directly without having to download them locally.

A good example is https://duckdb.org/docs/extensions/httpfs.html where S3 resources can be referenced seamlessly from the query.

[luigi-asprino]

Hi,

You're right. It's not implemented yet, but it would be very nice to have it!
I'm not familiar with S3 resources, but I assume there should be a way to open an input stream from them.
There seems to be a Java library for this https://github.com/aws/aws-sdk-java/tree/master

[sahuguet]

The kind of logic to implement to access S3 over http can be found at https://github.com/duckdb/duckdb/blob/main/extension/httpfs/s3fs.cpp#L19 . But it is probably cleaner and simpler to use a Java library that does the work.