Writing output into AWS S3 does not work

[xuanpan]

I understand dbeam officially does not support S3. I kinda of hopping it might work out of the box because Beam supports S3 as its file io. However I got the following error:

"No filesystem found for scheme s3"

Do you have a clue why is this happening? Do you plan to add S3 output in the future?

[rulle-io]

@pan-xuan Can you provide your parameters and logs error (may be abfuscated :) )?

[xijingchen]

Hi.
Below are parameters

--connectionUrl=jdbc:mysql://35.228.186.7:3306/keychaindb
--username=xxxxx
--password=xxxxx
--table=keychain_dump
--output=s3://com.privacyone.bigdata/2020-05-28/18

There is only the error from caught Exception.getMessage()
"No filesystem found for scheme s3"

[rulle-io]

Reproducible. I will have a look.

[xuanpan]

That would be awesome Ruslan. We have tried to configure dbeam during startup with AWS option and S3 option. None of the attempts helped.

[rulle-io]

Interesting, I have given it a try, it will not be a one-liner :) . Will continue this week.

[xuanpan]

I had the feeling it would be a "one liner" because Beam essentially support aws s3. I had assumed it is how dbeam initiated beam lib to caused the issue. But after a few attempts in patching the factory with S3 and AWS options with no result. I became clueless. We appreciate your continuous investigation.