Buckets Not Displayed if Not Owned by the AWS Account

[nicolasbriere1]

I am using AWS and an IAM role to connect to S3. I want to access buckets that are not owned by my AWS account. Although the IAM policy is correctly configured, the plugin has a mechanism that lists only the buckets accessible to the account:

  const bucketList = await s3Client.listBuckets({});

  const buckets =
    bucketList.Buckets?.map(b => b.Name || '')
      .filter(b => b)
      .filter(b => {
        const bucketsAllowed =
          allowedBuckets.find(a => a.platform === platform.endpointName)
            ?.buckets || [];

        // If no allowedBuckets defined for the platform, all its buckets are allowed by default
        if (bucketsAllowed.length === 0) {
          return true;
        }

        return bucketsAllowed.some(a => {
          // Add the start/end of regular expression, so no unexpected matches happen
          // Example: `test` should't match `test-one`, but `test.*` should.
          return b.match(`^${a}$`);
        });
      }) || [];

  return buckets;

The s3Client.listBuckets method only displays buckets owned by the account, and my IAM role used by Backstage is not the owner, although it has read access. I would like to create a pull request to support this use case, but I am unsure how to do so without disrupting the existing logic.

[heyLu]

I'm not sure I fully understand the use-case, but some info:

• s3Client.listBuckets is the method from the AWS SDK, so not something we control:

  backstage-plugin-s3/plugins/s3-viewer-backend/src/credentials-provider/utils.ts

  Line 19 in 37cfe9c

  const bucketList = await s3Client.listBuckets({});

• that function with listBuckets is called in

  backstage-plugin-s3/plugins/s3-viewer-backend/src/credentials-provider/IAMRoleCredentialsProvider.ts

  Line 47 in 37cfe9c

  const buckets = await fetchBucketsForPlatform(

  , so it might be possible to implement a custom CredentialsProvider for the use-case. it probably wouldn't need to be part of this plugin either, you could just add your own locally and use that.

I'm not sure if we list buckets elsewhere though, @ivangonzalezacuna should know more about that.

[nicolasbriere1]

@heyLu, yes, you're right. I can create a custom provider, but I believe this is a common use case in the AWS context (accessing buckets not owned by the current account). Perhaps we can make the IAM Role credentials provider work with this.

[ivangonzalezacuna]

Hi @nicolasbriere1. Since we don't really have a proper AWS setup to test the iam-role mode, it would be really helpful if you can help us here.

As @heyLu said, the listBuckets is called within the fetchBucketsForPlatform for each credentials provider. I believe you could try to create a custom credentials provider one based on the IAMRoleCredentialsProvider and make the adjustments to work for your use case.

If you get into that, I believe we could extend the configuration with the needed things to make it work for the use-case you describe here and also for the use-case that it currently covers.

The iam-role config can be found, and extended if needed, in this file.

Also, after our last changes we merged a few days ago, it's pretty easy to test the s3-viewer locally. You can customize the app-config.yaml as desired for your tests. And then you could make the changes to the IAMRoleCredentialsProvider directly.

Hope this info is enough for you. If not, let us know