Buckets creation error

[NyakudyaA]

Problem

Sometime we get the following error message

Traceback (most recent call last):

  File "/scripts/create_default_buckets.py", line 42, in <module>

    main()

  File "/scripts/create_default_buckets.py", line 36, in main

    s3.create_bucket(Bucket=i)

  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 514, in _api_call

    return self._make_api_call(operation_name, kwargs)

  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 938, in _make_api_call

    raise error_class(parsed_response, operation_name)

botocore.exceptions.ClientError: An error occurred (InternalError) when calling the CreateBucket operation (reached max retries: 4): We encountered an internal error, please try again.

This doesn't really tell whether it's a permission error or it could not reach the s3 endpoint. It would be nice to have a friendly message here

@zacharlie Any hints

[zacharlie]

Could it be related to an issue in the default docker-compose-s3.yml?

I think

    volumes:
      - ./minio_data:/mapproxy

on the minio container should be

    volumes:
      - ./minio_data:/data

which could be causing the S3 endpoint permissions to fail.

Otherwise I think we need to figure out how to reproduce the issue. If it's waiting for the S3 endpoint to spin up on first run we could use a try/ except block to catch the ClientError and add a sleep interval before retrying again.