[PATCH s3storage] [storages] custom s3 endpoint URL

Nicola Spanti nicola.spanti at logilab.fr
Tue Jan 28 15:50:42 CET 2020


Published. Thanks.

Le 23/01/2020 à 14:54, Arthur Lutz a écrit :
> # HG changeset patch
> # User Arthur Lutz <arthur.lutz at logilab.fr>
> # Date 1567092519 -7200
> #      Thu Aug 29 17:28:39 2019 +0200
> # Node ID dd3da601de4b5a1876a55dc8c5f049a004db29ff
> # Parent  e89aad4c19c2b3c39c8da4ad578b2c2dc23f0871
> # EXP-Topic custom-s3-endpoint
> [storages] custom s3 endpoint URL
> 
> diff --git a/README.rst b/README.rst
> --- a/README.rst
> +++ b/README.rst
> @@ -50,13 +50,29 @@
>   
>   - AWS_SECRET_ACCESS_KEY: the secret key for your AWS account.
>   
> -
>   Please read `boto3 documentation`_ for more details.
>   
> +Further configuration
> +---------------------
>   
> +In some cases you might want to use compatible S3 storage which is not on
> +AWS.
> +
> +For this you have the following variables :
> +
> +- AWS_S3_ENDPOINT_URL: endpoint url for your alternative S3 storage
> +
> +This enables the use of a `minio server <https://min.io>`_, see
> +`their documentation about endpoints
> +<https://docs.min.io/docs/aws-cli-with-minio>`_.
> +
> +Note : this type of variable `has been requested upstream
> +<https://github.com/boto/boto3/issues/2099>`_
> +(and related `aws-cli <https://github.com/aws/aws-cli/issues/4454>`_).
>   
>   
>   .. _`boto3 documentation`: https://boto3.readthedocs.io/en/stable/guide/configuration.html#shared-credentials-file
>   .. _boto3: https://github.com/boto/boto3
>   .. _Cubicweb: https://www.cubicweb.org/project/cubicweb
>   .. _BytesFileSystemStorage: http://cubicweb.readthedocs.io/en/3.26.2/tutorials/advanced/part03_bfss
> +
> diff --git a/cubicweb_s3storage/storages.py b/cubicweb_s3storage/storages.py
> --- a/cubicweb_s3storage/storages.py
> +++ b/cubicweb_s3storage/storages.py
> @@ -22,6 +22,7 @@
>   from logging import getLogger
>   
>   from six import PY3
> +import os
>   import boto3
>   
>   from cubicweb import Binary, set_log_methods
> @@ -38,7 +39,11 @@
>   
>       @classmethod
>       def _s3_client(cls):
> -        return boto3.client('s3')
> +        endpoint_url = os.environ.get('AWS_S3_ENDPOINT_URL')
> +        if endpoint_url:
> +            cls.debug('Using custom S3 endpoint url {}'.format(endpoint_url))
> +        return boto3.client('s3',
> +                            endpoint_url=endpoint_url)
>   
>       def callback(self, source, cnx, value):
>           """see docstring for prototype, which vary according to is_source_callback
> 



More information about the cubicweb-devel mailing list