[ADD] fs_storage: Get access to filesystem storage

This addon define a new model  used to get access to an external filesystem storage (ftp, sftp, s3, azure, ...) through a unified interface provided by the  (https://filesystem-spec.readthedocs.io/en/latest) python library

fs_storage/README.rst

@@ -0,0 +1,222 @@
+==========================
+Filesystem Storage Backend
+==========================
+
+.. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+   !! This file is generated by oca-gen-addon-readme !!
+   !! changes will be overwritten.                   !!
+   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+
+.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
+    :target: https://odoo-community.org/page/development-status
+    :alt: Beta
+.. |badge2| image:: https://img.shields.io/badge/licence-LGPL--3-blue.png
+    :target: http://www.gnu.org/licenses/lgpl-3.0-standalone.html
+    :alt: License: LGPL-3
+.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fstorage-lightgray.png?logo=github
+    :target: https://github.com/OCA/storage/tree/16.0/fs_storage
+    :alt: OCA/storage
+.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
+    :target: https://translation.odoo-community.org/projects/storage-16-0/storage-16-0-fs_storage
+    :alt: Translate me on Weblate
+.. |badge5| image:: https://img.shields.io/badge/runbot-Try%20me-875A7B.png
+    :target: https://runbot.odoo-community.org/runbot/275/16.0
+    :alt: Try me on Runbot
+
+|badge1| |badge2| |badge3| |badge4| |badge5| 
+
+This addon is a technical addon that allows you to define filesystem like
+storage for your data. It's used by other addons to store their data in a
+transparent way into different kind of storages.
+
+Through the fs.storage record, you get access to an object that implements
+the `fsspec.spec.AbstractFileSystem <https://filesystem-spec.readthedocs.io/en/
+latest/api.html#fsspec.spec.AbstractFileSystem>`_ interface and therefore give
+you an unified interface to access your data whatever the storage protocol you
+decide to use.
+
+The list of supported protocols depends on the installed fsspec implementations.
+By default, the addon will install the following protocols:
+
+* LocalFileSystem
+* MemoryFileSystem
+* ZipFileSystem
+* TarFileSystem
+* FTPFileSystem
+* CachingFileSystem
+* WholeFileSystem
+* SimplCacheFileSystem
+* ReferenceFileSystem
+* GenericFileSystem
+* DirFileSystem
+* DatabricksFileSystem
+* GitHubFileSystem
+* JupiterFileSystem
+* OdooFileSystem
+
+The OdooFileSystem is the one that allows you to store your data into a directory
+mounted into your Odoo's storage directory. This is the default FS Storage
+when creating a new fs.storage record.
+
+Others protocols are available through the installation of additional
+python packages:
+
+* DropboxDriveFileSystem -> `pip install fsspec[dropbox]`
+* HTTPFileSystem -> `pip install fsspec[http]`
+* HTTPSFileSystem -> `pip install fsspec[http]`
+* GCSFileSystem -> `pip install fsspec[gcs]`
+* GSFileSystem -> `pip install fsspec[gs]`
+* GoogleDriveFileSystem -> `pip install gdrivefs`
+* SFTPFileSystem -> `pip install fsspec[sftp]`
+* HaddoopFileSystem -> `pip install fsspec[hdfs]`
+* S3FileSystem -> `pip install fsspec[s3]`
+* WandbFS -> `pip install wandbfs`
+* OCIFileSystem -> `pip install fsspec[oci]`
+* AsyncLocalFileSystem -> `pip install 'morefs[asynclocalfs]`
+* AzureDatalakeFileSystem -> `pip install fsspec[adl]`
+* AzureBlobFileSystem -> `pip install fsspec[abfs]`
+* DaskWorkerFileSystem -> `pip install fsspec[dask]`
+* GitFileSystem -> `pip install fsspec[git]`
+* SMBFileSystem -> `pip install fsspec[smb]`
+* LibArchiveFileSystem -> `pip install fsspec[libarchive]`
+* OSSFileSystem -> `pip install ossfs`
+* WebdavFileSystem -> `pip install webdav4`
+* DVCFileSystem -> `pip install dvc`
+* XRootDFileSystem -> `pip install fsspec-xrootd`
+
+This list of supported protocols is not exhaustive or could change in the future
+depending on the fsspec releases. You can find more information about the
+supported protocols on the `fsspec documentation
+<https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem>`_.
+
+**Table of contents**
+
+.. contents::
+   :local:
+
+Usage
+=====
+
+Configuration
+~~~~~~~~~~~~~
+
+When you create a new backend, you must specify the following:
+
+* The name of the backend. This is the name that will be used to
+  identify the backend into Odoo
+* The code of the backend. This code will identify the backend into the store_fname
+  field of the ir.attachment model. This code must be unique. It will be used
+  as scheme. example of the store_fname field: ``odoofs://abs34Tg11``.
+* The protocol used by the backend. The protocol refers to the supported
+  protocols of the fsspec python package.
+* A directory path. This is a root directory from which the filesystem will
+  be mounted. This directory must exist.
+* The protocol options. These are the options that will be passed to the
+  fsspec python package when creating the filesystem. These options depend
+  on the protocol used and are described in the fsspec documentation.
+
+Some protocols defined in the fsspec package are wrappers around other
+protocols. For example, the SimpleCacheFileSystem protocol is a wrapper
+around any local filesystem protocol. In such cases, you must specify into the
+protocol options the protocol to be wrapped and the options to be passed to
+the wrapped protocol.
+
+For example, if you want to create a backend that uses the SimpleCacheFileSystem
+protocol, after selecting the SimpleCacheFileSystem protocol, you must specify
+the protocol options as follows:
+
+.. code-block:: python
+
+    {
+        "directory_path": "/tmp/my_backend",
+        "target_protocol": "odoofs",
+        "target_options": {...},
+    }
+
+In this example, the SimpleCacheFileSystem protocol will be used as a wrapper
+around the odoofs protocol.
+
+Server Environment
+~~~~~~~~~~~~~~~~~~
+
+To ease the management of the filesystem storages configuration accross the different
+environments, the configuration of the filesystem storages can be defined in
+environment files or directly in the main configuration file. For example, the
+configuration of a filesystem storage with the code `fsprod` can be provided in the
+main configuration file as follows:
+
+.. code-block:: ini
+
+  [fs_storage.fsprod]
+  protocol=s3
+  options={"endpoint_url": "https://my_s3_server/", "key": "KEY", "secret": "SECRET"}
+  directory_path=my_bucket
+
+To work, a `storage.backend` record must exist with the code `fsprod` into the database.
+In your configuration section, you can specify the value for the following fields:
+
+* `protocol`
+* `options`
+* `directory_path`
+
+
+Known issues / Roadmap
+======================
+
+* Transactions: fsspec comes with a transactional mechanism that once started,
+  gathers all the files created during the transaction, and if the transaction
+  is committed, moves them to their final locations. It would be useful to
+  bridge this with the transactional mechanism of odoo. This would allow to
+  ensure that all the files created during a transaction are either all
+  moved to their final locations, or all deleted if the transaction is rolled
+  back. This mechanism is only valid for files created during the transaction
+  by a call to the `open` method of the file system. It is not valid for others
+  operations, such as `rm`, `mv_file`, ... .
+
+Bug Tracker
+===========
+
+Bugs are tracked on `GitHub Issues <https://github.com/OCA/storage/issues>`_.
+In case of trouble, please check there if your issue has already been reported.
+If you spotted it first, help us smashing it by providing a detailed and welcomed
+`feedback <https://github.com/OCA/storage/issues/new?body=module:%20fs_storage%0Aversion:%2016.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.
+
+Do not contact contributors directly about support or help with technical issues.
+
+Credits
+=======
+
+Authors
+~~~~~~~
+
+* Akretion
+* ACSONE SA/NV
+
+Contributors
+~~~~~~~~~~~~
+
+* Sébastien BEAU <[email protected]>
+* Raphaël Reverdy <[email protected]>
+* Florian da Costa <[email protected]>
+* Cédric Pigeon <[email protected]>
+* Renato Lima <[email protected]>
+* Benoît Guillot <[email protected]>
+* Laurent Mignon <[email protected]>
+* Denis Roussel <[email protected]>
+
+Maintainers
+~~~~~~~~~~~
+
+This module is maintained by the OCA.
+
+.. image:: https://odoo-community.org/logo.png
+   :alt: Odoo Community Association
+   :target: https://odoo-community.org
+
+OCA, or the Odoo Community Association, is a nonprofit organization whose
+mission is to support the collaborative development of Odoo features and
+promote its widespread use.
+
+This module is part of the `OCA/storage <https://github.com/OCA/storage/tree/16.0/fs_storage>`_ project on GitHub.
+
+You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

fs_storage/__init__.py

@@ -0,0 +1,6 @@
+# register protocols first
+from . import odoo_file_system
+from . import rooted_dir_file_system
+
+# then add normal imports
+from . import models

fs_storage/__manifest__.py

@@ -0,0 +1,22 @@
+# Copyright 2017 Akretion (http://www.akretion.com).
+# @author Sébastien BEAU <[email protected]>
+# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
+
+{
+    "name": "Filesystem Storage Backend",
+    "summary": "Implement the concept of Storage with amazon S3, sftp...",
+    "version": "16.0.1.0.1",
+    "category": "FS Storage",
+    "website": "https://github.com/OCA/storage",
+    "author": " Akretion, ACSONE SA/NV, Odoo Community Association (OCA)",
+    "license": "LGPL-3",
+    "development_status": "Beta",
+    "installable": True,
+    "depends": ["base", "base_sparse_field", "server_environment"],
+    "data": [
+        "views/fs_storage_view.xml",
+        "data/data.xml",
+        "security/ir.model.access.csv",
+    ],
+    "external_dependencies": {"python": ["fsspec"]},
+}

fs_storage/data/data.xml

@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="utf-8" ?>
+<odoo noupdate="1">
+    <record id="default_fs_storage" model="fs.storage">
+        <field name="name">Odoo Filesystem Backend</field>
+        <field name="protocol">odoofs</field>
+        <field name="code">odoofs</field>
+    </record>
+</odoo>