-
Notifications
You must be signed in to change notification settings - Fork 207
Blobfuse Configuring and Running
blobfuse uses local file system as a buffer cache. On Azure, use the ephemeral drive (usually SSD) or ramdisk to get the most performance out of the buffer cache. Note that blobfuse will need to have write access to the directory you choose as a buffer cache.
Sample mount script provided in this repository uses /mnt/blobfusetmp as the cache location. Create this directory and change owner to the user that mounts blobfuse.
sudo mkdir /mnt/blobfusetmp
sudo chown <youruser> /mnt/blobfusetmp
Note for RHEL on Azure: Use /mnt/resource/blobfusetmp as the ephemeral drive is mounted on /mnt/resource in Azure. On Ubuntu, this is mounted on /mnt.
Optionally, you could use ramdisk. Create a ramdisk sized enough for your workload:
sudo mount -t tmpfs -o size=16g tmpfs /mnt/blobfusetmp
-
Option 1 - Store credentials in a file You will need a configuration file containing the Azure Storage connection information (account name, account key or SAS, blob endpoint and container name.) The owner of this file should be the user mounting blobfuse. The container must already exist (if not, you can create it through the Azure portal.) A sample configuration file is at https://github.com/Azure/azure-storage-fuse/blob/master/connection.cfg
-
Option 2 - Store credentials in environment variables You can store your account credentials in the environment variables. Use AZURE_STORAGE_ACCOUNT for the account name, AZURE_STORAGE_ACCESS_KEY for the account key, and AZURE_STORAGE_SAS_TOKEN for the SAS token. Only provide one of AZURE_STORAGE_ACCESS_KEY or AZURE_STORAGE_SAS_TOKEN.Check the ReadMe.txt file for the list of other environment variables.
export AZURE_STORAGE_ACCOUNT=myaccountname
export AZURE_STORAGE_ACCESS_KEY=myaccesskey
or
export AZURE_STORAGE_SAS_TOKEN=?tokenhere
NOTE Use absolute paths when providing directory paths in the commands below. Relative paths, and shortcuts do not work.
Sample command line for Option 1:
blobfuse /path/to/desired/mountpoint --tmp-path=/mnt/blobfusetmp -o attr_timeout=240 -o entry_timeout=240 -o negative_timeout=120 --config-file=../connection.cfg --log-level=LOG_DEBUG --file-cache-timeout-in-seconds=120
Sample command line for Option 2:
blobfuse /path/to/desired/mountpoint --tmp-path=/mnt/blobfusetmp -o attr_timeout=240 -o entry_timeout=240 -o negative_timeout=120 --container-name=mycontainer --log-level=LOG_DEBUG --file-cache-timeout-in-seconds=120
-
/path/to/desired/mountpointis the directory where you want the container to be mounted -
--tmp-path=/mnt/blobfusetmpis the cache directory, created above. -
--config-file=connection.cfgis the configuration file, created above. - attr_timeout, entry_timeout, and negative_timeout are standard FUSE configuration options. We recommend these values as defaults.
- Container provided in the option
--container-namemust exist prior to mounting. You can create the container from the Azure Portal
There is a sample mount script, to help simplify this:
https://github.com/Azure/azure-storage-fuse/blob/master/mount.sh
To run:
./mount.sh </path/to/desired/mountpoint>
Blobfuse allows multiple mounts to support additional containers. Additional containers can be mounted using the command above but the mount directory and --tmp-path should be unique and not conflict with any of the earlier mount points or temp directories. NOTE config files are recommended for multiple mounts as environment variables for container names and storage accounts are shared among all blobfuse mounts.
Use -o allow_other or -o allow_root flags while mounting in the blobfuse command to export the file system not just to the current user, but to others, or root user.
- Make sure the fuse package is installed (e.g. yum install fuse)
- Update connection.cfg file with your storage account information.
- Edit /etc/fstab with the blobfuse script.
Add the following line to use mount.sh:
/<path_to_blobfuse_mount.sh_file>/mount.sh </path/to/desired/mountpoint> fuse _netdev
OR
Add the following line to run without mount.sh
blobfuse /home/azureuser/mntblobfuse fuse defaults,_netdev,--tmp-path=/home/azureuser/tmppath,--config-file=/home/azureuser/connection.cfg,--log-level=LOG_DEBUG,allow_other 0 0
The standard way to unmount a FUSE adapter is to use 'fusermount':
fusermount -u </path/to/desired/mountpoint>
Blobfuse now supports Sovereign Clouds such as USGovernment (Fairfax), Azure China (Mooncake) and Azure Germany (Black Forest) via a new configuration option: "blobEndpoint". To access a sovereign cloud, you must supply the blobEndpoint parameter with the full blob endpoint for your storage account.
Ex:
| Environment | blobEndpoint |
|---|---|
| Azure USGovernment | <youraccountname>.blob.core.usgovcloudapi.net |
| Azure China | <youraccountname>.blob.core.chinacloudapi.cn |
| Azure Germany | <youraccountname>.blob.core.cloudapi.de |
An example USGovernment configuration file is at: https://github.com/Azure/azure-storage-fuse/blob/master/connectionUSGov.cfg
Blobfuse mounts storage accounts that have private links just the same way as it mounts storage accounts with a public IP only. The Blobfuse config file should specify the accountName as the original Storage account name and not the privatelink storage account name. For Eg: myblobstorageaccount.blob.core.windows.net is correct while privatelink.myblobstorageaccount.blob.core.windows.net is wrong. The VNET or the on-prem machine hosting Blobfuse should be able to translate the original storage account name to the privatelink name and thereafter the private link IP. Behind the scenes, Blobfuse accesses the storage account with its private Ip. If for some reason the translation fails please confirm the VNet settings to ensure that it is forwarding DNS translation requests to Azure Provided DNS 168.63.129.16. In case the Blobfuse hosting VM is set up to forward to a Custom DNS Server, the Custom DNS settings should be verified, it should forward DNS requests to the Azure Provided DNS 168.63.129.16.