From cbba413282acf8c871f17c2c16619ce0361e652d Mon Sep 17 00:00:00 2001 From: Fokko Driesprong Date: Tue, 28 Nov 2023 16:21:36 -0800 Subject: [PATCH] Python: Clarify docs on environment variables (#165) --- mkdocs/docs/configuration.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/mkdocs/docs/configuration.md b/mkdocs/docs/configuration.md index a56baff7b5..c74f1bea25 100644 --- a/mkdocs/docs/configuration.md +++ b/mkdocs/docs/configuration.md @@ -32,13 +32,19 @@ There are three ways to pass in configuration: - Through environment variables - By passing in credentials through the CLI or the Python API -The configuration file is recommended since that's the most transparent way. If you prefer environment configuration: +The configuration file is recommended since that's the easiest way to manage the credentials. + +Another option is through environment variables: ```sh export PYICEBERG_CATALOG__DEFAULT__URI=thrift://localhost:9083 +export PYICEBERG_CATALOG__DEFAULT__S3__ACCESS_KEY_ID=username +export PYICEBERG_CATALOG__DEFAULT__S3__SECRET_ACCESS_KEY=password ``` -The environment variable picked up by Iceberg starts with `PYICEBERG_` and then follows the yaml structure below, where a double underscore `__` represents a nested field. +The environment variable picked up by Iceberg starts with `PYICEBERG_` and then follows the yaml structure below, where a double underscore `__` represents a nested field, and the underscore `_` is converted into a dash `-`. + +For example, `PYICEBERG_CATALOG__DEFAULT__S3__ACCESS_KEY_ID`, sets `s3.access-key-id` on the `default` catalog. ## FileIO @@ -80,8 +86,6 @@ For the FileIO there are several configuration options available: ### Azure Data lake -### Azure Data lake - | Key | Example | Description | | ----------------------- | ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | adlfs.connection-string | AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqF...;BlobEndpoint=http://localhost/ | A [connection string](https://learn.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string). This could be used to use FileIO with any adlfs-compatible object storage service that has a different endpoint (like [azurite](https://github.com/azure/azurite)). |