Updated environment documentation in guides to account for env files.
This commit is contained in:
@ -23,25 +23,76 @@ simply to use the optional ``--env`` argument when running bonobo from the shell
|
|||||||
syntax ``VAR_NAME=VAR_VALUE``. Multiple environment variables can be passed by using multiple ``--env`` / ``-e`` flags
|
syntax ``VAR_NAME=VAR_VALUE``. Multiple environment variables can be passed by using multiple ``--env`` / ``-e`` flags
|
||||||
(i.e. ``bonobo run --env FIZZ=buzz ...`` and ``bonobo run --env FIZZ=buzz --env Foo=bar ...``). Additionally, in bash
|
(i.e. ``bonobo run --env FIZZ=buzz ...`` and ``bonobo run --env FIZZ=buzz --env Foo=bar ...``). Additionally, in bash
|
||||||
you can also set environment variables by listing those you wish to set before the `bonobo run` command with space
|
you can also set environment variables by listing those you wish to set before the `bonobo run` command with space
|
||||||
separating the key-value pairs (i.e. ``FIZZ=buzz bonobo run ...`` or ``FIZZ=buzz FOO=bar bonobo run ...``).
|
separating the key-value pairs (i.e. ``FIZZ=buzz bonobo run ...`` or ``FIZZ=buzz FOO=bar bonobo run ...``). Additionally,
|
||||||
|
bonobo is able to pull environment variables from local '.env' files rather than having to pass each key-value pair
|
||||||
|
individually at runtime. Importantly, a strict 'order of priority' is followed when setting environment variables so
|
||||||
|
it is advisable to read and understand the order listed below to prevent
|
||||||
|
|
||||||
|
|
||||||
|
The order of priority is from lower to higher with the higher "winning" if set:
|
||||||
|
|
||||||
|
1. default values
|
||||||
|
``os.getenv("VARNAME", default_value)``
|
||||||
|
The user/writer/creator of the graph is responsible for setting these.
|
||||||
|
|
||||||
|
2. ``--default-env-file`` values
|
||||||
|
Specify file to read default env values from. Each env var in the file is used if the var isn't already a corresponding value set at the system environment (system environment vars not overwritten).
|
||||||
|
|
||||||
|
3. ``--default-env`` values
|
||||||
|
Works like #2 but the default ``NAME=var`` are passed individually, with one ``key=value`` pair for each ``--default-env`` flag rather than gathered from a specified file.
|
||||||
|
|
||||||
|
4. system environment values
|
||||||
|
Env vars already set at the system level. It is worth noting that passed env vars via ``NAME=value bonobo run ...`` falls here in the order of priority.
|
||||||
|
|
||||||
|
5. ``--env-file`` values
|
||||||
|
Env vars specified here are set like those in #2 albeit that these values have priority over those set at the system level.
|
||||||
|
|
||||||
|
6. ``--env`` values
|
||||||
|
Env vars set using the ``--env`` / ``-e`` flag work like #3 but take priority over all other env vars.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Examples
|
||||||
|
::::::::
|
||||||
|
|
||||||
The Examples below demonstrate setting one or multiple variables using both of these methods:
|
The Examples below demonstrate setting one or multiple variables using both of these methods:
|
||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
# Using one environment variable via --env flag:
|
# Using one environment variable via --env and --defualt-env flags:
|
||||||
bonobo run csvsanitizer --env SECRET_TOKEN=secret123
|
bonobo run csvsanitizer --env SECRET_TOKEN=secret123
|
||||||
|
bonobo run csvsanitizer --defaul-env SECRET_TOKEN=secret123
|
||||||
|
|
||||||
# Using multiple environment variables via -e (env) flag:
|
# Using multiple environment variables via -e (env) and --default-env flags:
|
||||||
bonobo run csvsanitizer -e SRC_FILE=inventory.txt -e DST_FILE=inventory_processed.csv
|
bonobo run csvsanitizer -e SRC_FILE=inventory.txt -e DST_FILE=inventory_processed.csv
|
||||||
|
bonobo run csvsanitizer --default-env SRC_FILE=inventory.txt --default-env DST_FILE=inventory_processed.csv
|
||||||
|
|
||||||
# Using one environment variable inline (bash only):
|
# Using one environment variable inline (bash-like shells only):
|
||||||
SECRET_TOKEN=secret123 bonobo run csvsanitizer
|
SECRET_TOKEN=secret123 bonobo run csvsanitizer
|
||||||
|
|
||||||
# Using multiple environment variables inline (bash only):
|
# Using multiple environment variables inline (bash-like shells only):
|
||||||
SRC_FILE=inventory.txt DST_FILE=inventory_processed.csv bonobo run csvsanitizer
|
SRC_FILE=inventory.txt DST_FILE=inventory_processed.csv bonobo run csvsanitizer
|
||||||
|
|
||||||
*Though not-yet implemented, the bonobo roadmap includes implementing environment / .env files as well.*
|
# Using an env file for default env values:
|
||||||
|
bonobo run csvsanitizer --default-env-file .default_env
|
||||||
|
|
||||||
|
# Using an env file for env values:
|
||||||
|
bonobo run csvsanitizer --env-file '.env'
|
||||||
|
|
||||||
|
|
||||||
|
ENV File Structure
|
||||||
|
::::::::::::::::::
|
||||||
|
|
||||||
|
The file structure for env files is incredibly simple. The only text in the file
|
||||||
|
should be `NAME=value` pairs with one pair per line like the below.
|
||||||
|
|
||||||
|
.. code-block:: text
|
||||||
|
|
||||||
|
# .env
|
||||||
|
|
||||||
|
DB_USER='bonobo'
|
||||||
|
DB_PASS='cicero'
|
||||||
|
|
||||||
|
|
||||||
Accessing Environment Variables from within the Graph Context
|
Accessing Environment Variables from within the Graph Context
|
||||||
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
|
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
|
||||||
|
|||||||
Reference in New Issue
Block a user