Updated environment documentation in guides to account for env files.

This commit is contained in:
cwandrews
2017-10-15 16:43:28 -04:00
parent cb7a18f20f
commit d6d063ad43

View File

@ -23,25 +23,76 @@ simply to use the optional ``--env`` argument when running bonobo from the shell
syntax ``VAR_NAME=VAR_VALUE``. Multiple environment variables can be passed by using multiple ``--env`` / ``-e`` flags syntax ``VAR_NAME=VAR_VALUE``. Multiple environment variables can be passed by using multiple ``--env`` / ``-e`` flags
(i.e. ``bonobo run --env FIZZ=buzz ...`` and ``bonobo run --env FIZZ=buzz --env Foo=bar ...``). Additionally, in bash (i.e. ``bonobo run --env FIZZ=buzz ...`` and ``bonobo run --env FIZZ=buzz --env Foo=bar ...``). Additionally, in bash
you can also set environment variables by listing those you wish to set before the `bonobo run` command with space you can also set environment variables by listing those you wish to set before the `bonobo run` command with space
separating the key-value pairs (i.e. ``FIZZ=buzz bonobo run ...`` or ``FIZZ=buzz FOO=bar bonobo run ...``). separating the key-value pairs (i.e. ``FIZZ=buzz bonobo run ...`` or ``FIZZ=buzz FOO=bar bonobo run ...``). Additionally,
bonobo is able to pull environment variables from local '.env' files rather than having to pass each key-value pair
individually at runtime. Importantly, a strict 'order of priority' is followed when setting environment variables so
it is advisable to read and understand the order listed below to prevent
The order of priority is from lower to higher with the higher "winning" if set:
1. default values
``os.getenv("VARNAME", default_value)``
The user/writer/creator of the graph is responsible for setting these.
2. ``--default-env-file`` values
Specify file to read default env values from. Each env var in the file is used if the var isn't already a corresponding value set at the system environment (system environment vars not overwritten).
3. ``--default-env`` values
Works like #2 but the default ``NAME=var`` are passed individually, with one ``key=value`` pair for each ``--default-env`` flag rather than gathered from a specified file.
4. system environment values
Env vars already set at the system level. It is worth noting that passed env vars via ``NAME=value bonobo run ...`` falls here in the order of priority.
5. ``--env-file`` values
Env vars specified here are set like those in #2 albeit that these values have priority over those set at the system level.
6. ``--env`` values
Env vars set using the ``--env`` / ``-e`` flag work like #3 but take priority over all other env vars.
Examples
::::::::
The Examples below demonstrate setting one or multiple variables using both of these methods: The Examples below demonstrate setting one or multiple variables using both of these methods:
.. code-block:: bash .. code-block:: bash
# Using one environment variable via --env flag: # Using one environment variable via --env and --defualt-env flags:
bonobo run csvsanitizer --env SECRET_TOKEN=secret123 bonobo run csvsanitizer --env SECRET_TOKEN=secret123
bonobo run csvsanitizer --defaul-env SECRET_TOKEN=secret123
# Using multiple environment variables via -e (env) flag: # Using multiple environment variables via -e (env) and --default-env flags:
bonobo run csvsanitizer -e SRC_FILE=inventory.txt -e DST_FILE=inventory_processed.csv bonobo run csvsanitizer -e SRC_FILE=inventory.txt -e DST_FILE=inventory_processed.csv
bonobo run csvsanitizer --default-env SRC_FILE=inventory.txt --default-env DST_FILE=inventory_processed.csv
# Using one environment variable inline (bash only):
# Using one environment variable inline (bash-like shells only):
SECRET_TOKEN=secret123 bonobo run csvsanitizer SECRET_TOKEN=secret123 bonobo run csvsanitizer
# Using multiple environment variables inline (bash only): # Using multiple environment variables inline (bash-like shells only):
SRC_FILE=inventory.txt DST_FILE=inventory_processed.csv bonobo run csvsanitizer SRC_FILE=inventory.txt DST_FILE=inventory_processed.csv bonobo run csvsanitizer
*Though not-yet implemented, the bonobo roadmap includes implementing environment / .env files as well.* # Using an env file for default env values:
bonobo run csvsanitizer --default-env-file .default_env
# Using an env file for env values:
bonobo run csvsanitizer --env-file '.env'
ENV File Structure
::::::::::::::::::
The file structure for env files is incredibly simple. The only text in the file
should be `NAME=value` pairs with one pair per line like the below.
.. code-block:: text
# .env
DB_USER='bonobo'
DB_PASS='cicero'
Accessing Environment Variables from within the Graph Context Accessing Environment Variables from within the Graph Context
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::