From d6d063ad43e499a69dcd94630c26f2cc54f02368 Mon Sep 17 00:00:00 2001 From: cwandrews Date: Sun, 15 Oct 2017 16:43:28 -0400 Subject: [PATCH] Updated environment documentation in guides to account for env files. --- docs/guide/environment.rst | 67 +++++++++++++++++++++++++++++++++----- 1 file changed, 59 insertions(+), 8 deletions(-) diff --git a/docs/guide/environment.rst b/docs/guide/environment.rst index 203368d..ba21e62 100644 --- a/docs/guide/environment.rst +++ b/docs/guide/environment.rst @@ -23,25 +23,76 @@ simply to use the optional ``--env`` argument when running bonobo from the shell syntax ``VAR_NAME=VAR_VALUE``. Multiple environment variables can be passed by using multiple ``--env`` / ``-e`` flags (i.e. ``bonobo run --env FIZZ=buzz ...`` and ``bonobo run --env FIZZ=buzz --env Foo=bar ...``). Additionally, in bash you can also set environment variables by listing those you wish to set before the `bonobo run` command with space -separating the key-value pairs (i.e. ``FIZZ=buzz bonobo run ...`` or ``FIZZ=buzz FOO=bar bonobo run ...``). +separating the key-value pairs (i.e. ``FIZZ=buzz bonobo run ...`` or ``FIZZ=buzz FOO=bar bonobo run ...``). Additionally, +bonobo is able to pull environment variables from local '.env' files rather than having to pass each key-value pair +individually at runtime. Importantly, a strict 'order of priority' is followed when setting environment variables so +it is advisable to read and understand the order listed below to prevent + + +The order of priority is from lower to higher with the higher "winning" if set: + + 1. default values + ``os.getenv("VARNAME", default_value)`` + The user/writer/creator of the graph is responsible for setting these. + + 2. ``--default-env-file`` values + Specify file to read default env values from. Each env var in the file is used if the var isn't already a corresponding value set at the system environment (system environment vars not overwritten). + + 3. ``--default-env`` values + Works like #2 but the default ``NAME=var`` are passed individually, with one ``key=value`` pair for each ``--default-env`` flag rather than gathered from a specified file. + + 4. system environment values + Env vars already set at the system level. It is worth noting that passed env vars via ``NAME=value bonobo run ...`` falls here in the order of priority. + + 5. ``--env-file`` values + Env vars specified here are set like those in #2 albeit that these values have priority over those set at the system level. + + 6. ``--env`` values + Env vars set using the ``--env`` / ``-e`` flag work like #3 but take priority over all other env vars. + + + +Examples +:::::::: The Examples below demonstrate setting one or multiple variables using both of these methods: .. code-block:: bash - # Using one environment variable via --env flag: + # Using one environment variable via --env and --defualt-env flags: bonobo run csvsanitizer --env SECRET_TOKEN=secret123 + bonobo run csvsanitizer --defaul-env SECRET_TOKEN=secret123 - # Using multiple environment variables via -e (env) flag: + # Using multiple environment variables via -e (env) and --default-env flags: bonobo run csvsanitizer -e SRC_FILE=inventory.txt -e DST_FILE=inventory_processed.csv - - # Using one environment variable inline (bash only): + bonobo run csvsanitizer --default-env SRC_FILE=inventory.txt --default-env DST_FILE=inventory_processed.csv + + # Using one environment variable inline (bash-like shells only): SECRET_TOKEN=secret123 bonobo run csvsanitizer - # Using multiple environment variables inline (bash only): + # Using multiple environment variables inline (bash-like shells only): SRC_FILE=inventory.txt DST_FILE=inventory_processed.csv bonobo run csvsanitizer - -*Though not-yet implemented, the bonobo roadmap includes implementing environment / .env files as well.* + + # Using an env file for default env values: + bonobo run csvsanitizer --default-env-file .default_env + + # Using an env file for env values: + bonobo run csvsanitizer --env-file '.env' + + +ENV File Structure +:::::::::::::::::: + +The file structure for env files is incredibly simple. The only text in the file +should be `NAME=value` pairs with one pair per line like the below. + +.. code-block:: text + + # .env + + DB_USER='bonobo' + DB_PASS='cicero' + Accessing Environment Variables from within the Graph Context :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::