Merge remote-tracking branch 'upstream/develop' into develop
This commit is contained in:
@ -1,44 +1,48 @@
|
|||||||
Environmental Variables
|
Environment Variables
|
||||||
=======================
|
=======================
|
||||||
|
|
||||||
Best practice holds that variables should be passed to graphs via environmental variables.
|
Best practice holds that variables should be passed to graphs via environment variables.
|
||||||
Doing this is important for keeping sensitive data out of the code - such as an
|
Doing this is important for keeping sensitive data out of the code - such as an
|
||||||
API token or username and password used to access a database. Not only is this
|
API token or username and password used to access a database. Not only is this
|
||||||
approach more secure, it also makes graphs more flexible by allowing adjustments
|
approach more secure, it also makes graphs more flexible by allowing adjustments
|
||||||
for a variety of environments and contexts. Importantly, environmental variables
|
for a variety of environments and contexts. Importantly, environment variables
|
||||||
are also the means by-which arguments can be passed to graphs.
|
are also the means by-which arguments can be passed to graphs.
|
||||||
|
|
||||||
|
|
||||||
Passing / Setting Environmental Variables
|
Passing / Setting Environment Variables
|
||||||
::::::::::::::::::::::::::::::::::::::::::::
|
::::::::::::::::::::::::::::::::::::::::::::
|
||||||
|
|
||||||
The recommended way to set environmental variables for a given graph is simply to use
|
Setting environment variables for your graphs to use can be done in a variety of ways and which one used can vary
|
||||||
the optional ``--env`` argument when running bonobo from the shell (bash, command prompt, etc).
|
based-upon context. Perhaps the most immediate and simple way to set/override a variable for a given graph is
|
||||||
|
simply to use the optional ``--env`` argument when running bonobo from the shell (bash, command prompt, etc).
|
||||||
``--env`` (or ``-e`` for short) should then be followed by the variable name and value using the
|
``--env`` (or ``-e`` for short) should then be followed by the variable name and value using the
|
||||||
syntax `VAR_NAME=VAR_VALUE`. Multiple environmental variables can be passed by using
|
syntax ``VAR_NAME=VAR_VALUE``. Multiple environment variables can be passed by using multiple ``--env`` / ``-e`` flags (i.e. ``bonobo run --env FIZZ=buzz ...`` and ``bonobo run --env FIZZ=buzz --env Foo=bar ...``). Additionally, in bash you can also set environment variables by listing those you wish to set before the `bonobo run` command with space separating the key-value pairs (i.e. ``FIZZ=buzz bonobo run ...`` or ``FIZZ=buzz FOO=bar bonobo run ...``).
|
||||||
multiple ``--env`` / ``-e`` flags.
|
|
||||||
|
|
||||||
Example:
|
The Examples below demonstrate setting one or multiple variables using both of these methods:
|
||||||
|
|
||||||
.. code-block:: bash
|
.. code-block:: bash
|
||||||
|
|
||||||
# Using one environmental variable:
|
# Using one environment variable via --env flag:
|
||||||
bonobo run csvsanitizer --env SECRET_TOKEN=secret123
|
bonobo run csvsanitizer --env SECRET_TOKEN=secret123
|
||||||
|
|
||||||
# Using multiple environmental variables:
|
# Using multiple environment variables via -e (env) flag:
|
||||||
bonobo run csvsanitizer -e SRC_FILE=inventory.txt -e DST_FILE=inventory_processed.csv
|
bonobo run csvsanitizer -e SRC_FILE=inventory.txt -e DST_FILE=inventory_processed.csv
|
||||||
|
|
||||||
If you're naming something which is configurable, that is will need to be instantiated or called to obtain something that
|
# Using one environment variable in bash (*bash only):
|
||||||
can be used as a graph node, then use camelcase names:
|
SECRET_TOKEN=secret123 bonobo run csvsanitizer
|
||||||
|
|
||||||
|
# Using multiple environment variables in bash (*bash only):
|
||||||
|
SRC_FILE=inventory.txt DST_FILE=inventory_processed.csv bonobo run csvsanitizer
|
||||||
|
|
||||||
Accessing Environmental Variables from within the Graph Context
|
*Though not-yet implemented, the bonobo roadmap includes implementing environment / .env files as well.*
|
||||||
|
|
||||||
|
Accessing Environment Variables from within the Graph Context
|
||||||
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
|
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
|
||||||
|
|
||||||
Environmental variables, whether global or only for the scope of the graph,
|
Environment variables, whether set globally or only for the scope of the graph,
|
||||||
can be can be accessed using any of the normal means. It is important to note
|
can be can be accessed using any of the normal means. It is important to note
|
||||||
that whether set globally for the system or just for the graph context,
|
that whether set globally for the system or just for the graph context,
|
||||||
environmental variables are accessed by bonobo in the same way. In the example
|
environment variables are accessed by bonobo in the same way. In the example
|
||||||
below the database user and password are accessed via the ``os`` module's ``getenv``
|
below the database user and password are accessed via the ``os`` module's ``getenv``
|
||||||
function and used to get data from the database.
|
function and used to get data from the database.
|
||||||
|
|
||||||
@ -1,6 +1,6 @@
|
|||||||
Bonobo with Jupyter
|
Bonobo with Jupyter
|
||||||
===================
|
===================
|
||||||
minimalistically
|
|
||||||
There is a builtin plugin that integrates (somewhat minimallistically, for now) bonobo within jupyter notebooks, so
|
There is a builtin plugin that integrates (somewhat minimallistically, for now) bonobo within jupyter notebooks, so
|
||||||
you can read the execution status of a graph within a nice (ok, not so nice) html/javascript widget.
|
you can read the execution status of a graph within a nice (ok, not so nice) html/javascript widget.
|
||||||
|
|
||||||
|
|||||||
@ -12,7 +12,7 @@ There are a few things that you should know while writing transformations graphs
|
|||||||
purity
|
purity
|
||||||
transformations
|
transformations
|
||||||
services
|
services
|
||||||
envrionmental_variables
|
envrionment_variables
|
||||||
|
|
||||||
Third party integrations
|
Third party integrations
|
||||||
::::::::::::::::::::::::
|
::::::::::::::::::::::::
|
||||||
|
|||||||
Reference in New Issue
Block a user