Merge remote-tracking branch 'upstream/master' into develop

This commit is contained in:
Romain Dorgueil
2017-12-02 18:24:56 +01:00
3 changed files with 12 additions and 13 deletions

View File

@ -1,4 +1,4 @@
# Generated by Medikit 0.4.2 on 2017-12-01. # Generated by Medikit 0.4.2 on 2017-12-02.
# All changes will be overriden. # All changes will be overriden.
PACKAGE ?= bonobo PACKAGE ?= bonobo

View File

@ -1,9 +1,8 @@
Graphs Graphs
====== ======
Graphs are the glue that ties transformations together. It's the only data-structure bonobo can execute directly. Graphs Graphs are the glue that ties transformations together. They are the only data-structure bonobo can execute directly. Graphs
must be acyclic, and can contain as much nodes as your system can handle. Although this number can be rather high in must be acyclic, and can contain as many nodes as your system can handle. However, although in theory the number of nodes can be rather high, practical use cases usually do not exceed more than a few hundred nodes and only then in extreme cases.
theory, extreme practical cases usually do not exceed hundreds of nodes (and this is already extreme, really).
Definitions Definitions
@ -50,7 +49,7 @@ Non-linear graphs
Divergences / forks Divergences / forks
------------------- -------------------
To create two or more divergent data streams ("fork"), you should specify `_input` kwarg to `add_chain`. To create two or more divergent data streams ("forks"), you should specify the `_input` kwarg to `add_chain`.
.. code-block:: python .. code-block:: python
@ -74,12 +73,12 @@ Resulting graph:
"b" -> "f" -> "g"; "b" -> "f" -> "g";
} }
.. note:: Both branch will receive the same data, at the same time. .. note:: Both branches will receive the same data and at the same time.
Convergences / merges Convergence / merges
--------------------- ---------------------
To merge two data streams ("merge"), you can use the `_output` kwarg to `add_chain`, or use named nodes (see below). To merge two data streams, you can use the `_output` kwarg to `add_chain`, or use named nodes (see below).
.. code-block:: python .. code-block:: python
@ -88,7 +87,7 @@ To merge two data streams ("merge"), you can use the `_output` kwarg to `add_cha
graph = bonobo.Graph() graph = bonobo.Graph()
# Here we mark _input to None, so normalize won't get the "begin" impulsion. # Here we set _input to None, so normalize won't start on its own but only after it receives input from the other chains.
graph.add_chain(normalize, store, _input=None) graph.add_chain(normalize, store, _input=None)
# Add two different chains # Add two different chains
@ -122,7 +121,7 @@ Resulting graph:
Named nodes Named nodes
::::::::::: :::::::::::
Using above code to create convergences can lead to hard to read code, because you have to define the "target" stream Using above code to create convergences often leads to code which is hard to read, because you have to define the "target" stream
before the streams that logically goes to the beginning of the transformation graph. To overcome that, one can use before the streams that logically goes to the beginning of the transformation graph. To overcome that, one can use
"named" nodes: "named" nodes:
@ -194,7 +193,7 @@ You can also run a python module:
$ bonobo run -m my.own.etlmod $ bonobo run -m my.own.etlmod
In each case, bonobo's CLI will look for an instance of :class:`bonobo.Graph` in your file/module, create the plumbery In each case, bonobo's CLI will look for an instance of :class:`bonobo.Graph` in your file/module, create the plumbing
needed to execute it, and run it. needed to execute it, and run it.
If you're in an interactive terminal context, it will use :class:`bonobo.ext.console.ConsoleOutputPlugin` for display. If you're in an interactive terminal context, it will use :class:`bonobo.ext.console.ConsoleOutputPlugin` for display.

View File

@ -3,7 +3,7 @@ appnope==0.1.0
bleach==2.1.1 bleach==2.1.1
decorator==4.1.2 decorator==4.1.2
entrypoints==0.2.3 entrypoints==0.2.3
html5lib==0.999999999 html5lib==1.0b10
ipykernel==4.6.1 ipykernel==4.6.1
ipython-genutils==0.2.0 ipython-genutils==0.2.0
ipython==6.2.1 ipython==6.2.1
@ -28,7 +28,7 @@ prompt-toolkit==1.0.15
ptyprocess==0.5.2 ptyprocess==0.5.2
pygments==2.2.0 pygments==2.2.0
python-dateutil==2.6.1 python-dateutil==2.6.1
pyzmq==16.0.3 pyzmq==17.0.0b3
qtconsole==4.3.1 qtconsole==4.3.1
simplegeneric==0.8.1 simplegeneric==0.8.1
six==1.11.0 six==1.11.0