Merge remote-tracking branch 'upstream/master' into develop
This commit is contained in:
2
Makefile
2
Makefile
@ -1,4 +1,4 @@
|
|||||||
# Generated by Medikit 0.4.2 on 2017-12-01.
|
# Generated by Medikit 0.4.2 on 2017-12-02.
|
||||||
# All changes will be overriden.
|
# All changes will be overriden.
|
||||||
|
|
||||||
PACKAGE ?= bonobo
|
PACKAGE ?= bonobo
|
||||||
|
|||||||
@ -1,9 +1,8 @@
|
|||||||
Graphs
|
Graphs
|
||||||
======
|
======
|
||||||
|
|
||||||
Graphs are the glue that ties transformations together. It's the only data-structure bonobo can execute directly. Graphs
|
Graphs are the glue that ties transformations together. They are the only data-structure bonobo can execute directly. Graphs
|
||||||
must be acyclic, and can contain as much nodes as your system can handle. Although this number can be rather high in
|
must be acyclic, and can contain as many nodes as your system can handle. However, although in theory the number of nodes can be rather high, practical use cases usually do not exceed more than a few hundred nodes and only then in extreme cases.
|
||||||
theory, extreme practical cases usually do not exceed hundreds of nodes (and this is already extreme, really).
|
|
||||||
|
|
||||||
|
|
||||||
Definitions
|
Definitions
|
||||||
@ -50,7 +49,7 @@ Non-linear graphs
|
|||||||
Divergences / forks
|
Divergences / forks
|
||||||
-------------------
|
-------------------
|
||||||
|
|
||||||
To create two or more divergent data streams ("fork"), you should specify `_input` kwarg to `add_chain`.
|
To create two or more divergent data streams ("forks"), you should specify the `_input` kwarg to `add_chain`.
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
@ -74,12 +73,12 @@ Resulting graph:
|
|||||||
"b" -> "f" -> "g";
|
"b" -> "f" -> "g";
|
||||||
}
|
}
|
||||||
|
|
||||||
.. note:: Both branch will receive the same data, at the same time.
|
.. note:: Both branches will receive the same data and at the same time.
|
||||||
|
|
||||||
Convergences / merges
|
Convergence / merges
|
||||||
---------------------
|
---------------------
|
||||||
|
|
||||||
To merge two data streams ("merge"), you can use the `_output` kwarg to `add_chain`, or use named nodes (see below).
|
To merge two data streams, you can use the `_output` kwarg to `add_chain`, or use named nodes (see below).
|
||||||
|
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
@ -88,7 +87,7 @@ To merge two data streams ("merge"), you can use the `_output` kwarg to `add_cha
|
|||||||
|
|
||||||
graph = bonobo.Graph()
|
graph = bonobo.Graph()
|
||||||
|
|
||||||
# Here we mark _input to None, so normalize won't get the "begin" impulsion.
|
# Here we set _input to None, so normalize won't start on its own but only after it receives input from the other chains.
|
||||||
graph.add_chain(normalize, store, _input=None)
|
graph.add_chain(normalize, store, _input=None)
|
||||||
|
|
||||||
# Add two different chains
|
# Add two different chains
|
||||||
@ -122,7 +121,7 @@ Resulting graph:
|
|||||||
Named nodes
|
Named nodes
|
||||||
:::::::::::
|
:::::::::::
|
||||||
|
|
||||||
Using above code to create convergences can lead to hard to read code, because you have to define the "target" stream
|
Using above code to create convergences often leads to code which is hard to read, because you have to define the "target" stream
|
||||||
before the streams that logically goes to the beginning of the transformation graph. To overcome that, one can use
|
before the streams that logically goes to the beginning of the transformation graph. To overcome that, one can use
|
||||||
"named" nodes:
|
"named" nodes:
|
||||||
|
|
||||||
@ -194,7 +193,7 @@ You can also run a python module:
|
|||||||
|
|
||||||
$ bonobo run -m my.own.etlmod
|
$ bonobo run -m my.own.etlmod
|
||||||
|
|
||||||
In each case, bonobo's CLI will look for an instance of :class:`bonobo.Graph` in your file/module, create the plumbery
|
In each case, bonobo's CLI will look for an instance of :class:`bonobo.Graph` in your file/module, create the plumbing
|
||||||
needed to execute it, and run it.
|
needed to execute it, and run it.
|
||||||
|
|
||||||
If you're in an interactive terminal context, it will use :class:`bonobo.ext.console.ConsoleOutputPlugin` for display.
|
If you're in an interactive terminal context, it will use :class:`bonobo.ext.console.ConsoleOutputPlugin` for display.
|
||||||
|
|||||||
@ -3,7 +3,7 @@ appnope==0.1.0
|
|||||||
bleach==2.1.1
|
bleach==2.1.1
|
||||||
decorator==4.1.2
|
decorator==4.1.2
|
||||||
entrypoints==0.2.3
|
entrypoints==0.2.3
|
||||||
html5lib==0.999999999
|
html5lib==1.0b10
|
||||||
ipykernel==4.6.1
|
ipykernel==4.6.1
|
||||||
ipython-genutils==0.2.0
|
ipython-genutils==0.2.0
|
||||||
ipython==6.2.1
|
ipython==6.2.1
|
||||||
@ -28,7 +28,7 @@ prompt-toolkit==1.0.15
|
|||||||
ptyprocess==0.5.2
|
ptyprocess==0.5.2
|
||||||
pygments==2.2.0
|
pygments==2.2.0
|
||||||
python-dateutil==2.6.1
|
python-dateutil==2.6.1
|
||||||
pyzmq==16.0.3
|
pyzmq==17.0.0b3
|
||||||
qtconsole==4.3.1
|
qtconsole==4.3.1
|
||||||
simplegeneric==0.8.1
|
simplegeneric==0.8.1
|
||||||
six==1.11.0
|
six==1.11.0
|
||||||
|
|||||||
Reference in New Issue
Block a user