Merge remote-tracking branch 'upstream/master' into develop
This commit is contained in:
@ -8,7 +8,7 @@ The major problem we have is that one message (underlying implementation: :class
|
|||||||
through more than one component, and at the same time. If you wanna be safe, you tend to :func:`copy.copy()` everything
|
through more than one component, and at the same time. If you wanna be safe, you tend to :func:`copy.copy()` everything
|
||||||
between two calls to two different components, but that's very expensive.
|
between two calls to two different components, but that's very expensive.
|
||||||
|
|
||||||
Instead of that, we chosed the oposite: copies are never made, and you should not modify in place the inputs of your
|
Instead, we chose the opposite: copies are never made, and you should not modify in place the inputs of your
|
||||||
component before yielding them, and that mostly means that you want to recreate dicts and lists before yielding (or
|
component before yielding them, and that mostly means that you want to recreate dicts and lists before yielding (or
|
||||||
returning) them. Numeric values, strings and tuples being immutable in python, modifying a variable of one of those
|
returning) them. Numeric values, strings and tuples being immutable in python, modifying a variable of one of those
|
||||||
type will already return a different instance.
|
type will already return a different instance.
|
||||||
@ -130,10 +130,10 @@ Now let's see how to do it correctly:
|
|||||||
|
|
||||||
I hear you think «Yeah, but if I create like millions of dicts ...».
|
I hear you think «Yeah, but if I create like millions of dicts ...».
|
||||||
|
|
||||||
Let's say we chosed the oposite way and copy the dict outside the transformation (in fact, `it's what we did in bonobo's
|
Let's say we chose the opposite way and copied the dict outside the transformation (in fact, `it's what we did in bonobo's
|
||||||
ancestor <https://github.com/rdcli/rdc.etl/blob/dev/rdc/etl/io/__init__.py#L187>`_). This means you will also create the
|
ancestor <https://github.com/rdcli/rdc.etl/blob/dev/rdc/etl/io/__init__.py#L187>`_). This means you will also create the
|
||||||
same number of dicts, the difference is that you won't even notice it. Also, it means that if you want to yield 1 million
|
same number of dicts, the difference is that you won't even notice it. Also, it means that if you want to yield the same
|
||||||
times the same dict, going "pure" makes it efficient (you'll just yield the same object 1 million times) while going "copy
|
dict 1 million times , going "pure" makes it efficient (you'll just yield the same object 1 million times) while going "copy
|
||||||
crazy" will create 1 million objects.
|
crazy" will create 1 million objects.
|
||||||
|
|
||||||
Using dicts like this will create a lot of dicts, but also free them as soon as all the future components that take this dict
|
Using dicts like this will create a lot of dicts, but also free them as soon as all the future components that take this dict
|
||||||
|
|||||||
@ -1,10 +1,14 @@
|
|||||||
References
|
References
|
||||||
==========
|
==========
|
||||||
|
|
||||||
|
Reference documents of all stable APIs and modules. If something is not here, please be careful about using it as it
|
||||||
|
means that the api is not yet 1.0-proof.
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 4
|
:maxdepth: 4
|
||||||
|
|
||||||
api
|
api
|
||||||
api_config
|
api_config
|
||||||
commands
|
commands
|
||||||
|
settings
|
||||||
examples
|
examples
|
||||||
|
|||||||
56
docs/reference/settings.rst
Normal file
56
docs/reference/settings.rst
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
Settings & Environment
|
||||||
|
======================
|
||||||
|
|
||||||
|
.. module:: bonobo.settings
|
||||||
|
|
||||||
|
All settings that you can find in the :module:`bonobo.settings` module.
|
||||||
|
|
||||||
|
Debug
|
||||||
|
:::::
|
||||||
|
|
||||||
|
:Purpose: Sets the debug mode, which is more verbose. Loglevel will be lowered to DEBUG instead of INFO.
|
||||||
|
:Environment: `DEBUG`
|
||||||
|
:Setting: `bonobo.settings.DEBUG`
|
||||||
|
:Default: `False`
|
||||||
|
|
||||||
|
Profile
|
||||||
|
:::::::
|
||||||
|
|
||||||
|
:Purpose: Sets profiling, which adds memory/cpu usage output. Not yet fully implemented. It is expected that setting
|
||||||
|
this to true will have a non-neglictible performance impact.
|
||||||
|
:Environment: `PROFILE`
|
||||||
|
:Setting: `bonobo.settings.PROFILE`
|
||||||
|
:Default: `False`
|
||||||
|
|
||||||
|
Quiet
|
||||||
|
:::::
|
||||||
|
|
||||||
|
:Purpose: Sets the quiet mode, which ask any output to be computer parsable. Formating will be removed, but it will
|
||||||
|
allow to use unix pipes, etc. Not yet fully implemented, few transformations already use it. Probably, it
|
||||||
|
should be the default on non-interactive terminals.
|
||||||
|
:Environment: `QUIET`
|
||||||
|
:Setting: `bonobo.settings.QUIET`
|
||||||
|
:Default: `False`
|
||||||
|
|
||||||
|
Logging Level
|
||||||
|
:::::::::::::
|
||||||
|
|
||||||
|
:Purpose: Sets the python minimum logging level.
|
||||||
|
:Environment: `LOGGING_LEVEL`
|
||||||
|
:Setting: `bonobo.settings.LOGGING_LEVEL`
|
||||||
|
:Default: `DEBUG` if DEBUG is False, otherwise `INFO`
|
||||||
|
:Values: `CRITICAL`, `FATAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG`, `NOTSET`
|
||||||
|
|
||||||
|
I/O Format
|
||||||
|
::::::::::
|
||||||
|
|
||||||
|
:Purpose: Sets default input/output format for builtin transformations. It can be overriden on each node. The `kwargs`
|
||||||
|
value means that each node will try to read its input from keywords arguments (and write similar formated
|
||||||
|
output), while `arg0` means it will try to read its input from the first positional argument (and write
|
||||||
|
similar formated output).
|
||||||
|
:Environment: `IOFORMAT`
|
||||||
|
:Setting: `bonobo.settings.IOFORMAT`
|
||||||
|
:Default: `kwargs`
|
||||||
|
:Values: `kwargs`, `arg0`
|
||||||
|
|
||||||
|
|
||||||
Reference in New Issue
Block a user