[doc] proofreading the guides, refactoring the reference.

This commit is contained in:
Romain Dorgueil
2018-01-16 06:27:25 +01:00
parent ed7887ba31
commit aa6e426768
41 changed files with 767 additions and 288 deletions

View File

@ -0,0 +1,56 @@
Services
========
.. warning::
This is a "future" document, that does not exist, it's only kept here not to lose the data until we organize better
documentation versioning.
Future and proposals
::::::::::::::::::::
This is a first implementation and it will evolve. Base concepts will stay the same though.
May or may not happen, depending on discussions.
* Singleton or prototype based injection (to use spring terminology, see
https://www.tutorialspoint.com/spring/spring_bean_scopes.htm), allowing smart factory usage and efficient sharing of
resources.
* Lazily resolved parameters, eventually overriden by command line or environment, so you can for example override the
database DSN or target filesystem on command line (or with shell environment vars).
* Pool based locks that ensure that only one (or n) transformations are using a given service at the same time.
* Simple config implementation, using a python file for config (ex: bonobo run ... --services=services_prod.py).
* Default configuration for services, using an optional callable (`def get_services(args): ...`). Maybe tie default
configuration to graph, but not really a fan because this is unrelated to graph logic.
* Default implementation for a service in a transformation or in the descriptor. Maybe not a good idea, because it
tends to push forward multiple instances of the same thing, but maybe...
A few ideas on how it can be implemented, from the user perspective.
.. code-block:: python
# using call
http = Service('http.client')(requests)
# using more explicit call
http = Service('http.client').set_default_impl(requests)
# using a decorator
@Service('http.client')
def http(self, services):
import requests
return requests
# as a default in a subclass of Service
class HttpService(Service):
def get_default_impl(self, services):
import requests
return requests
# ... then use it as another service
http = HttpService('http.client')
This is under development, let us know what you think (slack may be a good place for this).
The basics already work, and you can try it.

View File

@ -0,0 +1,83 @@
Transformations
===============
.. warning::
This is a "future" document, that does not exist, it's only kept here not to lose the data until we organize better
documentation versioning.
Output
------
Let's see the rules (first to match wins).
1. A flag, eventually followed by something else, marks a special behaviour. If it supports it, the remaining part of
the output line will be interpreted using the same rules, and some flags can be combined.
**NOT_MODIFIED**
**NOT_MODIFIED** tells bonobo to use the input row unmodified as the output.
*CANNOT be combined*
Example:
.. code-block:: python
from bonobo import NOT_MODIFIED
def output_will_be_same_as_input(*args, **kwargs):
yield NOT_MODIFIED
2. Once all flags are "consumed", the remaining part is interpreted.
* If it is a :class:`bonobo.Bag` instance, then it's used directly.
* If it is a :class:`dict` then a kwargs-only :class:`bonobo.Bag` will be created.
* If it is a :class:`tuple` then an args-only :class:`bonobo.Bag` will be created, unless its last argument is a
:class:`dict` in which case a args+kwargs :class:`bonobo.Bag` will be created.
* If it's something else, it will be used to create a one-arg-only :class:`bonobo.Bag`.
**APPEND**
**APPEND** tells bonobo to append this output to the input (positional arguments will equal `input_args + output_args`,
keyword arguments will equal `{**input_kwargs, **output_kwargs}`).
*CAN be combined, but not with itself*
.. code-block:: python
from bonobo import APPEND
def output_will_be_appended_to_input(*args, **kwargs):
yield APPEND, 'foo', 'bar', {'eat_at': 'joe'}
**LOOPBACK**
**LOOPBACK** tells bonobo that this output must be looped back into our own input queue, allowing to create the stream
processing version of recursive algorithms.
*CAN be combined, but not with itself*
.. code-block:: python
from bonobo import LOOPBACK
def output_will_be_sent_to_self(*args, **kwargs):
yield LOOPBACK, 'Hello, I am the future "you".'
**CHANNEL(...)**
**CHANNEL(...)** tells bonobo that this output does not use the default channel and is routed through another path.
This is something you should probably not use unless your data flow design is complex, and if you're not certain
about it, it probably means that it is not the feature you're looking for.
*CAN be combined, but not with itself*
.. code-block:: python
from bonobo import CHANNEL
def output_will_be_sent_to_self(*args, **kwargs):
yield CHANNEL("errors"), 'That is not cool.'