[doc] proofreading the guides, refactoring the reference.
This commit is contained in:
56
docs/guide/future/services.rst
Normal file
56
docs/guide/future/services.rst
Normal file
@ -0,0 +1,56 @@
|
||||
Services
|
||||
========
|
||||
|
||||
.. warning::
|
||||
|
||||
This is a "future" document, that does not exist, it's only kept here not to lose the data until we organize better
|
||||
documentation versioning.
|
||||
|
||||
Future and proposals
|
||||
::::::::::::::::::::
|
||||
|
||||
This is a first implementation and it will evolve. Base concepts will stay the same though.
|
||||
|
||||
May or may not happen, depending on discussions.
|
||||
|
||||
* Singleton or prototype based injection (to use spring terminology, see
|
||||
https://www.tutorialspoint.com/spring/spring_bean_scopes.htm), allowing smart factory usage and efficient sharing of
|
||||
resources.
|
||||
* Lazily resolved parameters, eventually overriden by command line or environment, so you can for example override the
|
||||
database DSN or target filesystem on command line (or with shell environment vars).
|
||||
* Pool based locks that ensure that only one (or n) transformations are using a given service at the same time.
|
||||
* Simple config implementation, using a python file for config (ex: bonobo run ... --services=services_prod.py).
|
||||
* Default configuration for services, using an optional callable (`def get_services(args): ...`). Maybe tie default
|
||||
configuration to graph, but not really a fan because this is unrelated to graph logic.
|
||||
* Default implementation for a service in a transformation or in the descriptor. Maybe not a good idea, because it
|
||||
tends to push forward multiple instances of the same thing, but maybe...
|
||||
|
||||
A few ideas on how it can be implemented, from the user perspective.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# using call
|
||||
http = Service('http.client')(requests)
|
||||
|
||||
# using more explicit call
|
||||
http = Service('http.client').set_default_impl(requests)
|
||||
|
||||
# using a decorator
|
||||
@Service('http.client')
|
||||
def http(self, services):
|
||||
import requests
|
||||
return requests
|
||||
|
||||
# as a default in a subclass of Service
|
||||
class HttpService(Service):
|
||||
def get_default_impl(self, services):
|
||||
import requests
|
||||
return requests
|
||||
|
||||
# ... then use it as another service
|
||||
http = HttpService('http.client')
|
||||
|
||||
|
||||
This is under development, let us know what you think (slack may be a good place for this).
|
||||
The basics already work, and you can try it.
|
||||
|
||||
83
docs/guide/future/transformations.rst
Normal file
83
docs/guide/future/transformations.rst
Normal file
@ -0,0 +1,83 @@
|
||||
Transformations
|
||||
===============
|
||||
|
||||
.. warning::
|
||||
|
||||
This is a "future" document, that does not exist, it's only kept here not to lose the data until we organize better
|
||||
documentation versioning.
|
||||
|
||||
|
||||
Output
|
||||
------
|
||||
|
||||
Let's see the rules (first to match wins).
|
||||
|
||||
1. A flag, eventually followed by something else, marks a special behaviour. If it supports it, the remaining part of
|
||||
the output line will be interpreted using the same rules, and some flags can be combined.
|
||||
|
||||
**NOT_MODIFIED**
|
||||
|
||||
**NOT_MODIFIED** tells bonobo to use the input row unmodified as the output.
|
||||
|
||||
*CANNOT be combined*
|
||||
|
||||
Example:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from bonobo import NOT_MODIFIED
|
||||
|
||||
def output_will_be_same_as_input(*args, **kwargs):
|
||||
yield NOT_MODIFIED
|
||||
|
||||
2. Once all flags are "consumed", the remaining part is interpreted.
|
||||
|
||||
* If it is a :class:`bonobo.Bag` instance, then it's used directly.
|
||||
* If it is a :class:`dict` then a kwargs-only :class:`bonobo.Bag` will be created.
|
||||
* If it is a :class:`tuple` then an args-only :class:`bonobo.Bag` will be created, unless its last argument is a
|
||||
:class:`dict` in which case a args+kwargs :class:`bonobo.Bag` will be created.
|
||||
* If it's something else, it will be used to create a one-arg-only :class:`bonobo.Bag`.
|
||||
|
||||
**APPEND**
|
||||
|
||||
**APPEND** tells bonobo to append this output to the input (positional arguments will equal `input_args + output_args`,
|
||||
keyword arguments will equal `{**input_kwargs, **output_kwargs}`).
|
||||
|
||||
*CAN be combined, but not with itself*
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from bonobo import APPEND
|
||||
|
||||
def output_will_be_appended_to_input(*args, **kwargs):
|
||||
yield APPEND, 'foo', 'bar', {'eat_at': 'joe'}
|
||||
|
||||
**LOOPBACK**
|
||||
|
||||
**LOOPBACK** tells bonobo that this output must be looped back into our own input queue, allowing to create the stream
|
||||
processing version of recursive algorithms.
|
||||
|
||||
*CAN be combined, but not with itself*
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from bonobo import LOOPBACK
|
||||
|
||||
def output_will_be_sent_to_self(*args, **kwargs):
|
||||
yield LOOPBACK, 'Hello, I am the future "you".'
|
||||
|
||||
**CHANNEL(...)**
|
||||
|
||||
**CHANNEL(...)** tells bonobo that this output does not use the default channel and is routed through another path.
|
||||
This is something you should probably not use unless your data flow design is complex, and if you're not certain
|
||||
about it, it probably means that it is not the feature you're looking for.
|
||||
|
||||
*CAN be combined, but not with itself*
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from bonobo import CHANNEL
|
||||
|
||||
def output_will_be_sent_to_self(*args, **kwargs):
|
||||
yield CHANNEL("errors"), 'That is not cool.'
|
||||
|
||||
Reference in New Issue
Block a user