Documenting transformations and configurables.

This commit is contained in:
Romain Dorgueil
2017-05-20 13:05:07 +02:00
parent 577a781de3
commit a018cca20e
11 changed files with 210 additions and 80 deletions

View File

@ -10,6 +10,7 @@ There are a few things that you should know while writing transformations graphs
:maxdepth: 2
purity
transformations
services
Third party integrations

View File

@ -1,20 +1,18 @@
Services and dependencies (draft implementation)
================================================
Services and dependencies
=========================
:Status: Draft implementation
:Stability: Alpha
:Last-Modified: 28 apr 2017
:Last-Modified: 20 may 2017
Most probably, you'll want to use external systems within your transformations. Those systems may include databases,
apis (using http, for example), filesystems, etc.
You'll probably want to use external systems within your transformations. Those systems may include databases, apis
(using http, for example), filesystems, etc.
You can start by hardcoding those services. That does the job, at first.
If you're going a little further than that, you'll feel limited, for a few reasons:
* Hardcoded and tightly linked dependencies make your transformations hard to test, and hard to reuse.
* Processing data on your laptop is great, but being able to do it on different systems (or stages), in different
environments, is more realistic? You probably want to contigure a different database on a staging environment,
* Processing data on your laptop is great, but being able to do it on different target systems (or stages), in different
environments, is more realistic. You'll want to contigure a different database on a staging environment,
preprod environment or production system. Maybe you have silimar systems for different clients and want to select
the system at runtime. Etc.
@ -52,10 +50,11 @@ injected to your calls under the parameter name "database".
Function-based transformations
------------------------------
No implementation yet, but expect something similar to CBT API, maybe using a `@Service(...)` decorator.
No implementation yet, but expect something similar to CBT API, maybe using a `@Service(...)` decorator. See
`issue #70 <https://github.com/python-bonobo/bonobo/issues/70>`_.
Execution
---------
Provide implementation at run time
----------------------------------
Let's see how to execute it:

View File

@ -0,0 +1,89 @@
Transformations
===============
Here is some guidelines on how to write transformations, to avoid the convention-jungle that could happen without
a few rules.
Naming conventions
::::::::::::::::::
The naming convention used is the following.
If you're naming something which is an actual transformation, that can be used directly as a graph node, then use
underscores and lowercase names:
.. code-block:: python
# instance of a class based transformation
filter = Filter(...)
# function based transformation
def uppercase(s: str) -> str:
return s.upper()
If you're naming something which is configurable, that will need to be instanciated or called to obtain something that
can be used as a graph node, then use camelcase names:
.. code-block:: python
# configurable
class ChangeCase(Configurable):
modifier = Option(default='upper')
def call(self, s: str) -> str:
return getattr(s, self.modifier)()
# transformation factory
def Apply(method):
@functools.wraps(method)
def apply(s: str) -> str:
return method(s)
return apply
# result is a graph node candidate
upper = Apply(str.upper)
Function based transformations
::::::::::::::::::::::::::::::
The most basic transformations are function-based. Which means that you define a function, and it will be used directly
in a graph.
.. code-block:: python
def get_representation(row):
return repr(row)
graph = bonobo.Graph(
[...],
get_representation,
)
It does not allow any configuration, but if it's an option, prefer it as it's simpler to write.
Class based transformations
:::::::::::::::::::::::::::
A lot of logic is a bit more complex, and you'll want to use classes to define some of your transformations.
The :class:`bonobo.config.Configurable` class gives you a few toys to write configurable transformations.
Options
-------
.. autoclass:: bonobo.config.Option
Services
--------
.. autoclass:: bonobo.config.Service
Method
------
.. autoclass:: bonobo.config.Method