Update services.rst
This commit is contained in:
@ -1,21 +1,33 @@
|
|||||||
Services and dependencies (draft implementation)
|
Services and dependencies (draft implementation)
|
||||||
================================================
|
================================================
|
||||||
|
|
||||||
|
:Status: Draft implementation
|
||||||
|
:Stability: Alpha
|
||||||
|
:Last-Modified: 27 apr 2017
|
||||||
|
|
||||||
Most probably, you'll want to use external systems within your transformations. Those systems may include databases,
|
Most probably, you'll want to use external systems within your transformations. Those systems may include databases,
|
||||||
apis (using http, for example), filesystems, etc.
|
apis (using http, for example), filesystems, etc.
|
||||||
|
|
||||||
For a start, including those services hardcoded in your transformations can do the job, but you'll pretty soon feel
|
Hardocing those services in your code can do the job at first, but you'll pretty soon feel limited, for a few reasons:
|
||||||
limited, for two main reasons:
|
|
||||||
|
|
||||||
* Hardcoded and tightly linked dependencies make your transformation atoms hard to test.
|
* Hardcoded and tightly linked dependencies make your transformations hard to test, and hard to reuse.
|
||||||
* Processing data on your laptop is great, but being able to do it on different systems (or stages), in different
|
* Processing data on your laptop is great, but being able to do it on different systems (or stages), in different
|
||||||
environments, is more realistic.
|
environments, is more realistic? You probably want to contigure a different database on a staging environment,
|
||||||
|
preprod environment or production system. Maybe you have silimar systems for different clients and want to select
|
||||||
|
the system at runtime. Etc.
|
||||||
|
|
||||||
Service injection
|
Service injection
|
||||||
:::::::::::::::::
|
:::::::::::::::::
|
||||||
|
|
||||||
To solve this problem, we introduce a light dependency injection system that basically allows you to define named
|
To solve this problem, we introduce a light dependency injection system. It allows to define named dependencies in
|
||||||
dependencies in your transformations, and provide an implementation at runtime.
|
your transformations, and provide an implementation at runtime.
|
||||||
|
|
||||||
|
Class-based transformations
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
To define a service dependency in a class-based transformation, use :class:`bonobo.config.Service`, a special
|
||||||
|
descriptor (and subclass of :class:`bonobo.config.Option`) that will hold the service names and act as a marker
|
||||||
|
for runtime resolution of service instances.
|
||||||
|
|
||||||
Let's define such a transformation:
|
Let's define such a transformation:
|
||||||
|
|
||||||
@ -24,7 +36,7 @@ Let's define such a transformation:
|
|||||||
from bonobo.config import Configurable, Service
|
from bonobo.config import Configurable, Service
|
||||||
|
|
||||||
class JoinDatabaseCategories(Configurable):
|
class JoinDatabaseCategories(Configurable):
|
||||||
database = Service(default='primary_sql_database')
|
database = Service('primary_sql_database')
|
||||||
|
|
||||||
def __call__(self, database, row):
|
def __call__(self, database, row):
|
||||||
return {
|
return {
|
||||||
@ -35,6 +47,14 @@ Let's define such a transformation:
|
|||||||
This piece of code tells bonobo that your transformation expect a sercive called "primary_sql_database", that will be
|
This piece of code tells bonobo that your transformation expect a sercive called "primary_sql_database", that will be
|
||||||
injected to your calls under the parameter name "database".
|
injected to your calls under the parameter name "database".
|
||||||
|
|
||||||
|
Function-based transformations
|
||||||
|
------------------------------
|
||||||
|
|
||||||
|
No implementation yet, but expect something similar to CBT API, maybe using a `@Service(...)` decorator.
|
||||||
|
|
||||||
|
Execution
|
||||||
|
---------
|
||||||
|
|
||||||
Let's see how to execute it:
|
Let's see how to execute it:
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
@ -50,6 +70,11 @@ Let's see how to execute it:
|
|||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
|
A dictionary, or dictionary-like, "services" named argument can be passed to the :func:`bonobo.run` helper. The
|
||||||
|
"dictionary-like" part is the real keyword here. Bonobo is not a DIC library, and won't become one. So the implementation
|
||||||
|
provided is pretty basic, and feature-less. But you can use much more evolved libraries instead of the provided
|
||||||
|
stub, and as long as it works the same (a.k.a implements a dictionary-like interface), the system will use it.
|
||||||
|
|
||||||
Future
|
Future
|
||||||
::::::
|
::::::
|
||||||
|
|
||||||
@ -65,10 +90,10 @@ You can expect to see the following features pretty soon:
|
|||||||
database DSN or target filesystem on command line (or with shell environment).
|
database DSN or target filesystem on command line (or with shell environment).
|
||||||
* Pool based locks that ensure that only one (or n) transformations are using a given service at the same time.
|
* Pool based locks that ensure that only one (or n) transformations are using a given service at the same time.
|
||||||
|
|
||||||
This is under heavy development, let us know what you think (slack may be a good place for this).
|
This is under development, let us know what you think (slack may be a good place for this).
|
||||||
|
|
||||||
|
|
||||||
Read more
|
Read more
|
||||||
:::::::::
|
:::::::::
|
||||||
|
|
||||||
todo: example code.
|
* See https://github.com/hartym/bonobo-sqlalchemy/blob/work-in-progress/bonobo_sqlalchemy/writers.py#L19 for example usage (work in progress).
|
||||||
|
|||||||
Reference in New Issue
Block a user