@ -36,6 +36,25 @@ The `sqlalchemy.engine` name is the default name used by the provided transforma
|
||||
example if you need more than one connection) and specify the service name using `engine='myengine'` while building your
|
||||
transformations.
|
||||
|
||||
Lets create some tables and add some data. (You may need to edit the SQL if your database server uses a different
|
||||
version of SQL.)
|
||||
|
||||
.. code-block:: sql
|
||||
|
||||
CREATE TABLE test_in (
|
||||
id INTEGER PRIMARY KEY NOT NULL,
|
||||
text TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE test_out (
|
||||
id INTEGER PRIMARY KEY NOT NULL,
|
||||
text TEXT
|
||||
);
|
||||
|
||||
INSERT INTO test_in (id, text) VALUES (1, 'Cat');
|
||||
INSERT INTO test_in (id, text) VALUES (2, 'Dog');
|
||||
|
||||
|
||||
There are two transformation classes provided by this extension.
|
||||
|
||||
One reader, one writer.
|
||||
@ -50,12 +69,29 @@ Let's select some data:
|
||||
def get_graph():
|
||||
graph = bonobo.Graph()
|
||||
graph.add_chain(
|
||||
bonobo_sqlalchemy.Select('SELECT * FROM example', limit=100),
|
||||
bonobo_sqlalchemy.Select('SELECT * FROM test_in', limit=100),
|
||||
bonobo.PrettyPrinter(),
|
||||
)
|
||||
return graph
|
||||
|
||||
And let's insert some data:
|
||||
You should see:
|
||||
|
||||
.. code-block:: shell-session
|
||||
|
||||
$ python tutorial.py
|
||||
┌
|
||||
│ id[0] = 1
|
||||
│ text[1] = 'Cat'
|
||||
└
|
||||
┌
|
||||
│ id[0] = 2
|
||||
│ text[1] = 'Dog'
|
||||
└
|
||||
- Select in=1 out=2 [done]
|
||||
- PrettyPrinter in=2 out=2 [done]
|
||||
|
||||
|
||||
Now let's insert some data:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
@ -66,12 +102,14 @@ And let's insert some data:
|
||||
def get_graph(**options):
|
||||
graph = bonobo.Graph()
|
||||
graph.add_chain(
|
||||
...,
|
||||
bonobo_sqlalchemy.InsertOrUpdate('example')
|
||||
bonobo_sqlalchemy.Select('SELECT * FROM test_in', limit=100),
|
||||
bonobo_sqlalchemy.InsertOrUpdate('test_out')
|
||||
)
|
||||
|
||||
return graph
|
||||
|
||||
If you check the `test_out` table, it should now have the data.
|
||||
|
||||
Reference
|
||||
:::::::::
|
||||
|
||||
|
||||
@ -41,7 +41,7 @@ Now, we need to write a `writer` transformation, and apply this context processo
|
||||
|
||||
@use_context_processor(with_opened_file)
|
||||
def write_repr_to_file(f, *row):
|
||||
f.write(repr(row))
|
||||
f.write(repr(row) + "\n")
|
||||
|
||||
The `f` parameter will contain the value yielded by the context processors, in order of appearance (you can chain
|
||||
multiple context processors).
|
||||
@ -50,6 +50,20 @@ Please note that the :func:`bonobo.config.use_context_processor` decorator will
|
||||
modify its behaviour. If you want to call it out of the |bonobo| job context, it's your responsibility to provide
|
||||
the right parameters (and here, the opened file).
|
||||
|
||||
To run this, change the last stage in the pipeline in get_graph to write_repr_to_file
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def get_graph(**options):
|
||||
graph = bonobo.Graph()
|
||||
graph.add_chain(
|
||||
extract_fablabs,
|
||||
bonobo.Limit(10),
|
||||
write_repr_to_file,
|
||||
)
|
||||
return graph
|
||||
|
||||
Now run tutorial.py and check the output.txt file.
|
||||
|
||||
Using the filesystem
|
||||
::::::::::::::::::::
|
||||
|
||||
@ -3,7 +3,7 @@ Part 4: Services
|
||||
|
||||
All external dependencies (like filesystems, network clients, database connections, etc.) should be provided to
|
||||
transformations as a service. It allows great flexibility, including the ability to test your transformations isolated
|
||||
from the external world, and being friendly to the infrastructure guys (and if you're one of them, it's also nice to
|
||||
from the external world, and being friendly to the infrastructure people (and if you're one of them, it's also nice to
|
||||
treat yourself well).
|
||||
|
||||
In the last section, we used the `fs` service to access filesystems, we'll go even further by switching our `requests`
|
||||
|
||||
Reference in New Issue
Block a user