diff --git a/docs/extension/sqlalchemy.rst b/docs/extension/sqlalchemy.rst index b92ade6..d37ca68 100644 --- a/docs/extension/sqlalchemy.rst +++ b/docs/extension/sqlalchemy.rst @@ -36,6 +36,25 @@ The `sqlalchemy.engine` name is the default name used by the provided transforma example if you need more than one connection) and specify the service name using `engine='myengine'` while building your transformations. +Lets create some tables and add some data. (You may need to edit the SQL if your database server uses a different +version of SQL.) + +.. code-block:: sql + + CREATE TABLE test_in ( + id INTEGER PRIMARY KEY NOT NULL, + text TEXT + ); + + CREATE TABLE test_out ( + id INTEGER PRIMARY KEY NOT NULL, + text TEXT + ); + + INSERT INTO test_in (id, text) VALUES (1, 'Cat'); + INSERT INTO test_in (id, text) VALUES (2, 'Dog'); + + There are two transformation classes provided by this extension. One reader, one writer. @@ -50,12 +69,29 @@ Let's select some data: def get_graph(): graph = bonobo.Graph() graph.add_chain( - bonobo_sqlalchemy.Select('SELECT * FROM example', limit=100), + bonobo_sqlalchemy.Select('SELECT * FROM test_in', limit=100), bonobo.PrettyPrinter(), ) + return graph -And let's insert some data: +You should see: +.. code-block:: shell-session + + $ python tutorial.py + ┌ + │ id[0] = 1 + │ text[1] = 'Cat' + └ + ┌ + │ id[0] = 2 + │ text[1] = 'Dog' + └ + - Select in=1 out=2 [done] + - PrettyPrinter in=2 out=2 [done] + + +Now let's insert some data: .. code-block:: python @@ -66,12 +102,14 @@ And let's insert some data: def get_graph(**options): graph = bonobo.Graph() graph.add_chain( - ..., - bonobo_sqlalchemy.InsertOrUpdate('example') + bonobo_sqlalchemy.Select('SELECT * FROM test_in', limit=100), + bonobo_sqlalchemy.InsertOrUpdate('test_out') ) return graph +If you check the `test_out` table, it should now have the data. + Reference ::::::::: diff --git a/docs/tutorial/3-files.rst b/docs/tutorial/3-files.rst index 7306726..f23faa7 100644 --- a/docs/tutorial/3-files.rst +++ b/docs/tutorial/3-files.rst @@ -41,7 +41,7 @@ Now, we need to write a `writer` transformation, and apply this context processo @use_context_processor(with_opened_file) def write_repr_to_file(f, *row): - f.write(repr(row)) + f.write(repr(row) + "\n") The `f` parameter will contain the value yielded by the context processors, in order of appearance (you can chain multiple context processors). @@ -50,6 +50,20 @@ Please note that the :func:`bonobo.config.use_context_processor` decorator will modify its behaviour. If you want to call it out of the |bonobo| job context, it's your responsibility to provide the right parameters (and here, the opened file). +To run this, change the last stage in the pipeline in get_graph to write_repr_to_file + +.. code-block:: python + + def get_graph(**options): + graph = bonobo.Graph() + graph.add_chain( + extract_fablabs, + bonobo.Limit(10), + write_repr_to_file, + ) + return graph + +Now run tutorial.py and check the output.txt file. Using the filesystem :::::::::::::::::::: diff --git a/docs/tutorial/4-services.rst b/docs/tutorial/4-services.rst index 967fa51..51b8683 100644 --- a/docs/tutorial/4-services.rst +++ b/docs/tutorial/4-services.rst @@ -3,7 +3,7 @@ Part 4: Services All external dependencies (like filesystems, network clients, database connections, etc.) should be provided to transformations as a service. It allows great flexibility, including the ability to test your transformations isolated -from the external world, and being friendly to the infrastructure guys (and if you're one of them, it's also nice to +from the external world, and being friendly to the infrastructure people (and if you're one of them, it's also nice to treat yourself well). In the last section, we used the `fs` service to access filesystems, we'll go even further by switching our `requests`