paulin/bonobo
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/master' into develop

Browse Source
This commit is contained in:
Romain Dorgueil
2017-07-15 12:18:20 +02:00
parent f2a9a45fd1 ba1ab72493
commit 0a640bf2cd
2 changed files with 15 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/tutorial/tut02.rst
View File

@ -59,7 +59,7 @@ available in **Bonobo**'s repository:
.. code-block:: shell-session .. code-block:: shell-session
$ curl https://raw.githubusercontent.com/python-bonobo/bonobo/master/bonobo/examples/datasets/coffeeshops.txt > `python -c 'import bonobo; print(bonobo.get_examples_path("datasets/coffeeshops.txt"))'` $ curl https://raw.githubusercontent.com/python-bonobo/bonobo/master/bonobo/examples/datasets/coffeeshops.txt > `python3 -c 'import bonobo; print(bonobo.get_examples_path("datasets/coffeeshops.txt"))'`
.. note:: .. note::

16
docs/tutorial/tut04.rst
View File

@ -18,8 +18,8 @@ specialized packages, like SQLAlchemy, or other database access libraries from t
First, read https://www.bonobo-project.org/with/sqlalchemy for instructions on how to install. You **do need** the First, read https://www.bonobo-project.org/with/sqlalchemy for instructions on how to install. You **do need** the
bleeding edge version of `bonobo` and `bonobo-sqlalchemy` to make this work. bleeding edge version of `bonobo` and `bonobo-sqlalchemy` to make this work.
Additional requirements Requirements
::::::::::::::::::::::: ::::::::::::
Once you installed `bonobo_sqlalchemy` (read https://www.bonobo-project.org/with/sqlalchemy to use bleeding edge Once you installed `bonobo_sqlalchemy` (read https://www.bonobo-project.org/with/sqlalchemy to use bleeding edge
version), install the following additional packages: version), install the following additional packages:
@ -62,6 +62,9 @@ file and add values for one or more of `POSTGRES_NAME`, `POSTGRES_USER`, 'POSTGR
`POSTGRES_PORT`. Please note that kwargs always have precedence on environment, but that you should prefer using `POSTGRES_PORT`. Please note that kwargs always have precedence on environment, but that you should prefer using
environment variables for anything that is not immutable from one platform to another. environment variables for anything that is not immutable from one platform to another.
Add database operation to the graph
:::::::::::::::::::::::::::::::::::
Let's create a `tutorial/pgdb.py` job: Let's create a `tutorial/pgdb.py` job:
.. code-block:: python .. code-block:: python
@ -110,6 +113,9 @@ If we run this transformation (with `bonobo run tutorial/pgdb.py`), we should ge
The database we requested do not exist. It is not the role of bonobo to do database administration, and thus there is The database we requested do not exist. It is not the role of bonobo to do database administration, and thus there is
no tool here to create neither the database, nor the tables we want to use. no tool here to create neither the database, nor the tables we want to use.
Create database and table
:::::::::::::::::::::::::
There are however tools in `sqlalchemy` to manage tables, so we'll create the database by ourselves, and ask sqlalchemy There are however tools in `sqlalchemy` to manage tables, so we'll create the database by ourselves, and ask sqlalchemy
to create the table: to create the table:
@ -170,6 +176,9 @@ Now run:
Database and table should now exist. Database and table should now exist.
Format the data
:::::::::::::::
Let's prepare our data for database, and change the `.add_chain(..)` call to do it prior to `InsertOrUpdate(...)` Let's prepare our data for database, and change the `.add_chain(..)` call to do it prior to `InsertOrUpdate(...)`
.. code-block:: python .. code-block:: python
@ -193,6 +202,9 @@ Let's prepare our data for database, and change the `.add_chain(..)` call to do
_input=split_one_to_map _input=split_one_to_map
) )
Run!
::::
You can now run the script (either with `bonobo run tutorial/pgdb.py` or directly with the python interpreter, as we You can now run the script (either with `bonobo run tutorial/pgdb.py` or directly with the python interpreter, as we
added a "main" section) and the dataset should be inserted in your database. If you run it again, no new rows are added a "main" section) and the dataset should be inserted in your database. If you run it again, no new rows are
created. created.