Update stable doc with warnings.

This commit is contained in:
Romain Dorgueil
2018-01-08 08:02:38 +01:00
parent 93195aa79f
commit e93ea8a803
16 changed files with 148 additions and 20 deletions

View File

@ -0,0 +1,9 @@
.. warning::
This tutorial was written for |bonobo| 0.5, while the current stable version is |bonobo| 0.6.
Please be aware that some things changed.
A summary of changes is available in the `migration guide from 0.5 to 0.6 <https://news.bonobo-project.org/migration-guide-for-bonobo-0-6-alpha-c1d36b0a9d35>`_.

View File

@ -0,0 +1,65 @@
First steps
===========
.. include:: _outdated_note.rst
What is Bonobo?
:::::::::::::::
Bonobo is an ETL (Extract-Transform-Load) framework for python 3.5. The goal is to define data-transformations, with
python code in charge of handling similar shaped independent lines of data.
Bonobo *is not* a statistical or data-science tool. If you're looking for a data-analysis tool in python, use Pandas.
Bonobo is a lean manufacturing assembly line for data that let you focus on the actual work instead of the plumbery
(execution contexts, parallelism, error handling, console output, logging, ...).
Bonobo uses simple python and should be quick and easy to learn.
Tutorial
::::::::
.. note::
Good documentation is not easy to write. We do our best to make it better and better.
Although all content here should be accurate, you may feel a lack of completeness, for which we plead guilty and
apologize.
If you're stuck, please come and ask on our `slack channel <https://bonobo-slack.herokuapp.com/>`_, we'll figure
something out.
If you're not stuck but had trouble understanding something, please consider contributing to the docs (via GitHub
pull requests).
.. toctree::
:maxdepth: 2
tut01
tut02
tut03
tut04
What's next?
::::::::::::
Read a few examples
-------------------
* :doc:`/reference/examples`
Read about best development practices
-------------------------------------
* :doc:`/guide/index`
* :doc:`/guide/purity`
Read about integrating external tools with bonobo
-------------------------------------------------
* :doc:`/extension/docker`: run transformation graphs in isolated containers.
* :doc:`/extension/jupyter`: run transformations within jupyter notebooks.
* :doc:`/extension/selenium`: crawl the web using a real browser and work with the gathered data.
* :doc:`/extension/sqlalchemy`: everything you need to interract with SQL databases.

View File

@ -0,0 +1,13 @@
Just enough Python for Bonobo
=============================
.. include:: _outdated_note.rst
.. todo::
This is a work in progress and it is not yet available. Please come back later or even better, help us write this
guide!
This guide is intended to help programmers or enthusiasts to grasp the python basics necessary to use Bonobo. It
should definately not be considered as a general python introduction, neither a deep dive into details.

View File

@ -1,7 +1,10 @@
Let's get started!
==================
To get started with Bonobo, you need to install it in a working python 3.5+ environment:
.. include:: _outdated_note.rst
To begin with Bonobo, you need to install it in a working python 3.5+ environment, and you'll also need cookiecutter
to bootstrap your project.
.. code-block:: shell-session
@ -13,24 +16,21 @@ See :doc:`/install` for more options.
Create an empty project
:::::::::::::::::::::::
Your ETL code will live in standard python files and packages.
Your ETL code will live in ETL projects, which are basically a bunch of files, including python code, that bonobo
can run.
.. code-block:: shell-session
$ bonobo create tutorial.py
$ bonobo init tutorial
This will create a simple example job in a `tutorial.py` file.
This will create a `tutorial` directory (`content description here <https://www.bonobo-project.org/with/cookiecutter>`_).
Now, try to execute it:
To run this project, use:
.. code-block:: shell-session
$ python tutorial.py
$ bonobo run tutorial
Congratulations, you just ran your first ETL job!
.. todo:: XXX **CHANGES NEEDED BELOW THIS POINTS BEFORE 0.6** XXX
Write a first transformation
::::::::::::::::::::::::::::
@ -107,9 +107,6 @@ To do this, it needs to know what data-flow you want to achieve, and you'll use
The `if __name__ == '__main__':` section is not required, unless you want to run it directly using the python
interpreter.
The name of the `graph` variable is arbitrary, but this variable must be global and available unconditionally.
Do not put it in its own function or in the `if __name__ == '__main__':` section.
Execute the job
:::::::::::::::
@ -133,9 +130,9 @@ Rewrite it using builtins
There is a much simpler way to describe an equivalent graph:
.. literalinclude:: ../../bonobo/examples/tutorials/tut01e02.py
:language: python
:language: python
The `extract()` generator has been replaced by a list, as Bonobo will interpret non-callable iterables as a no-input
The `extract()` generator has been replaced by a list, as Bonobo will interpret non-callable iterables as a no-input
generator.
This example is also available in :mod:`bonobo.examples.tutorials.tut01e02`, and you can also run it as a module:

View File

@ -1,6 +1,8 @@
Working with files
==================
.. include:: _outdated_note.rst
Bonobo would be pointless if the aim was just to uppercase small lists of strings.
In fact, Bonobo should not be used if you don't expect any gain from parallelization/distribution of tasks.
@ -59,7 +61,13 @@ available in **Bonobo**'s repository:
.. code-block:: shell-session
$ bonobo download examples/datasets/coffeeshops.txt
$ curl https://raw.githubusercontent.com/python-bonobo/bonobo/master/bonobo/examples/datasets/coffeeshops.txt > `python3 -c 'import bonobo; print(bonobo.get_examples_path("datasets/coffeeshops.txt"))'`
.. note::
The "example dataset download" step will be easier in the future.
https://github.com/python-bonobo/bonobo/issues/134
.. literalinclude:: ../../bonobo/examples/tutorials/tut02e01_read.py
:language: python

View File

@ -1,6 +1,8 @@
Configurables and Services
==========================
.. include:: _outdated_note.rst
.. note::
This section lacks completeness, sorry for that (but you can still read it!).
@ -30,7 +32,7 @@ Configurables allows to use the following features:
class PrefixIt(Configurable):
prefix = Option(str, positional=True, default='>>>')
def __call__(self, row):
def call(self, row):
return self.prefix + ' ' + row
prefixer = PrefixIt('$')
@ -48,7 +50,7 @@ Configurables allows to use the following features:
url = Option(default='https://jsonplaceholder.typicode.com/users')
http = Service('http.client')
def __call__(self, http):
def call(self, http):
resp = http.get(self.url)
for row in resp.json():
@ -68,7 +70,7 @@ Configurables allows to use the following features:
class Applier(Configurable):
apply = Method()
def __call__(self, row):
def call(self, row):
return self.apply(row)
@Applier
@ -114,7 +116,7 @@ Let's see how to use it, starting from the previous service example:
url = Option(default='https://jsonplaceholder.typicode.com/users')
http = Service('http.client')
def __call__(self, http):
def call(self, http):
resp = http.get(self.url)
for row in resp.json():

View File

@ -1,6 +1,8 @@
Working with databases
======================
.. include:: _outdated_note.rst
Databases (and especially SQL databases here) are not the focus of Bonobo, thus support for it is not (and will never
be) included in the main package. Instead, working with databases is done using third party, well maintained and
specialized packages, like SQLAlchemy, or other database access libraries from the python cheese shop.

View File

@ -1,6 +1,8 @@
Part 2: Writing ETL Jobs
========================
.. include:: _wip_note.rst
What's an ETL job ?
:::::::::::::::::::

View File

@ -1,6 +1,8 @@
Part 3: Working with Files
==========================
.. include:: _wip_note.rst
* Filesystems
* Reading files

View File

@ -1,6 +1,7 @@
Part 4: Services and Configurables
==================================
.. include:: _wip_note.rst
In the last section, we used a few new tools.

View File

@ -1,6 +1,8 @@
Part 5: Projects and Packaging
==============================
.. include:: _wip_note.rst
Until then, we worked with one file managing a job.
Real life often involves more complicated setups, with relations and imports between different files.

View File

@ -0,0 +1,12 @@
.. warning::
This section is being rewritten for |bonobo| 0.6, and it's now in a "work in progress" state.
You can read :doc:`the tutorial for the previous version (0.5) <0.5/index>`. Please note that things changed a bit
since then and you'll have quirks here and there.
You can also read the `migration guide from 0.5 to 0.6 <https://news.bonobo-project.org/migration-guide-for-bonobo-0-6-alpha-c1d36b0a9d35>`_
that will give you a good overview of the changes.
Hopefully, this document will be updated soon, and please accept our apologies about this doc status until then.

View File

@ -1,3 +1,8 @@
Working with Django
===================
.. warning::
This section does not exist yet, but it's in the plans to write it quite soon. Meanwhile, you can check the source
code and other links provided below.

View File

@ -53,3 +53,4 @@ out.
If you're not stuck but had trouble understanding something, please consider contributing to the docs (using GitHub
pull requests).
.. include:: _wip_note.rst

View File

@ -1,4 +1,8 @@
Working with Jupyter Notebooks
==============================
.. warning::
This section does not exist yet, but it's in the plans to write it quite soon. Meanwhile, you can check the source
code and other links provided below.

View File

@ -1,4 +1,7 @@
Working with SQL Databases
==========================
.. warning::
This section does not exist yet, but it's in the plans to write it quite soon. Meanwhile, you can check the source
code and other links provided below.