Commit Graph

127 Commits

Author SHA1 Message Date
5e0b6567cd Rewritting Bags from scratch using a namedtuple approach, along with other (less major) updates.
New bag implementation improves a lot how bonobo works, even if this is
highly backward incompatible (sorry, that's needed, and better sooner
than later).

* New implementation uses the same approach as python's namedtuple,
  by dynamically creating the python type's code. This has drawbacks, as
  it feels like not the right way, but also a lot of benefits that
  cannot be achieved using a regular approach, especially the
  constructor parameter order, hardcoded.
* Memory usage is now much more efficient. The "keys" memory space will
  be used only once per "io type", being spent in the underlying type
  definition instead of in the actual instances.
* Transformations now needs to use tuples as output, which will be bound
  to its "output type". The output type can be infered from the tuple
  length, or explicitely set by the user using either
  `context.set_output_type(...)` or `context.set_output_fields(...)` (to
  build a bag type from a list of field names).

Jupyter/Graphviz integration is more tight, allowing to easily display
graphs in a notebook, or displaying the live transformation status in an
html table instead of a simple <div>.

For now, context processors were hacked to stay working as before but
the current API is not satisfactory, and should be replaced. This new
big change being unreasonable without some time to work on it properly,
it is postponed for next versions (0.7, 0.8, ...). Maybe the best idea
is to have some kind of "local services", that would use the same
dependency injection mechanism as the execution-wide services.

Services are now passed by keywoerd arguments only, to avoid confusion
with data-arguments.
2017-11-27 00:04:51 +01:00
d1481fbfe8 Moves jupyter extension to both bonobo.contrib.jupyter (for the jupyter widget) and to bonobo.plugins (for the executor-side plugin). 2017-11-12 09:08:05 +01:00
8f3c4252b4 Working on the new version of the tutorial. Only Step1 implemented. 2017-11-05 19:41:27 +01:00
e6596cf3f3 Removes cookiecutter. 2017-11-01 13:19:24 +01:00
69bb3cb091 Switch logger setup to mondrian (deps). 2017-11-01 13:12:19 +01:00
32e73316e1 Merge pull request #210 from mpenkov/streamjson
Support line-delimited JSON
2017-10-28 17:45:09 +02:00
10a82cce62 Support line-delimited JSON
New nodes for handling line-delimited JSON.

https://en.wikipedia.org/wiki/JSON_Streaming
2017-10-28 17:34:13 +02:00
ce0bd9ea37 Merge branch 'develop' of https://github.com/mpenkov/bonobo into mpenkov-develop 2017-10-28 17:27:04 +02:00
209c4f3176 Merge branch 'develop' of github.com:python-bonobo/bonobo into develop 2017-10-28 17:24:39 +02:00
edc2321c54 Issue #134: update documentation 2017-10-28 14:20:53 +02:00
7d9b547a53 Merge branch 'zimmermann/import' of github.com:userzimmermann/bonobo into develop 2017-10-28 14:16:08 +02:00
9e86abca60 Issue #134: add a bonobo download url command
This enables users on different platforms to download the examples in
the tutorial using the same command.
2017-10-28 14:08:53 +02:00
bf5f870304 Add a note about the graph variable
It needs to be global and publicly available.  Some people reading the
tutorial may want to put it in its own function or the __main__
conditional, which won't work.
2017-10-28 12:10:39 +02:00
3fdefcda72 Update sidebarinfos.html 2017-10-24 11:14:11 +02:00
321bb83aa4 Documentation for new behaviour. 2017-10-22 16:57:46 +02:00
650b49a41a [django, google] Implements basic extensions for django and google oauth systems.
Using those extensions means you have the correct dependencies installed, and that you know about the external system.

Django: just provide an ETLCommand class that contains all the shortcuts
to write django management commands based on Bonobo.

Google: shortcuts to create the necessary objects for oauth flow, with
local caching of credentials.

Both those extensions are not stable and will evolve.
2017-10-22 16:26:36 +02:00
7560412bc7 Merge branch 'develop' of github.com:python-bonobo/bonobo into develop 2017-10-21 12:56:06 +02:00
c7f39aa851 release: 0.5.1 2017-10-21 12:49:34 +02:00
12e4250166 Merge pull request #195 from cw-andrews/feature/env_file
Feature/env file
2017-10-17 13:55:48 +02:00
7c8625dda6 Update conf.py 2017-10-16 16:59:47 +02:00
5d41f6df2d Update environment.rst
Updated examples to use preferred naming conventions for default and private/local .env files per request of @hartym.
2017-10-15 17:15:30 -04:00
945d850139 Update environment.rst
Fixed typo (I think).
2017-10-15 17:11:10 -04:00
d6d063ad43 Updated environment documentation in guides to account for env files. 2017-10-15 16:43:28 -04:00
99351a638b [doc] formating 2017-10-12 19:01:47 +02:00
2532dd1b89 [doc] fix typos and errors. 2017-10-12 11:41:23 -05:00
8174257bd0 Fighting with google analytics properties linked to multiple websites. 2017-10-12 08:20:59 +02:00
14a61305bc Update base.html 2017-10-10 08:47:43 +02:00
0ca766a7b9 Update base.html 2017-10-10 08:39:09 +02:00
0c310543e4 Update base.html 2017-10-10 08:30:03 +02:00
8bef85704f Update faq.rst
Spelling, grammar, and wording changes.
2017-10-08 13:54:31 -04:00
3b20a40bb7 [doc] Missing description. 2017-10-08 17:42:48 +02:00
f40de8b386 [doc] adds alabaster source 2017-10-08 13:22:21 +02:00
7fa9a2be5b [doc] Updating guides in documentation 2017-10-08 13:13:20 +02:00
32e222787d release: 0.5.0 2017-10-05 19:16:03 +02:00
f7cf7ca482 [doc] homepage 2017-10-04 19:14:36 +02:00
c337aad14d [doc] fix typos and errors. 2017-10-04 08:40:08 +02:00
75a68d1322 [doc] fix links 2017-10-03 08:41:03 +02:00
d936e164ac [doc] cleanup & refactorings 2017-10-03 08:37:46 +02:00
2ab48080e6 [doc] refactors guides index so the toc is showing up correctly in sidebar. 2017-10-03 08:25:15 +02:00
e9dfcfe0d8 [doc] adds cross ref in env vars page 2017-10-03 08:15:50 +02:00
0bfa873743 [doc] fix typo 2017-10-03 08:13:17 +02:00
bfa43e65e6 [doc] tuning a few things in documentation. 2017-10-03 08:09:50 +02:00
a1e381fad5 [doc] fix spaces in links. 2017-10-03 07:57:30 +02:00
142a6d60df Updated inline examples and fixed code markup. 2017-10-02 16:11:41 -04:00
504e8920f2 Update and rename environmental_variables.rst to environment_variables.rst
Made review requests per @hardym.
2017-10-02 16:06:04 -04:00
56d8f32910 Update environmental_variables.rst 2017-10-02 15:34:00 -04:00
5f83aef47d Update jupyter.rst 2017-10-02 15:25:13 -04:00
b83ba99beb Update index.rst 2017-10-02 15:24:41 -04:00
074d173ea7 Added envrionmental_variables page to the guides section. 2017-10-01 21:26:10 -04:00
c1a5750b60 Made a few spelling and grammar corrections/adjustments to the docs under guide. 2017-10-01 20:16:24 -04:00