Commit Graph

17 Commits

Author SHA1 Message Date
5e0b6567cd Rewritting Bags from scratch using a namedtuple approach, along with other (less major) updates.
New bag implementation improves a lot how bonobo works, even if this is
highly backward incompatible (sorry, that's needed, and better sooner
than later).

* New implementation uses the same approach as python's namedtuple,
  by dynamically creating the python type's code. This has drawbacks, as
  it feels like not the right way, but also a lot of benefits that
  cannot be achieved using a regular approach, especially the
  constructor parameter order, hardcoded.
* Memory usage is now much more efficient. The "keys" memory space will
  be used only once per "io type", being spent in the underlying type
  definition instead of in the actual instances.
* Transformations now needs to use tuples as output, which will be bound
  to its "output type". The output type can be infered from the tuple
  length, or explicitely set by the user using either
  `context.set_output_type(...)` or `context.set_output_fields(...)` (to
  build a bag type from a list of field names).

Jupyter/Graphviz integration is more tight, allowing to easily display
graphs in a notebook, or displaying the live transformation status in an
html table instead of a simple <div>.

For now, context processors were hacked to stay working as before but
the current API is not satisfactory, and should be replaced. This new
big change being unreasonable without some time to work on it properly,
it is postponed for next versions (0.7, 0.8, ...). Maybe the best idea
is to have some kind of "local services", that would use the same
dependency injection mechanism as the execution-wide services.

Services are now passed by keywoerd arguments only, to avoid confusion
with data-arguments.
2017-11-27 00:04:51 +01:00
c2f17296f6 [stdlib] Adds Update(...) and FixedWindow(...) the the standard nodes provided with bonobo. 2017-11-12 10:06:15 +01:00
9a54f7b4aa [core] Still refactoring the core behaviour of bags, starting to be much simpler. 2017-10-23 21:18:02 +02:00
3c453f0be7 Code formating. 2017-10-22 18:05:39 +02:00
28fe41c0bd [core] Testing and fixing new args/kwargs behaviour. 2017-10-22 18:00:16 +02:00
c7f39aa851 release: 0.5.1 2017-10-21 12:49:34 +02:00
5d59a72310 [core] Adds a .copy() method to graph structure. 2017-07-06 11:29:55 +02:00
4d9b579a60 Tuning ValueHolder as I could not find better option to generate the double-underscore methods. 2017-05-21 19:22:45 +02:00
2b3ef05fac Removes unused functions and test bag application to iterables. 2017-05-20 14:47:30 +02:00
4342035f12 Refactoring: dispatch "core" package modules into structs and utils, as it does not have a lot of sense anymore. 2017-05-20 13:11:37 +02:00
d5cfa0281d Topological sort is non deterministic, only check for logic in test and not exact result so it will pass tests under py3.5 2017-05-19 18:38:41 +02:00
e747bc1da8 Topological sort of a graph, allowing better console (and other) outputs.
Uses algorithm borrowed from networkx graph library to sort a graph in
topological order. The method is only used by output plugins, as
internal plumbery does not really care about the node order.

Also includes a bonobo.util.python.require function that helps importing
thing in a package-less context, or when there are conflict with site
package names.
2017-05-19 13:28:31 +02:00
171fa3415b first draft of enhancers. 2017-05-08 11:33:02 +02:00
92fa1fb7e2 mock module is in unittest (py3) 2017-05-01 10:43:13 +02:00
eacd0cc8c5 wip: filesystem as a service. 2017-04-28 06:21:26 +02:00
3ecb2c2ccd Minor stuff, cleanup and formating. 2017-04-25 07:47:51 +02:00
ef2c881075 Attempt to refactor a bit of context, new count transform that counts the calls, new bonobo.structs package with basic data structures, removal of blessings. 2017-04-24 23:44:29 +02:00