293 lines
11 KiB
ReStructuredText
293 lines
11 KiB
ReStructuredText
Changelog
|
|
=========
|
|
|
|
v.0.5.0 - 5 october 2017
|
|
::::::::::::::::::::::::
|
|
|
|
Important highlights
|
|
--------------------
|
|
|
|
* `bonobo.pprint` and `bonobo.PrettyPrint` have been removed, in favor of `bonobo.PrettyPrinter` (BC break).
|
|
* The `bonobo.config` API has suffered a major refactoring. It has been done carefully and most of your code should
|
|
work unchanged, but you may have surprises. This was necessary for this API to be more uniform (potential BC break).
|
|
* bonobo.pprint and bonobo.PrettyPrint have been removed, in favor of new bonobo.PrettyPrinter() generic printer. If
|
|
you're still using the old versions, time to switch (BC break).
|
|
* Secondary APIs start to be more uniform (bonobo.config, bonobo.util).
|
|
|
|
New features
|
|
------------
|
|
|
|
Graphs & Nodes
|
|
..............
|
|
|
|
* Graphs now have a .copy() method.
|
|
* New helper transformations arg0_to_kwargs and kwargs_to_arg0.
|
|
* The unique pretty printer provided by the core library is now bonobo.PrettyPrinter().
|
|
* Services now have "fs" and "http" configured by default.
|
|
|
|
Command line
|
|
............
|
|
|
|
* New `bonobo convert` command now allows to run simple conversion jobs without coding anything.
|
|
* New `bonobo inspect` command now allows to generate graphviz source for graph visualization.
|
|
* Passing environment variables to graph executions now can be done using -e/--env. (cwandrews)
|
|
* Add ability to install requirements with for a requirements.txt residing in the same dir (Alex Vykaliuk)
|
|
|
|
Preview
|
|
.......
|
|
|
|
* A "transformation factory" makes its first appearance. It is considered a preview unstable feature. Stay
|
|
tuned.
|
|
|
|
Internals
|
|
---------
|
|
|
|
* Configurables have undergone a refactoring, all types of descriptors should now behave in the same way.
|
|
* An UnrecoverrableError exception subclass allows for some errors to stop the whole execution.
|
|
* Refactoring of Settings (bonobo.settings).
|
|
* Add a reference to graph context (private) in service container.
|
|
* Few internal APIs changes and refactorings.
|
|
|
|
Bugfixes
|
|
--------
|
|
|
|
* Check if PluginExecutionContext was started before shutting it down. (Vitalii Vokhmin)
|
|
* Move patch one level up because importlib brakes all the CI tools. (Alex Vykaliuk)
|
|
* Do not fail in ipykernel without ipywidgets. (Alex Vykaliuk)
|
|
* Escaping issues (Tomas Zubiri)
|
|
|
|
Miscellaneous
|
|
-------------
|
|
|
|
* Windows console output should now be correct. (Parthiv20)
|
|
* Various bugfixes.
|
|
* More readable statistics on Ubuntu workstation standard terminal (spagoc)
|
|
* Documentation, more documentation, documentation again.
|
|
|
|
|
|
v.0.4.3 - 16 july 2017
|
|
::::::::::::::::::::::
|
|
|
|
* #113 - Add flush() method to IOBuffer (Vitalii Vokhmin)
|
|
* Dependencies updated.
|
|
* Minor project artifacts updated.
|
|
|
|
v.0.4.2 - 18 june 2017
|
|
::::::::::::::::::::::
|
|
|
|
* [config] Implements a "requires()" service injection decorator for functions (api may change).
|
|
* [core] Execution contexts are now context managers.
|
|
* [fs] adds a defaut to current working directory in open_fs(...).
|
|
* [logging] Adds logging alias for easier imports.
|
|
* [stdlib] Fix I/O related nodes (especially json), there were bad bugs with ioformat.
|
|
|
|
Dependency updates
|
|
------------------
|
|
|
|
* Update bonobo-docker from 0.2.6 to 0.2.8
|
|
* Update dependencies.
|
|
* Update fs from 2.0.3 to 2.0.4
|
|
* Update requests from 2.17.3 to 2.18.1
|
|
|
|
v.0.4.0 - 10 june 2017
|
|
::::::::::::::::::::::
|
|
|
|
Important highlights
|
|
--------------------
|
|
|
|
* **BC BREAK WARNING** New IOFORMAT option determines the default expected input and output format of transformations.
|
|
New default input/output format of transformations is now kwargs-based, instead of first-argument based. The
|
|
rationale behind this is that it does not make any sense to put a dict as the only argument of a transformation
|
|
knowing that python has a well supported syntax to do so already. Of course, it may break some of your
|
|
transformations but you can require the old behaviour by setting the IOFORMAT=arg0 environment variable.
|
|
|
|
New features
|
|
------------
|
|
|
|
Command line interface
|
|
......................
|
|
|
|
* Allow to run directories or modules using "bonobo run".
|
|
* Bonobo version command now shows where the package is installed, and an optional "--all/-a" flag show all
|
|
extensions in the same way. (#81)
|
|
* Bonobo run flag "--install/-I" allow to pip install a requirements.txt file if run targets a directory. (#71)
|
|
* Adds python logging facility configuration in bonobo cli commands.
|
|
* Bonobo init now uses cookiecutter template.
|
|
|
|
Configuration
|
|
.............
|
|
|
|
* `Exclusive(...)` context manager locks an object usage to one thread at a time.
|
|
([docs](http://docs-dev.bonobo-project.org/en/develop/guide/services.html#solving-concurrency-problems))
|
|
|
|
Standard library
|
|
................
|
|
|
|
* New PrettyPrinter and deprecate old crappy modules.
|
|
* New pickle reader and writer (thanks @jelloslinger).
|
|
|
|
Internals
|
|
---------
|
|
|
|
* ConsoleOutputPlugin now buffers stdout to avoid terminal conflicts. Side effect, output is only done every few tenth
|
|
of a second.
|
|
|
|
Bugfixes
|
|
--------
|
|
|
|
* Fixes jupyter widget.
|
|
|
|
Extensions
|
|
----------
|
|
|
|
* First release officially supporting bonobo-docker extension. See https://www.bonobo-project.org/with/docker.
|
|
* Docker extension can be now installed using the "docker" extra on bonobo (`pip install bonobo[docker]`).
|
|
* Jupyter widget now displays the status in topological order, like console.
|
|
|
|
Miscellaneous
|
|
-------------
|
|
|
|
* Allow "main.py" as well as "__main__.py" to be the main entrypoint of an etl job.
|
|
* Better error display (329296c).
|
|
* Better testing.
|
|
* Code sweeping (ecfdc81).
|
|
* Dependencies updated.
|
|
* Filesystem now resolve (expand) ~ in path.
|
|
* Moving project artifact management (Projectfile) to edgy.project 0.3 format.
|
|
* Refactoring and fixes around ioformats.
|
|
* Some really minor changes.
|
|
|
|
v.0.3.2 - 10 june 2017
|
|
::::::::::::::::::::::
|
|
|
|
Weekly maintenance release.
|
|
|
|
* Updated frozen version numbers in requirements.
|
|
|
|
* pytest==3.1.1
|
|
* requests==2.17.3
|
|
* sphinx==1.6.2
|
|
* stevedore==1.22.0
|
|
|
|
Note: this does not change anything when used as a dependency if you freeze your requirements, as the setup.py
|
|
requirement specifiers did not change.
|
|
|
|
v.0.3.1 - 28 may 2017
|
|
:::::::::::::::::::::
|
|
|
|
Weekly maintenance release.
|
|
|
|
* Updated project management model to edgy.project 0.3 format.
|
|
* Updated frozen version numbers in requirements.
|
|
|
|
* certifi==2017.4.17
|
|
* chardet==3.0.3
|
|
* coverage==4.4.1
|
|
* idna==2.5
|
|
* nbconvert==5.2.1
|
|
* pbr==3.0.1
|
|
* pytest-cov==2.5.1
|
|
* pytest==3.1.0
|
|
* requests==2.16.5
|
|
* sphinx==1.6.1
|
|
* sphinxcontrib-websupport==1.0.1
|
|
* testpath==0.3.1
|
|
* typing==3.6.1
|
|
* urllib3==1.21.1
|
|
|
|
Note: this does not change anything when used as a dependency if you freeze your requirements, as the setup.py
|
|
requirement specifiers did not change.
|
|
|
|
v.0.3.0 - 22 may 2017
|
|
:::::::::::::::::::::
|
|
|
|
Features
|
|
--------
|
|
|
|
* ContextProcessors can now be implemented by getting the "yield" value (v = yield x), shortening the teardown-only
|
|
context processors by one line.
|
|
* File related writers (file, csv, json ...) now returns NOT_MODIFIED, making it easier to chain something after.
|
|
* More consistent console output, nodes are now sorted in a topological order before display.
|
|
* Graph.add_chain(...) now takes _input and _output parameters the same way, accepting indexes, instances or names
|
|
(subject to change).
|
|
* Graph.add_chain(...) now allows to "name" a chain, using _name keyword argument, to easily reference its output later
|
|
(subject to change).
|
|
* New settings module (bonobo.settings) read environment for some global configuration stuff (DEBUG and PROFILE, for
|
|
now).
|
|
* New Method subclass of Option allows to use Configurable objects as decorator (see bonobo.nodes.filter.Filter for a
|
|
simple example).
|
|
* New Filter transformation in standard library.
|
|
|
|
Internal features
|
|
-----------------
|
|
|
|
* Better ContextProcessor implementation, avoiding to use a decorator on the parent class. Now works with Configurable
|
|
instances like Option, Service and Method.
|
|
* ContextCurrifier replaces the logic that was in NodeExecutionContext, that setup and teardown the context stack. Maybe
|
|
the name is not ideal.
|
|
* All builtin transformations are of course updated to use the improved API, and should be 100% backward compatible.
|
|
* The "core" package has been dismantled, and its rare remaining members are now in "structs" and "util" packages.
|
|
* Standard transformation library has been moved under the bonobo.nodes package. It does not change anything if you used
|
|
bonobo.* (which you should).
|
|
* ValueHolder is now more restrictive, not allowing to use .value anymore.
|
|
|
|
Miscellaneous
|
|
-------------
|
|
|
|
* Code cleanup, dead code removal, more tests, etc.
|
|
* More documentation.
|
|
|
|
v.0.2.4 - 2 may 2017
|
|
::::::::::::::::::::
|
|
|
|
* Cosmetic release for PyPI package page formating. Same content as v.0.2.3.
|
|
|
|
v.0.2.3 - 1 may 2017
|
|
:::::::::::::::::::::
|
|
|
|
* Positional options now supported, backward compatible. All FileHandler subclasses supports their path argument as
|
|
positional.
|
|
* Better transformation lifecycle management (still work needed here).
|
|
* Windows continuous integration now works.
|
|
* Refactoring the "API" a lot to have a much cleaner first glance at it.
|
|
* More documentation, tutorials, and tuning project artifacts.
|
|
|
|
v.0.2.2 - 28 apr 2017
|
|
:::::::::::::::::::::
|
|
|
|
* First implementation of services and basic injection.
|
|
* Default service configuration for directories and files.
|
|
* Code structure refactoring.
|
|
* Critical bug fix in default strategy causing end of pipeline not to terminate correctly.
|
|
* Force tighter dependency management to avoid unexpected upgrade problems.
|
|
* Filesystems are now injected as a service, using new filesystem2 (fs) dependency.
|
|
|
|
v.0.2.1 - 25 apr 2017
|
|
:::::::::::::::::::::
|
|
|
|
* Plugins (jupyter, console) are now auto-activated depending on the environment when using bonobo.run(...).
|
|
* Remove dependencies to toolz (which was unused) and blessings (which caused problems on windows).
|
|
* New dependency on colorama, which has better cross-platform support than blessings.
|
|
* New bonobo.structs package containing basic datastructures, like graphs, tokens and bags.
|
|
* Enhancements of ValueHolder to implement basic operators on its value without referencing the value attribute.
|
|
* Fix issue with timezone argument of OpenDataSoftAPI (Sanket Dasgupta).
|
|
* Fix Jupyter plugin.
|
|
* Better continuous integration, testing and fixes in documentation.
|
|
* Version updates for dependencies (psutil install problem on windows).
|
|
|
|
Initial release
|
|
:::::::::::::::
|
|
|
|
* Migration from rdc.etl.
|
|
* New cool name (ok, that's debatable).
|
|
* Only supports python 3.5+, aggressively (which means, we can use async, and we remove all things from python 2/six
|
|
compat)
|
|
* Removes all thing deprecated and/or not really convincing from rdc.etl.
|
|
* We want transforms to be simple callables, so refactoring of the harness mess.
|
|
* We want to use plain python data structures, so hashes are removed. If you use python 3.6, you may even get sorted
|
|
dicts.
|
|
* Input/output MUX DEMUX removed, maybe no need for that in the real world. May come back, but not in 1.0
|
|
* Change dependency policy. We need to include only the very basic requirements (and very required). Everything related
|
|
to transforms that we may not use (bs, sqla, ...) should be optional dependencies.
|
|
* Execution strategies, threaded by default.
|