paulin/bonobo
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'develop' into transformation_factory

Browse Source
This commit is contained in:
Romain Dorgueil
2017-07-16 18:54:37 +02:00
parent f02d92c7e8 423a75d554
commit 25cb430b7b
49 changed files with 1219 additions and 502 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
CODE_OF_CONDUCT.md Normal file
View File

@ -0,0 +1,46 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at bonobo@rdc.li. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/

1
CONTRIBUTING.md Normal file
View File

@ -0,0 +1 @@
See http://docs.bonobo-project.org/en/latest/contribute/index.html

2
Makefile
View File

@ -1,7 +1,7 @@
# This file has been auto-generated.
# All changes will be lost, see Projectfile.
#
# Updated at 2017-07-04 10:50:55.775681
# Updated at 2017-07-16 10:52:31.093416
PACKAGE ?= bonobo
PYTHON ?= $(shell which python)

4
README.rst
View File

@ -55,6 +55,8 @@ Homepage: https://www.bonobo-project.org/ (`Roadmap <https://www.bonobo-project.
Documentation: http://docs.bonobo-project.org/
Contributing guide: http://docs.bonobo-project.org/en/latest/contribute/index.html
Issues: https://github.com/python-bonobo/bonobo/issues
Slack: https://bonobo-slack.herokuapp.com/
@ -64,7 +66,7 @@ Release announcements: http://eepurl.com/csHFKL
----
Made with ♥ by `Romain Dorgueil <https://twitter.com/rdorgueil>`_ and `contributors <https://github.com/python-bonobo/bonobo/graphs/contributors>`_.
.. image:: https://img.shields.io/pypi/l/bonobo.svg
:target: https://pypi.python.org/pypi/bonobo
:alt: License

38
bonobo/_api.py
View File

@ -1,6 +1,8 @@
import logging
from bonobo.structs import Bag, Graph, Token
from bonobo.nodes import CsvReader, CsvWriter, FileReader, FileWriter, Filter, JsonReader, JsonWriter, Limit, \
PrettyPrinter, PickleWriter, PickleReader, Tee, count, identity, noop, pprint
PickleReader, PickleWriter, PrettyPrinter, RateLimited, Tee, arg0_to_kwargs, count, identity, kwargs_to_arg0, noop
from bonobo.strategies import create_strategy
from bonobo.util.objects import get_name
@ -21,17 +23,17 @@ def register_api_group(*args):
def run(graph, strategy=None, plugins=None, services=None):
"""
Main entry point of bonobo. It takes a graph and creates all the necessary plumbery around to execute it.
The only necessary argument is a :class:`Graph` instance, containing the logic you actually want to execute.
By default, this graph will be executed using the "threadpool" strategy: each graph node will be wrapped in a
thread, and executed in a loop until there is no more input to this node.
You can provide plugins factory objects in the plugins list, this function will add the necessary plugins for
interactive console execution and jupyter notebook execution if it detects correctly that it runs in this context.
You'll probably want to provide a services dictionary mapping service names to service instances.
:param Graph graph: The :class:`Graph` to execute.
:param str strategy: The :class:`bonobo.strategies.base.Strategy` to use.
:param list plugins: The list of plugins to enhance execution.
@ -45,16 +47,24 @@ def run(graph, strategy=None, plugins=None, services=None):
from bonobo import settings
settings.check()
if not settings.QUIET: # pragma: no cover
if not settings.QUIET.get(): # pragma: no cover
if _is_interactive_console():
from bonobo.ext.console import ConsoleOutputPlugin
if ConsoleOutputPlugin not in plugins:
plugins.append(ConsoleOutputPlugin)
if _is_jupyter_notebook():
from bonobo.ext.jupyter import JupyterOutputPlugin
if JupyterOutputPlugin not in plugins:
plugins.append(JupyterOutputPlugin)
try:
from bonobo.ext.jupyter import JupyterOutputPlugin
except ImportError:
logging.warning(
'Failed to load jupyter widget. Easiest way is to install the optional "jupyter" '
'dependencies with «pip install bonobo[jupyter]», but you can also install a specific '
'version by yourself.'
)
else:
if JupyterOutputPlugin not in plugins:
plugins.append(JupyterOutputPlugin)
return strategy.execute(graph, plugins=plugins, services=services)
@ -71,7 +81,7 @@ register_api(create_strategy)
def open_fs(fs_url=None, *args, **kwargs):
"""
Wraps :func:`fs.open_fs` function with a few candies.
:param str fs_url: A filesystem URL
:param parse_result: A parsed filesystem URL.
:type parse_result: :class:`ParseResult`
@ -101,14 +111,16 @@ register_api_group(
JsonReader,
JsonWriter,
Limit,
PrettyPrinter,
PickleReader,
PickleWriter,
PrettyPrinter,
RateLimited,
Tee,
arg0_to_kwargs,
count,
identity,
kwargs_to_arg0,
noop,
pprint,
)

6
bonobo/commands/__init__.py
View File

@ -27,9 +27,9 @@ def entrypoint(args=None):
args = parser.parse_args(args).__dict__
if args.pop('debug', False):
settings.DEBUG = True
settings.LOGGING_LEVEL = logging.DEBUG
logging.set_level(settings.LOGGING_LEVEL)
settings.DEBUG.set(True)
settings.LOGGING_LEVEL.set(logging.DEBUG)
logging.set_level(settings.LOGGING_LEVEL.get())
logger.debug('Command: ' + args['command'] + ' Arguments: ' + repr(args))
commands[args.pop('command')](**args)

31
bonobo/commands/run.py
View File

@ -26,29 +26,35 @@ def get_default_services(filename, services=None):
return services or {}
def _install_requirements(requirements):
"""Install requirements given a path to requirements.txt file."""
import importlib
import pip
pip.main(['install', '-r', requirements])
# Some shenanigans to be sure everything is importable after this, especially .egg-link files which
# are referenced in *.pth files and apparently loaded by site.py at some magic bootstrap moment of the
# python interpreter.
pip.utils.pkg_resources = importlib.reload(pip.utils.pkg_resources)
import site
importlib.reload(site)
def execute(filename, module, install=False, quiet=False, verbose=False):
import runpy
from bonobo import Graph, run, settings
if quiet:
settings.QUIET = True
settings.QUIET.set(True)
if verbose:
settings.DEBUG = True
settings.DEBUG.set(True)
if filename:
if os.path.isdir(filename):
if install:
import importlib
import pip
requirements = os.path.join(filename, 'requirements.txt')
pip.main(['install', '-r', requirements])
# Some shenanigans to be sure everything is importable after this, especially .egg-link files which
# are referenced in *.pth files and apparently loaded by site.py at some magic bootstrap moment of the
# python interpreter.
pip.utils.pkg_resources = importlib.reload(pip.utils.pkg_resources)
import site
importlib.reload(site)
_install_requirements(requirements)
pathname = filename
for filename in DEFAULT_GRAPH_FILENAMES:
@ -58,7 +64,8 @@ def execute(filename, module, install=False, quiet=False, verbose=False):
if not os.path.exists(filename):
raise IOError('Could not find entrypoint (candidates: {}).'.format(', '.join(DEFAULT_GRAPH_FILENAMES)))
elif install:
raise RuntimeError('Cannot --install on a file (only available for dirs containing requirements.txt).')
requirements = os.path.join(os.path.dirname(filename), 'requirements.txt')
_install_requirements(requirements)
context = runpy.run_path(filename, run_name='__bonobo__')
elif module:
context = runpy.run_module(module, run_name='__bonobo__')

2
bonobo/config/__init__.py
View File

@ -3,7 +3,7 @@ from bonobo.config.options import Method, Option
from bonobo.config.processors import ContextProcessor
from bonobo.config.services import Container, Exclusive, Service, requires
# bonobo.config public programming interface
# Bonobo's Config API
__all__ = [
'Configurable',
'Container',

199
bonobo/config/configurables.py
View File

@ -1,12 +1,14 @@
from bonobo.config.options import Method, Option
from bonobo.config.processors import ContextProcessor
from bonobo.errors import ConfigurationError, AbstractError
from bonobo.util.inspect import isoption, iscontextprocessor
from bonobo.errors import AbstractError
from bonobo.util.collections import sortedlist
__all__ = [
'Configurable',
'Option',
]
get_creation_counter = lambda v: v._creation_counter
class ConfigurableMeta(type):
"""
@ -15,36 +17,78 @@ class ConfigurableMeta(type):
def __init__(cls, what, bases=None, dict=None):
super().__init__(what, bases, dict)
cls.__options__ = {}
cls.__positional_options__ = []
cls.__processors__ = []
cls.__wrappable__ = None
cls.__processors = sortedlist()
cls.__methods = sortedlist()
cls.__options = sortedlist()
cls.__names = set()
# cls.__kwoptions = []
for typ in cls.__mro__:
for name, value in typ.__dict__.items():
if isinstance(value, Option):
if isinstance(value, ContextProcessor):
cls.__processors__.append(value)
else:
if not value.name:
value.name = name
for name, value in filter(lambda x: isoption(x[1]), typ.__dict__.items()):
if iscontextprocessor(value):
cls.__processors.insort((value._creation_counter, value))
continue
if isinstance(value, Method):
if cls.__wrappable__:
raise ConfigurationError(
'Cannot define more than one "Method" option in a configurable. That may change in the future.'
)
cls.__wrappable__ = name
if not value.name:
value.name = name
if not name in cls.__options__:
cls.__options__[name] = value
if not name in cls.__names:
cls.__names.add(name)
cls.__options.insort((not value.positional, value._creation_counter, name, value))
if value.positional:
cls.__positional_options__.append(name)
@property
def __options__(cls):
return ((name, option) for _, _, name, option in cls.__options)
# This can be done before, more efficiently. Not so bad neither as this is only done at type() creation time
# (aka class Xxx(...) time) and there should not be hundreds of processors. Still not very elegant.
cls.__processors__ = sorted(cls.__processors__, key=lambda v: v._creation_counter)
@property
def __options_dict__(cls):
return dict(cls.__options__)
@property
def __processors__(cls):
return (processor for _, processor in cls.__processors)
def __repr__(self):
return ' '.join(('<Configurable', super(ConfigurableMeta, self).__repr__().split(' ', 1)[1], ))
try:
import _functools
except:
import functools
PartiallyConfigured = functools.partial
else:
class PartiallyConfigured(_functools.partial):
@property # TODO XXX cache this shit
def _options_values(self):
""" Simulate option values for partially configured objects. """
try:
return self.__options_values
except AttributeError:
self.__options_values = {**self.keywords}
position = 0
for name, option in self.func.__options__:
if not option.positional:
break # no positional left
if name in self.keywords:
continue # already fulfilled
self.__options_values[name] = self.args[position] if len(self.args) >= position + 1 else None
position += 1
return self.__options_values
def __getattr__(self, item):
_dict = self.func.__options_dict__
if item in _dict:
return _dict[item].__get__(self, self.func)
return getattr(self.func, item)
class Configurable(metaclass=ConfigurableMeta):
@ -54,61 +98,106 @@ class Configurable(metaclass=ConfigurableMeta):
"""
def __new__(cls, *args, **kwargs):
if cls.__wrappable__ and len(args) == 1 and hasattr(args[0], '__call__'):
return type(args[0].__name__, (cls, ), {cls.__wrappable__: args[0]})
def __new__(cls, *args, _final=False, **kwargs):
"""
Custom instance builder. If not all options are fulfilled, will return a :class:`PartiallyConfigured` instance
which is just a :class:`functools.partial` object that behaves like a :class:`Configurable` instance.
return super(Configurable, cls).__new__(cls)
def __init__(self, *args, **kwargs):
super().__init__()
# initialize option's value dictionary, used by descriptor implementation (see Option).
self.__options_values__ = {}
The special `_final` argument can be used to force final instance to be created, or an error raised if options
are missing.
:param args:
:param _final: bool
:param kwargs:
:return: Configurable or PartiallyConfigured
"""
options = tuple(cls.__options__)
# compute missing options, given the kwargs.
missing = set()
for name, option in type(self).__options__.items():
for name, option in options:
if option.required and not option.name in kwargs:
missing.add(name)
# transform positional arguments in keyword arguments if possible.
position = 0
for positional_option in self.__positional_options__:
if len(args) <= position:
break
kwargs[positional_option] = args[position]
position += 1
if positional_option in missing:
missing.remove(positional_option)
for name, option in options:
if not option.positional:
break # option orders make all positional options first, job done.
# complain if there are still missing options.
if len(missing):
raise TypeError(
'{}() missing {} required option{}: {}.'.format(
type(self).__name__,
len(missing), 's' if len(missing) > 1 else '', ', '.join(map(repr, sorted(missing)))
)
)
if not isoption(getattr(cls, name)):
missing.remove(name)
continue
if len(args) <= position:
break # no more positional arguments given.
position += 1
if name in missing:
missing.remove(name)
# complain if there is more options than possible.
extraneous = set(kwargs.keys()) - set(type(self).__options__.keys())
extraneous = set(kwargs.keys()) - (set(next(zip(*options))) if len(options) else set())
if len(extraneous):
raise TypeError(
'{}() got {} unexpected option{}: {}.'.format(
type(self).__name__,
cls.__name__,
len(extraneous), 's' if len(extraneous) > 1 else '', ', '.join(map(repr, sorted(extraneous)))
)
)
# missing options? we'll return a partial instance to finish the work later, unless we're required to be
# "final".
if len(missing):
if _final:
raise TypeError(
'{}() missing {} required option{}: {}.'.format(
cls.__name__,
len(missing), 's' if len(missing) > 1 else '', ', '.join(map(repr, sorted(missing)))
)
)
return PartiallyConfigured(cls, *args, **kwargs)
return super(Configurable, cls).__new__(cls)
def __init__(self, *args, **kwargs):
# initialize option's value dictionary, used by descriptor implementation (see Option).
self._options_values = {**kwargs}
# set option values.
for name, value in kwargs.items():
setattr(self, name, value)
position = 0
for name, option in self.__options__:
if not option.positional:
break # option orders make all positional options first
# value was overriden? Skip.
maybe_value = getattr(type(self), name)
if not isoption(maybe_value):
continue
if len(args) <= position:
break
if name in self._options_values:
raise ValueError('Already got a value for option {}'.format(name))
setattr(self, name, args[position])
position += 1
def __call__(self, *args, **kwargs):
""" You can implement a configurable callable behaviour by implemenenting the call(...) method. Of course, it is also backward compatible with legacy __call__ override.
"""
return self.call(*args, **kwargs)
@property
def __options__(self):
return type(self).__options__
@property
def __processors__(self):
return type(self).__processors__
def call(self, *args, **kwargs):
raise AbstractError('Not implemented.')

59
bonobo/config/options.py
View File

@ -1,3 +1,6 @@
from bonobo.util.inspect import istype
class Option:
"""
An Option is a descriptor for Configurable's parameters.
@ -14,7 +17,9 @@ class Option:
If an option is required, an error will be raised if no value is provided (at runtime). If it is not, option
will have the default value if user does not override it at runtime.
(default: False)
Ignored if a default is provided, meaning that the option cannot be required.
(default: True)
.. attribute:: positional
@ -48,10 +53,10 @@ class Option:
_creation_counter = 0
def __init__(self, type=None, *, required=False, positional=False, default=None):
def __init__(self, type=None, *, required=True, positional=False, default=None):
self.name = None
self.type = type
self.required = required
self.required = required if default is None else False
self.positional = positional
self.default = default
@ -60,12 +65,27 @@ class Option:
Option._creation_counter += 1
def __get__(self, inst, typ):
if not self.name in inst.__options_values__:
inst.__options_values__[self.name] = self.get_default()
return inst.__options_values__[self.name]
# XXX If we call this on the type, then either return overriden value or ... ???
if inst is None:
return vars(type).get(self.name, self)
if not self.name in inst._options_values:
inst._options_values[self.name] = self.get_default()
return inst._options_values[self.name]
def __set__(self, inst, value):
inst.__options_values__[self.name] = self.clean(value)
inst._options_values[self.name] = self.clean(value)
def __repr__(self):
return '<{positional}{typename} {type}{name} default={default!r}{required}>'.format(
typename=type(self).__name__,
type='({})'.format(self.type) if istype(self.type) else '',
name=self.name,
positional='*' if self.positional else '**',
default=self.default,
required=' (required)' if self.required else '',
)
def clean(self, value):
return self.type(value) if self.type else value
@ -105,20 +125,17 @@ class Method(Option):
"""
def __init__(self):
super().__init__(None, required=False)
def __get__(self, inst, typ):
if not self.name in inst.__options_values__:
inst.__options_values__[self.name] = getattr(inst, self.name)
return inst.__options_values__[self.name]
def __init__(self, *, required=True, positional=True):
super().__init__(None, required=required, positional=positional)
def __set__(self, inst, value):
if isinstance(value, str):
raise ValueError('should be callable')
inst.__options_values__[self.name] = self.type(value) if self.type else value
def clean(self, value):
if not hasattr(value, '__call__'):
raise ValueError('{} value must be callable.'.format(type(self).__name__))
return value
raise TypeError(
'Option of type {!r} is expecting a callable value, got {!r} object (which is not).'.
format(type(self).__name__, type(value).__name__)
)
inst._options_values[self.name] = self.type(value) if self.type else value
def __call__(self, *args, **kwargs):
# only here to trick IDEs into thinking this is callable.
raise NotImplementedError('You cannot call the descriptor')

10
bonobo/config/processors.py
View File

@ -74,8 +74,7 @@ class ContextCurrifier:
def __init__(self, wrapped, *initial_context):
self.wrapped = wrapped
self.context = tuple(initial_context)
self._stack = []
self._stack_values = []
self._stack, self._stack_values = None, None
def __iter__(self):
yield from self.wrapped
@ -86,8 +85,10 @@ class ContextCurrifier:
return self.wrapped(*self.context, *args, **kwargs)
def setup(self, *context):
if len(self._stack):
if self._stack is not None:
raise RuntimeError('Cannot setup context currification twice.')
self._stack, self._stack_values = list(), list()
for processor in resolve_processors(self.wrapped):
_processed = processor(self.wrapped, *context, *self.context)
_append_to_context = next(_processed)
@ -97,7 +98,7 @@ class ContextCurrifier:
self._stack.append(_processed)
def teardown(self):
while len(self._stack):
while self._stack:
processor = self._stack.pop()
try:
# todo yield from ? how to ?
@ -108,6 +109,7 @@ class ContextCurrifier:
else:
# No error ? We should have had StopIteration ...
raise RuntimeError('Context processors should not yield more than once.')
self._stack, self._stack_values = None, None
@contextmanager
def as_contextmanager(self, *context):

4
bonobo/config/services.py
View File

@ -53,7 +53,7 @@ class Service(Option):
super().__init__(str, required=False, default=name)
def __set__(self, inst, value):
inst.__options_values__[self.name] = validate_service_name(value)
inst._options_values[self.name] = validate_service_name(value)
def resolve(self, inst, services):
try:
@ -75,7 +75,7 @@ class Container(dict):
def args_for(self, mixed):
try:
options = mixed.__options__
options = dict(mixed.__options__)
except AttributeError:
options = {}

19
bonobo/errors.py
View File

@ -58,5 +58,22 @@ class ConfigurationError(Exception):
pass
class MissingServiceImplementationError(KeyError):
class UnrecoverableError(Exception):
"""Flag for errors that must interrupt the workflow, either because they will happen for sure on each node run, or
because you know that your transformation has no point continuing runnning after a bad event."""
class UnrecoverableValueError(UnrecoverableError, ValueError):
pass
class UnrecoverableRuntimeError(UnrecoverableError, RuntimeError):
pass
class UnrecoverableNotImplementedError(UnrecoverableError, NotImplementedError):
pass
class MissingServiceImplementationError(UnrecoverableError, KeyError):
pass

250
bonobo/examples/datasets/coffeeshops.json
View File

@ -1,182 +1,182 @@
{"Le Reynou": "2 bis quai de la m\u00e9gisserie, 75001 Paris, France",
"les montparnos": "65 boulevard Pasteur, 75015 Paris, France",
"Le Saint Jean": "23 rue des abbesses, 75018 Paris, France",
"Le Felteu": "1 rue Pecquay, 75004 Paris, France",
{"les montparnos": "65 boulevard Pasteur, 75015 Paris, France",
"Coffee Chope": "344Vrue Vaugirard, 75015 Paris, France",
"Caf\u00e9 Lea": "5 rue Claude Bernard, 75005 Paris, France",
"Le Bellerive": "71 quai de Seine, 75019 Paris, France",
"Le drapeau de la fidelit\u00e9": "21 rue Copreaux, 75015 Paris, France",
"O q de poule": "53 rue du ruisseau, 75018 Paris, France",
"Le caf\u00e9 des amis": "125 rue Blomet, 75015 Paris, France",
"Le chantereine": "51 Rue Victoire, 75009 Paris, France",
"Le M\u00fcller": "11 rue Feutrier, 75018 Paris, France",
"La Caravane": "Rue de la Fontaine au Roi, 75011 Paris, France",
"Le Pas Sage": "1 Passage du Grand Cerf, 75002 Paris, France",
"La Renaissance": "112 Rue Championnet, 75018 Paris, France",
"Ext\u00e9rieur Quai": "5, rue d'Alsace, 75010 Paris, France",
"Le Sully": "6 Bd henri IV, 75004 Paris, France",
"Le drapeau de la fidelit\u00e9": "21 rue Copreaux, 75015 Paris, France",
"Le caf\u00e9 des amis": "125 rue Blomet, 75015 Paris, France",
"Le Kleemend's": "34 avenue Pierre Mend\u00e8s-France, 75013 Paris, France",
"Assaporare Dix sur Dix": "75, avenue Ledru-Rollin, 75012 Paris, France",
"Caf\u00e9 Pierre": "202 rue du faubourg st antoine, 75012 Paris, France",
"Le Caf\u00e9 Livres": "10 rue Saint Martin, 75004 Paris, France",
"Le Chaumontois": "12 rue Armand Carrel, 75018 Paris, France",
"Le Square": "31 rue Saint-Dominique, 75007 Paris, France",
"Les Arcades": "61 rue de Ponthieu, 75008 Paris, France",
"La Bauloise": "36 rue du hameau, 75015 Paris, France",
"Le Dellac": "14 rue Rougemont, 75009 Paris, France",
"Le Bosquet": "46 avenue Bosquet, 75007 Paris, France",
"Le Sully": "6 Bd henri IV, 75004 Paris, France",
"Le Felteu": "1 rue Pecquay, 75004 Paris, France",
"Le bistrot de Ma\u00eblle et Augustin": "42 rue coquill\u00e8re, 75001 Paris, France",
"D\u00e9d\u00e9 la frite": "52 rue Notre-Dame des Victoires, 75002 Paris, France",
"Cardinal Saint-Germain": "11 boulevard Saint-Germain, 75005 Paris, France",
"Le Reynou": "2 bis quai de la m\u00e9gisserie, 75001 Paris, France",
"Aux cadrans": "21 ter boulevard Diderot, 75012 Paris, France",
"Le Saint Jean": "23 rue des abbesses, 75018 Paris, France",
"La Renaissance": "112 Rue Championnet, 75018 Paris, France",
"Le Square": "31 rue Saint-Dominique, 75007 Paris, France",
"Les Arcades": "61 rue de Ponthieu, 75008 Paris, France",
"Le Kleemend's": "34 avenue Pierre Mend\u00e8s-France, 75013 Paris, France",
"Assaporare Dix sur Dix": "75, avenue Ledru-Rollin, 75012 Paris, France",
"Caf\u00e9 Pierre": "202 rue du faubourg st antoine, 75012 Paris, France",
"Caf\u00e9 antoine": "17 rue Jean de la Fontaine, 75016 Paris, France",
"Au cerceau d'or": "129 boulevard sebastopol, 75002 Paris, France",
"Aux cadrans": "21 ter boulevard Diderot, 75012 Paris, France",
"Caf\u00e9 Lea": "5 rue Claude Bernard, 75005 Paris, France",
"Le Bellerive": "71 quai de Seine, 75019 Paris, France",
"La Bauloise": "36 rue du hameau, 75015 Paris, France",
"Le Dellac": "14 rue Rougemont, 75009 Paris, France",
"La Caravane": "Rue de la Fontaine au Roi, 75011 Paris, France",
"Le Pas Sage": "1 Passage du Grand Cerf, 75002 Paris, France",
"Le Caf\u00e9 Livres": "10 rue Saint Martin, 75004 Paris, France",
"Le Chaumontois": "12 rue Armand Carrel, 75018 Paris, France",
"Drole d'endroit pour une rencontre": "58 rue de Montorgueil, 75002 Paris, France",
"Le pari's caf\u00e9": "104 rue caulaincourt, 75018 Paris, France",
"Le Poulailler": "60 rue saint-sabin, 75011 Paris, France",
"Chai 33": "33 Cour Saint Emilion, 75012 Paris, France",
"L'Assassin": "99 rue Jean-Pierre Timbaud, 75011 Paris, France",
"l'Usine": "1 rue d'Avron, 75020 Paris, France",
"La Bricole": "52 rue Liebniz, 75018 Paris, France",
"le ronsard": "place maubert, 75005 Paris, France",
"Face Bar": "82 rue des archives, 75003 Paris, France",
"American Kitchen": "49 rue bichat, 75010 Paris, France",
"La Marine": "55 bis quai de valmy, 75010 Paris, France",
"Le Bloc": "21 avenue Brochant, 75017 Paris, France",
"La Recoleta au Manoir": "229 avenue Gambetta, 75020 Paris, France",
"Le Pareloup": "80 Rue Saint-Charles, 75015 Paris, France",
"La Brasserie Gait\u00e9": "3 rue de la Gait\u00e9, 75014 Paris, France",
"Caf\u00e9 Zen": "46 rue Victoire, 75009 Paris, France",
"O'Breizh": "27 rue de Penthi\u00e8vre, 75008 Paris, France",
"Le Petit Choiseul": "23 rue saint augustin, 75002 Paris, France",
"Invitez vous chez nous": "7 rue Ep\u00e9e de Bois, 75005 Paris, France",
"La Cordonnerie": "142 Rue Saint-Denis 75002 Paris, 75002 Paris, France",
"Le Supercoin": "3, rue Baudelique, 75018 Paris, France",
"Populettes": "86 bis rue Riquet, 75018 Paris, France",
"Au bon coin": "49 rue des Cloys, 75018 Paris, France",
"Le Couvent": "69 rue Broca, 75013 Paris, France",
"La Br\u00fblerie des Ternes": "111 rue mouffetard, 75005 Paris, France",
"L'\u00c9cir": "59 Boulevard Saint-Jacques, 75014 Paris, France",
"Le Chat bossu": "126, rue du Faubourg Saint Antoine, 75012 Paris, France",
"Denfert caf\u00e9": "58 boulvevard Saint Jacques, 75014 Paris, France",
"Le Caf\u00e9 frapp\u00e9": "95 rue Montmartre, 75002 Paris, France",
"La Perle": "78 rue vieille du temple, 75003 Paris, France",
"Le Descartes": "1 rue Thouin, 75005 Paris, France",
"Bagels & Coffee Corner": "Place de Clichy, 75017 Paris, France",
"Le petit club": "55 rue de la tombe Issoire, 75014 Paris, France",
"Le Plein soleil": "90 avenue Parmentier, 75011 Paris, France",
"Le Relais Haussmann": "146, boulevard Haussmann, 75008 Paris, France",
"Le Malar": "88 rue Saint-Dominique, 75007 Paris, France",
"Au panini de la place": "47 rue Belgrand, 75020 Paris, France",
"Le Village": "182 rue de Courcelles, 75017 Paris, France",
"Pause Caf\u00e9": "41 rue de Charonne, 75011 Paris, France",
"Le Pure caf\u00e9": "14 rue Jean Mac\u00e9, 75011 Paris, France",
"Extra old caf\u00e9": "307 fg saint Antoine, 75011 Paris, France",
"Chez Fafa": "44 rue Vinaigriers, 75010 Paris, France",
"En attendant l'or": "3 rue Faidherbe, 75011 Paris, France",
"Br\u00fblerie San Jos\u00e9": "30 rue des Petits-Champs, 75002 Paris, France",
"Caf\u00e9 de la Mairie (du VIII)": "rue de Lisbonne, 75008 Paris, France",
"Le General Beuret": "9 Place du General Beuret, 75015 Paris, France",
"Le Cap Bourbon": "1 rue Louis le Grand, 75002 Paris, France",
"En attendant l'or": "3 rue Faidherbe, 75011 Paris, France",
"Caf\u00e9 Martin": "2 place Martin Nadaud, 75001 Paris, France",
"Etienne": "14 rue Turbigo, Paris, 75001 Paris, France",
"L'ing\u00e9nu": "184 bd Voltaire, 75011 Paris, France",
"Le Biz": "18 rue Favart, 75002 Paris, France",
"L'Olive": "8 rue L'Olive, 75018 Paris, France",
"Le pari's caf\u00e9": "104 rue caulaincourt, 75018 Paris, France",
"Le Poulailler": "60 rue saint-sabin, 75011 Paris, France",
"La Marine": "55 bis quai de valmy, 75010 Paris, France",
"American Kitchen": "49 rue bichat, 75010 Paris, France",
"Chai 33": "33 Cour Saint Emilion, 75012 Paris, France",
"Face Bar": "82 rue des archives, 75003 Paris, France",
"Le Bloc": "21 avenue Brochant, 75017 Paris, France",
"La Bricole": "52 rue Liebniz, 75018 Paris, France",
"le ronsard": "place maubert, 75005 Paris, France",
"l'Usine": "1 rue d'Avron, 75020 Paris, France",
"La Cordonnerie": "142 Rue Saint-Denis 75002 Paris, 75002 Paris, France",
"Invitez vous chez nous": "7 rue Ep\u00e9e de Bois, 75005 Paris, France",
"Le sully": "13 rue du Faubourg Saint Denis, 75010 Paris, France",
"Le Ragueneau": "202 rue Saint-Honor\u00e9, 75001 Paris, France",
"Le Biz": "18 rue Favart, 75002 Paris, France",
"Le Cap Bourbon": "1 rue Louis le Grand, 75002 Paris, France",
"Le General Beuret": "9 Place du General Beuret, 75015 Paris, France",
"Le Germinal": "95 avenue Emile Zola, 75015 Paris, France",
"Le Ragueneau": "202 rue Saint-Honor\u00e9, 75001 Paris, France",
"Le refuge": "72 rue lamarck, 75018 Paris, France",
"Drole d'endroit pour une rencontre": "58 rue de Montorgueil, 75002 Paris, France",
"Le Petit Choiseul": "23 rue saint augustin, 75002 Paris, France",
"O'Breizh": "27 rue de Penthi\u00e8vre, 75008 Paris, France",
"Le Supercoin": "3, rue Baudelique, 75018 Paris, France",
"Populettes": "86 bis rue Riquet, 75018 Paris, France",
"La Recoleta au Manoir": "229 avenue Gambetta, 75020 Paris, France",
"L'Assassin": "99 rue Jean-Pierre Timbaud, 75011 Paris, France",
"Le Pareloup": "80 Rue Saint-Charles, 75015 Paris, France",
"Caf\u00e9 Zen": "46 rue Victoire, 75009 Paris, France",
"La Brasserie Gait\u00e9": "3 rue de la Gait\u00e9, 75014 Paris, France",
"Au bon coin": "49 rue des Cloys, 75018 Paris, France",
"La Br\u00fblerie des Ternes": "111 rue mouffetard, 75005 Paris, France",
"Le Chat bossu": "126, rue du Faubourg Saint Antoine, 75012 Paris, France",
"Denfert caf\u00e9": "58 boulvevard Saint Jacques, 75014 Paris, France",
"Le Couvent": "69 rue Broca, 75013 Paris, France",
"Bagels & Coffee Corner": "Place de Clichy, 75017 Paris, France",
"La Perle": "78 rue vieille du temple, 75003 Paris, France",
"Le Caf\u00e9 frapp\u00e9": "95 rue Montmartre, 75002 Paris, France",
"L'\u00c9cir": "59 Boulevard Saint-Jacques, 75014 Paris, France",
"Le Descartes": "1 rue Thouin, 75005 Paris, France",
"Le petit club": "55 rue de la tombe Issoire, 75014 Paris, France",
"Le Relais Haussmann": "146, boulevard Haussmann, 75008 Paris, France",
"Au panini de la place": "47 rue Belgrand, 75020 Paris, France",
"Extra old caf\u00e9": "307 fg saint Antoine, 75011 Paris, France",
"Le Plein soleil": "90 avenue Parmentier, 75011 Paris, France",
"Le Pure caf\u00e9": "14 rue Jean Mac\u00e9, 75011 Paris, France",
"Le Village": "182 rue de Courcelles, 75017 Paris, France",
"Le Malar": "88 rue Saint-Dominique, 75007 Paris, France",
"Pause Caf\u00e9": "41 rue de Charonne, 75011 Paris, France",
"Chez Fafa": "44 rue Vinaigriers, 75010 Paris, France",
"Caf\u00e9 dans l'aerogare Air France Invalides": "2 rue Robert Esnault Pelterie, 75007 Paris, France",
"Le relais de la victoire": "73 rue de la Victoire, 75009 Paris, France",
"Caprice caf\u00e9": "12 avenue Jean Moulin, 75014 Paris, France",
"Caves populaires": "22 rue des Dames, 75017 Paris, France",
"Cafe de grenelle": "188 rue de Grenelle, 75007 Paris, France",
"Chez Prune": "36 rue Beaurepaire, 75010 Paris, France",
"L'anjou": "1 rue de Montholon, 75009 Paris, France",
"Le Brio": "216, rue Marcadet, 75018 Paris, France",
"Tamm Bara": "7 rue Clisson, 75013 Paris, France",
"La chaumi\u00e8re gourmande": "Route de la Muette \u00e0 Neuilly",
"Club hippique du Jardin d\u2019Acclimatation": "75016 Paris, France",
"Les P\u00e8res Populaires": "46 rue de Buzenval, 75020 Paris, France",
"Epicerie Musicale": "55bis quai de Valmy, 75010 Paris, France",
"Le Centenaire": "104 rue amelot, 75011 Paris, France",
"Le Zazabar": "116 Rue de M\u00e9nilmontant, 75020 Paris, France",
"Ragueneau": "202 rue Saint Honor\u00e9, 75001 Paris, France",
"L'In\u00e9vitable": "22 rue Linn\u00e9, 75005 Paris, France",
"Le sully": "13 rue du Faubourg Saint Denis, 75010 Paris, France",
"Le Dunois": "77 rue Dunois, 75013 Paris, France",
"La Montagne Sans Genevi\u00e8ve": "13 Rue du Pot de Fer, 75005 Paris, France",
"Le Caminito": "48 rue du Dessous des Berges, 75013 Paris, France",
"Le petit Bretonneau": "Le petit Bretonneau - \u00e0 l'int\u00e9rieur de l'H\u00f4pital, 75018 Paris, France",
"La chaumi\u00e8re gourmande": "Route de la Muette \u00e0 Neuilly",
"Club hippique du Jardin d\u2019Acclimatation": "75016 Paris, France",
"Le bal du pirate": "60 rue des bergers, 75015 Paris, France",
"Le Zazabar": "116 Rue de M\u00e9nilmontant, 75020 Paris, France",
"L'antre d'eux": "16 rue DE MEZIERES, 75006 Paris, France",
"l'orillon bar": "35 rue de l'orillon, 75011 Paris, France",
"zic zinc": "95 rue claude decaen, 75012 Paris, France",
"Les P\u00e8res Populaires": "46 rue de Buzenval, 75020 Paris, France",
"Epicerie Musicale": "55bis quai de Valmy, 75010 Paris, France",
"Le relais de la victoire": "73 rue de la Victoire, 75009 Paris, France",
"Le Centenaire": "104 rue amelot, 75011 Paris, France",
"Cafe de grenelle": "188 rue de Grenelle, 75007 Paris, France",
"Ragueneau": "202 rue Saint Honor\u00e9, 75001 Paris, France",
"Caf\u00e9 Pistache": "9 rue des petits champs, 75001 Paris, France",
"La Cagnotte": "13 Rue Jean-Baptiste Dumay, 75020 Paris, France",
"bistrot les timbr\u00e9s": "14 rue d'alleray, 75015 Paris, France",
"Le Killy Jen": "28 bis boulevard Diderot, 75012 Paris, France",
"Caf\u00e9 beauveau": "9 rue de Miromesnil, 75008 Paris, France",
"le 1 cinq": "172 rue de vaugirard, 75015 Paris, France",
"Au Vin Des Rues": "21 rue Boulard, 75014 Paris, France",
"Les Artisans": "106 rue Lecourbe, 75015 Paris, France",
"Peperoni": "83 avenue de Wagram, 75001 Paris, France",
"Le BB (Bouchon des Batignolles)": "2 rue Lemercier, 75017 Paris, France",
"La Libert\u00e9": "196 rue du faubourg saint-antoine, 75012 Paris, France",
"Chez Rutabaga": "16 rue des Petits Champs, 75002 Paris, France",
"La cantoche de Paname": "40 Boulevard Beaumarchais, 75011 Paris, France",
"Le Saint Ren\u00e9": "148 Boulevard de Charonne, 75020 Paris, France",
"La Brocante": "10 rue Rossini, 75009 Paris, France",
"Caf\u00e9 Clochette": "16 avenue Richerand, 75010 Paris, France",
"L'europ\u00e9en": "21 Bis Boulevard Diderot, 75012 Paris, France",
"NoMa": "39 rue Notre Dame de Nazareth, 75003 Paris, France",
"O'Paris": "1 Rue des Envierges, 75020 Paris, France",
"Le Brio": "216, rue Marcadet, 75018 Paris, France",
"Tamm Bara": "7 rue Clisson, 75013 Paris, France",
"Caf\u00e9 dans l'aerogare Air France Invalides": "2 rue Robert Esnault Pelterie, 75007 Paris, France",
"bistrot les timbr\u00e9s": "14 rue d'alleray, 75015 Paris, France",
"Caprice caf\u00e9": "12 avenue Jean Moulin, 75014 Paris, France",
"Caves populaires": "22 rue des Dames, 75017 Paris, France",
"Au Vin Des Rues": "21 rue Boulard, 75014 Paris, France",
"Chez Prune": "36 rue Beaurepaire, 75010 Paris, France",
"L'In\u00e9vitable": "22 rue Linn\u00e9, 75005 Paris, France",
"L'anjou": "1 rue de Montholon, 75009 Paris, France",
"Botak cafe": "1 rue Paul albert, 75018 Paris, France",
"La cantine de Zo\u00e9": "136 rue du Faubourg poissonni\u00e8re, 75010 Paris, France",
"Les caves populaires": "22 rue des Dames, 75017 Paris, France",
"Le Plomb du cantal": "3 rue Ga\u00eet\u00e9, 75014 Paris, France",
"Trois pi\u00e8ces cuisine": "101 rue des dames, 75017 Paris, France",
"Le Zinc": "61 avenue de la Motte Picquet, 75015 Paris, France",
"L'avant comptoir": "3 carrefour de l'Od\u00e9on, 75006 Paris, France",
"Les Vendangeurs": "6/8 rue Stanislas, 75006 Paris, France",
"Chez Luna": "108 rue de M\u00e9nilmontant, 75020 Paris, France",
"Le bar Fleuri": "1 rue du Plateau, 75019 Paris, France",
"Bistrot Saint-Antoine": "58 rue du Fbg Saint-Antoine, 75012 Paris, France",
"Chez Oscar": "11/13 boulevard Beaumarchais, 75004 Paris, France",
"Le Piquet": "48 avenue de la Motte Picquet, 75015 Paris, France",
"L'avant comptoir": "3 carrefour de l'Od\u00e9on, 75006 Paris, France",
"le chateau d'eau": "67 rue du Ch\u00e2teau d'eau, 75010 Paris, France",
"Les Vendangeurs": "6/8 rue Stanislas, 75006 Paris, France",
"maison du vin": "52 rue des plantes, 75014 Paris, France",
"Le Tournebride": "104 rue Mouffetard, 75005 Paris, France",
"Le Fronton": "63 rue de Ponthieu, 75008 Paris, France",
"Le BB (Bouchon des Batignolles)": "2 rue Lemercier, 75017 Paris, France",
"La cantine de Zo\u00e9": "136 rue du Faubourg poissonni\u00e8re, 75010 Paris, France",
"Chez Rutabaga": "16 rue des Petits Champs, 75002 Paris, France",
"Les caves populaires": "22 rue des Dames, 75017 Paris, France",
"Le Plomb du cantal": "3 rue Ga\u00eet\u00e9, 75014 Paris, France",
"Trois pi\u00e8ces cuisine": "101 rue des dames, 75017 Paris, France",
"La Brocante": "10 rue Rossini, 75009 Paris, France",
"Le Zinc": "61 avenue de la Motte Picquet, 75015 Paris, France",
"Chez Luna": "108 rue de M\u00e9nilmontant, 75020 Paris, France",
"Le bar Fleuri": "1 rue du Plateau, 75019 Paris, France",
"La Libert\u00e9": "196 rue du faubourg saint-antoine, 75012 Paris, France",
"La cantoche de Paname": "40 Boulevard Beaumarchais, 75011 Paris, France",
"Le Saint Ren\u00e9": "148 Boulevard de Charonne, 75020 Paris, France",
"Caf\u00e9 Clochette": "16 avenue Richerand, 75010 Paris, France",
"L'europ\u00e9en": "21 Bis Boulevard Diderot, 75012 Paris, France",
"NoMa": "39 rue Notre Dame de Nazareth, 75003 Paris, France",
"le lutece": "380 rue de vaugirard, 75015 Paris, France",
"O'Paris": "1 Rue des Envierges, 75020 Paris, France",
"Rivolux": "16 rue de Rivoli, 75004 Paris, France",
"Brasiloja": "16 rue Ganneron, 75018 Paris, France",
"Le caf\u00e9 Monde et M\u00e9dias": "Place de la R\u00e9publique, 75003 Paris, France",
"L'entrep\u00f4t": "157 rue Bercy 75012 Paris, 75012 Paris, France",
"Coffee Chope": "344Vrue Vaugirard, 75015 Paris, France",
"l'El\u00e9phant du nil": "125 Rue Saint-Antoine, 75004 Paris, France",
"Le Parc Vaugirard": "358 rue de Vaugirard, 75015 Paris, France",
"Pari's Caf\u00e9": "174 avenue de Clichy, 75017 Paris, France",
"Le Comptoir": "354 bis rue Vaugirard, 75015 Paris, France",
"Caf\u00e9 Varenne": "36 rue de Varenne, 75007 Paris, France",
"Melting Pot": "3 rue de Lagny, 75020 Paris, France",
"le Zango": "58 rue Daguerre, 75014 Paris, France",
"Chez Miamophile": "6 rue M\u00e9lingue, 75019 Paris, France",
"Institut des Cultures d'Islam": "19-23 rue L\u00e9on, 75018 Paris, France",
"Canopy Caf\u00e9 associatif": "19 rue Pajol, 75018 Paris, France",
"Caf\u00e9 rallye tournelles": "11 Quai de la Tournelle, 75005 Paris, France",
"Petits Freres des Pauvres": "47 rue de Batignolles, 75017 Paris, France",
"Brasserie le Morvan": "61 rue du ch\u00e2teau d'eau, 75010 Paris, France",
"Le Lucernaire": "53 rue Notre-Dame des Champs, 75006 Paris, France",
"L'Angle": "28 rue de Ponthieu, 75008 Paris, France",
"Le Caf\u00e9 d'avant": "35 rue Claude Bernard, 75005 Paris, France",
"Caf\u00e9 Dupont": "198 rue de la Convention, 75015 Paris, France",
"Le S\u00e9vign\u00e9": "15 rue du Parc Royal, 75003 Paris, France",
"L'Entracte": "place de l'opera, 75002 Paris, France",
"Panem": "18 rue de Crussol, 75011 Paris, France",
"Au pays de Vannes": "34 bis rue de Wattignies, 75012 Paris, France",
"Le Lucernaire": "53 rue Notre-Dame des Champs, 75006 Paris, France",
"Le Caf\u00e9 d'avant": "35 rue Claude Bernard, 75005 Paris, France",
"l'El\u00e9phant du nil": "125 Rue Saint-Antoine, 75004 Paris, France",
"L'\u00e2ge d'or": "26 rue du Docteur Magnan, 75013 Paris, France",
"Le S\u00e9vign\u00e9": "15 rue du Parc Royal, 75003 Paris, France",
"Le Comptoir": "354 bis rue Vaugirard, 75015 Paris, France",
"L'horizon": "93, rue de la Roquette, 75011 Paris, France",
"L'empreinte": "54, avenue Daumesnil, 75012 Paris, France",
"Caf\u00e9 Victor": "10 boulevard Victor, 75015 Paris, France",
"Caf\u00e9 Varenne": "36 rue de Varenne, 75007 Paris, France",
"Le Brigadier": "12 rue Blanche, 75009 Paris, France",
"Waikiki": "10 rue d\"Ulm, 75005 Paris, France"}
"Waikiki": "10 rue d\"Ulm, 75005 Paris, France",
"Le Parc Vaugirard": "358 rue de Vaugirard, 75015 Paris, France",
"Pari's Caf\u00e9": "174 avenue de Clichy, 75017 Paris, France",
"Melting Pot": "3 rue de Lagny, 75020 Paris, France",
"le Zango": "58 rue Daguerre, 75014 Paris, France",
"Chez Miamophile": "6 rue M\u00e9lingue, 75019 Paris, France",
"Le caf\u00e9 Monde et M\u00e9dias": "Place de la R\u00e9publique, 75003 Paris, France",
"Caf\u00e9 rallye tournelles": "11 Quai de la Tournelle, 75005 Paris, France",
"Brasserie le Morvan": "61 rue du ch\u00e2teau d'eau, 75010 Paris, France",
"L'entrep\u00f4t": "157 rue Bercy 75012 Paris, 75012 Paris, France"}

248
bonobo/examples/datasets/coffeeshops.txt
View File

@ -1,182 +1,182 @@
Le Reynou, 2 bis quai de la mégisserie, 75001 Paris, France
les montparnos, 65 boulevard Pasteur, 75015 Paris, France
Le Saint Jean, 23 rue des abbesses, 75018 Paris, France
Le Felteu, 1 rue Pecquay, 75004 Paris, France
Coffee Chope, 344Vrue Vaugirard, 75015 Paris, France
Café Lea, 5 rue Claude Bernard, 75005 Paris, France
Le Bellerive, 71 quai de Seine, 75019 Paris, France
Le drapeau de la fidelité, 21 rue Copreaux, 75015 Paris, France
O q de poule, 53 rue du ruisseau, 75018 Paris, France
Le café des amis, 125 rue Blomet, 75015 Paris, France
Le chantereine, 51 Rue Victoire, 75009 Paris, France
Le Müller, 11 rue Feutrier, 75018 Paris, France
La Caravane, Rue de la Fontaine au Roi, 75011 Paris, France
Le Pas Sage, 1 Passage du Grand Cerf, 75002 Paris, France
La Renaissance, 112 Rue Championnet, 75018 Paris, France
Extérieur Quai, 5, rue d'Alsace, 75010 Paris, France
Le Sully, 6 Bd henri IV, 75004 Paris, France
Le drapeau de la fidelité, 21 rue Copreaux, 75015 Paris, France
Le café des amis, 125 rue Blomet, 75015 Paris, France
Le Kleemend's, 34 avenue Pierre Mendès-France, 75013 Paris, France
Assaporare Dix sur Dix, 75, avenue Ledru-Rollin, 75012 Paris, France
Café Pierre, 202 rue du faubourg st antoine, 75012 Paris, France
Le Café Livres, 10 rue Saint Martin, 75004 Paris, France
Le Chaumontois, 12 rue Armand Carrel, 75018 Paris, France
Le Square, 31 rue Saint-Dominique, 75007 Paris, France
Les Arcades, 61 rue de Ponthieu, 75008 Paris, France
La Bauloise, 36 rue du hameau, 75015 Paris, France
Le Dellac, 14 rue Rougemont, 75009 Paris, France
Le Bosquet, 46 avenue Bosquet, 75007 Paris, France
Le Sully, 6 Bd henri IV, 75004 Paris, France
Le Felteu, 1 rue Pecquay, 75004 Paris, France
Le bistrot de Maëlle et Augustin, 42 rue coquillère, 75001 Paris, France
Dédé la frite, 52 rue Notre-Dame des Victoires, 75002 Paris, France
Cardinal Saint-Germain, 11 boulevard Saint-Germain, 75005 Paris, France
Le Reynou, 2 bis quai de la mégisserie, 75001 Paris, France
Aux cadrans, 21 ter boulevard Diderot, 75012 Paris, France
Le Saint Jean, 23 rue des abbesses, 75018 Paris, France
La Renaissance, 112 Rue Championnet, 75018 Paris, France
Le Square, 31 rue Saint-Dominique, 75007 Paris, France
Les Arcades, 61 rue de Ponthieu, 75008 Paris, France
Le Kleemend's, 34 avenue Pierre Mendès-France, 75013 Paris, France
Assaporare Dix sur Dix, 75, avenue Ledru-Rollin, 75012 Paris, France
Café Pierre, 202 rue du faubourg st antoine, 75012 Paris, France
Café antoine, 17 rue Jean de la Fontaine, 75016 Paris, France
Au cerceau d'or, 129 boulevard sebastopol, 75002 Paris, France
Aux cadrans, 21 ter boulevard Diderot, 75012 Paris, France
Café Lea, 5 rue Claude Bernard, 75005 Paris, France
Le Bellerive, 71 quai de Seine, 75019 Paris, France
La Bauloise, 36 rue du hameau, 75015 Paris, France
Le Dellac, 14 rue Rougemont, 75009 Paris, France
La Caravane, Rue de la Fontaine au Roi, 75011 Paris, France
Le Pas Sage, 1 Passage du Grand Cerf, 75002 Paris, France
Le Café Livres, 10 rue Saint Martin, 75004 Paris, France
Le Chaumontois, 12 rue Armand Carrel, 75018 Paris, France
Drole d'endroit pour une rencontre, 58 rue de Montorgueil, 75002 Paris, France
Le pari's café, 104 rue caulaincourt, 75018 Paris, France
Le Poulailler, 60 rue saint-sabin, 75011 Paris, France
Chai 33, 33 Cour Saint Emilion, 75012 Paris, France
L'Assassin, 99 rue Jean-Pierre Timbaud, 75011 Paris, France
l'Usine, 1 rue d'Avron, 75020 Paris, France
La Bricole, 52 rue Liebniz, 75018 Paris, France
le ronsard, place maubert, 75005 Paris, France
Face Bar, 82 rue des archives, 75003 Paris, France
American Kitchen, 49 rue bichat, 75010 Paris, France
La Marine, 55 bis quai de valmy, 75010 Paris, France
Le Bloc, 21 avenue Brochant, 75017 Paris, France
La Recoleta au Manoir, 229 avenue Gambetta, 75020 Paris, France
Le Pareloup, 80 Rue Saint-Charles, 75015 Paris, France
La Brasserie Gaité, 3 rue de la Gaité, 75014 Paris, France
Café Zen, 46 rue Victoire, 75009 Paris, France
O'Breizh, 27 rue de Penthièvre, 75008 Paris, France
Le Petit Choiseul, 23 rue saint augustin, 75002 Paris, France
Invitez vous chez nous, 7 rue Epée de Bois, 75005 Paris, France
La Cordonnerie, 142 Rue Saint-Denis 75002 Paris, 75002 Paris, France
Le Supercoin, 3, rue Baudelique, 75018 Paris, France
Populettes, 86 bis rue Riquet, 75018 Paris, France
Au bon coin, 49 rue des Cloys, 75018 Paris, France
Le Couvent, 69 rue Broca, 75013 Paris, France
La Brûlerie des Ternes, 111 rue mouffetard, 75005 Paris, France
L'Écir, 59 Boulevard Saint-Jacques, 75014 Paris, France
Le Chat bossu, 126, rue du Faubourg Saint Antoine, 75012 Paris, France
Denfert café, 58 boulvevard Saint Jacques, 75014 Paris, France
Le Café frappé, 95 rue Montmartre, 75002 Paris, France
La Perle, 78 rue vieille du temple, 75003 Paris, France
Le Descartes, 1 rue Thouin, 75005 Paris, France
Bagels & Coffee Corner, Place de Clichy, 75017 Paris, France
Le petit club, 55 rue de la tombe Issoire, 75014 Paris, France
Le Plein soleil, 90 avenue Parmentier, 75011 Paris, France
Le Relais Haussmann, 146, boulevard Haussmann, 75008 Paris, France
Le Malar, 88 rue Saint-Dominique, 75007 Paris, France
Au panini de la place, 47 rue Belgrand, 75020 Paris, France
Le Village, 182 rue de Courcelles, 75017 Paris, France
Pause Café, 41 rue de Charonne, 75011 Paris, France
Le Pure café, 14 rue Jean Macé, 75011 Paris, France
Extra old café, 307 fg saint Antoine, 75011 Paris, France
Chez Fafa, 44 rue Vinaigriers, 75010 Paris, France
En attendant l'or, 3 rue Faidherbe, 75011 Paris, France
Brûlerie San José, 30 rue des Petits-Champs, 75002 Paris, France
Café de la Mairie (du VIII), rue de Lisbonne, 75008 Paris, France
Le General Beuret, 9 Place du General Beuret, 75015 Paris, France
Le Cap Bourbon, 1 rue Louis le Grand, 75002 Paris, France
En attendant l'or, 3 rue Faidherbe, 75011 Paris, France
Café Martin, 2 place Martin Nadaud, 75001 Paris, France
Etienne, 14 rue Turbigo, Paris, 75001 Paris, France
L'ingénu, 184 bd Voltaire, 75011 Paris, France
Le Biz, 18 rue Favart, 75002 Paris, France
L'Olive, 8 rue L'Olive, 75018 Paris, France
Le pari's café, 104 rue caulaincourt, 75018 Paris, France
Le Poulailler, 60 rue saint-sabin, 75011 Paris, France
La Marine, 55 bis quai de valmy, 75010 Paris, France
American Kitchen, 49 rue bichat, 75010 Paris, France
Chai 33, 33 Cour Saint Emilion, 75012 Paris, France
Face Bar, 82 rue des archives, 75003 Paris, France
Le Bloc, 21 avenue Brochant, 75017 Paris, France
La Bricole, 52 rue Liebniz, 75018 Paris, France
le ronsard, place maubert, 75005 Paris, France
l'Usine, 1 rue d'Avron, 75020 Paris, France
La Cordonnerie, 142 Rue Saint-Denis 75002 Paris, 75002 Paris, France
Invitez vous chez nous, 7 rue Epée de Bois, 75005 Paris, France
Le sully, 13 rue du Faubourg Saint Denis, 75010 Paris, France
Le Ragueneau, 202 rue Saint-Honoré, 75001 Paris, France
Le Biz, 18 rue Favart, 75002 Paris, France
Le Cap Bourbon, 1 rue Louis le Grand, 75002 Paris, France
Le General Beuret, 9 Place du General Beuret, 75015 Paris, France
Le Germinal, 95 avenue Emile Zola, 75015 Paris, France
Le Ragueneau, 202 rue Saint-Honoré, 75001 Paris, France
Le refuge, 72 rue lamarck, 75018 Paris, France
Drole d'endroit pour une rencontre, 58 rue de Montorgueil, 75002 Paris, France
Le Petit Choiseul, 23 rue saint augustin, 75002 Paris, France
O'Breizh, 27 rue de Penthièvre, 75008 Paris, France
Le Supercoin, 3, rue Baudelique, 75018 Paris, France
Populettes, 86 bis rue Riquet, 75018 Paris, France
La Recoleta au Manoir, 229 avenue Gambetta, 75020 Paris, France
L'Assassin, 99 rue Jean-Pierre Timbaud, 75011 Paris, France
Le Pareloup, 80 Rue Saint-Charles, 75015 Paris, France
Café Zen, 46 rue Victoire, 75009 Paris, France
La Brasserie Gaité, 3 rue de la Gaité, 75014 Paris, France
Au bon coin, 49 rue des Cloys, 75018 Paris, France
La Brûlerie des Ternes, 111 rue mouffetard, 75005 Paris, France
Le Chat bossu, 126, rue du Faubourg Saint Antoine, 75012 Paris, France
Denfert café, 58 boulvevard Saint Jacques, 75014 Paris, France
Le Couvent, 69 rue Broca, 75013 Paris, France
Bagels & Coffee Corner, Place de Clichy, 75017 Paris, France
La Perle, 78 rue vieille du temple, 75003 Paris, France
Le Café frappé, 95 rue Montmartre, 75002 Paris, France
L'Écir, 59 Boulevard Saint-Jacques, 75014 Paris, France
Le Descartes, 1 rue Thouin, 75005 Paris, France
Le petit club, 55 rue de la tombe Issoire, 75014 Paris, France
Le Relais Haussmann, 146, boulevard Haussmann, 75008 Paris, France
Au panini de la place, 47 rue Belgrand, 75020 Paris, France
Extra old café, 307 fg saint Antoine, 75011 Paris, France
Le Plein soleil, 90 avenue Parmentier, 75011 Paris, France
Le Pure café, 14 rue Jean Macé, 75011 Paris, France
Le Village, 182 rue de Courcelles, 75017 Paris, France
Le Malar, 88 rue Saint-Dominique, 75007 Paris, France
Pause Café, 41 rue de Charonne, 75011 Paris, France
Chez Fafa, 44 rue Vinaigriers, 75010 Paris, France
Café dans l'aerogare Air France Invalides, 2 rue Robert Esnault Pelterie, 75007 Paris, France
Le relais de la victoire, 73 rue de la Victoire, 75009 Paris, France
Caprice café, 12 avenue Jean Moulin, 75014 Paris, France
Caves populaires, 22 rue des Dames, 75017 Paris, France
Cafe de grenelle, 188 rue de Grenelle, 75007 Paris, France
Chez Prune, 36 rue Beaurepaire, 75010 Paris, France
L'anjou, 1 rue de Montholon, 75009 Paris, France
Le Brio, 216, rue Marcadet, 75018 Paris, France
Tamm Bara, 7 rue Clisson, 75013 Paris, France
La chaumière gourmande, Route de la Muette à Neuilly
Club hippique du Jardin d’Acclimatation, 75016 Paris, France
Les Pères Populaires, 46 rue de Buzenval, 75020 Paris, France
Epicerie Musicale, 55bis quai de Valmy, 75010 Paris, France
Le Centenaire, 104 rue amelot, 75011 Paris, France
Le Zazabar, 116 Rue de Ménilmontant, 75020 Paris, France
Ragueneau, 202 rue Saint Honoré, 75001 Paris, France
L'Inévitable, 22 rue Linné, 75005 Paris, France
Le sully, 13 rue du Faubourg Saint Denis, 75010 Paris, France
Le Dunois, 77 rue Dunois, 75013 Paris, France
La Montagne Sans Geneviève, 13 Rue du Pot de Fer, 75005 Paris, France
Le Caminito, 48 rue du Dessous des Berges, 75013 Paris, France
Le petit Bretonneau, Le petit Bretonneau - à l'intérieur de l'Hôpital, 75018 Paris, France
La chaumière gourmande, Route de la Muette à Neuilly
Club hippique du Jardin d’Acclimatation, 75016 Paris, France
Le bal du pirate, 60 rue des bergers, 75015 Paris, France
Le Zazabar, 116 Rue de Ménilmontant, 75020 Paris, France
L'antre d'eux, 16 rue DE MEZIERES, 75006 Paris, France
l'orillon bar, 35 rue de l'orillon, 75011 Paris, France
zic zinc, 95 rue claude decaen, 75012 Paris, France
Les Pères Populaires, 46 rue de Buzenval, 75020 Paris, France
Epicerie Musicale, 55bis quai de Valmy, 75010 Paris, France
Le relais de la victoire, 73 rue de la Victoire, 75009 Paris, France
Le Centenaire, 104 rue amelot, 75011 Paris, France
Cafe de grenelle, 188 rue de Grenelle, 75007 Paris, France
Ragueneau, 202 rue Saint Honoré, 75001 Paris, France
Café Pistache, 9 rue des petits champs, 75001 Paris, France
La Cagnotte, 13 Rue Jean-Baptiste Dumay, 75020 Paris, France
bistrot les timbrés, 14 rue d'alleray, 75015 Paris, France
Le Killy Jen, 28 bis boulevard Diderot, 75012 Paris, France
Café beauveau, 9 rue de Miromesnil, 75008 Paris, France
le 1 cinq, 172 rue de vaugirard, 75015 Paris, France
Au Vin Des Rues, 21 rue Boulard, 75014 Paris, France
Les Artisans, 106 rue Lecourbe, 75015 Paris, France
Peperoni, 83 avenue de Wagram, 75001 Paris, France
Le BB (Bouchon des Batignolles), 2 rue Lemercier, 75017 Paris, France
La Liberté, 196 rue du faubourg saint-antoine, 75012 Paris, France
Chez Rutabaga, 16 rue des Petits Champs, 75002 Paris, France
La cantoche de Paname, 40 Boulevard Beaumarchais, 75011 Paris, France
Le Saint René, 148 Boulevard de Charonne, 75020 Paris, France
La Brocante, 10 rue Rossini, 75009 Paris, France
Café Clochette, 16 avenue Richerand, 75010 Paris, France
L'européen, 21 Bis Boulevard Diderot, 75012 Paris, France
NoMa, 39 rue Notre Dame de Nazareth, 75003 Paris, France
O'Paris, 1 Rue des Envierges, 75020 Paris, France
Le Brio, 216, rue Marcadet, 75018 Paris, France
Tamm Bara, 7 rue Clisson, 75013 Paris, France
Café dans l'aerogare Air France Invalides, 2 rue Robert Esnault Pelterie, 75007 Paris, France
bistrot les timbrés, 14 rue d'alleray, 75015 Paris, France
Caprice café, 12 avenue Jean Moulin, 75014 Paris, France
Caves populaires, 22 rue des Dames, 75017 Paris, France
Au Vin Des Rues, 21 rue Boulard, 75014 Paris, France
Chez Prune, 36 rue Beaurepaire, 75010 Paris, France
L'Inévitable, 22 rue Linné, 75005 Paris, France
L'anjou, 1 rue de Montholon, 75009 Paris, France
Botak cafe, 1 rue Paul albert, 75018 Paris, France
La cantine de Zoé, 136 rue du Faubourg poissonnière, 75010 Paris, France
Les caves populaires, 22 rue des Dames, 75017 Paris, France
Le Plomb du cantal, 3 rue Gaîté, 75014 Paris, France
Trois pièces cuisine, 101 rue des dames, 75017 Paris, France
Le Zinc, 61 avenue de la Motte Picquet, 75015 Paris, France
L'avant comptoir, 3 carrefour de l'Odéon, 75006 Paris, France
Les Vendangeurs, 6/8 rue Stanislas, 75006 Paris, France
Chez Luna, 108 rue de Ménilmontant, 75020 Paris, France
Le bar Fleuri, 1 rue du Plateau, 75019 Paris, France
Bistrot Saint-Antoine, 58 rue du Fbg Saint-Antoine, 75012 Paris, France
Chez Oscar, 11/13 boulevard Beaumarchais, 75004 Paris, France
Le Piquet, 48 avenue de la Motte Picquet, 75015 Paris, France
L'avant comptoir, 3 carrefour de l'Odéon, 75006 Paris, France
le chateau d'eau, 67 rue du Château d'eau, 75010 Paris, France
Les Vendangeurs, 6/8 rue Stanislas, 75006 Paris, France
maison du vin, 52 rue des plantes, 75014 Paris, France
Le Tournebride, 104 rue Mouffetard, 75005 Paris, France
Le Fronton, 63 rue de Ponthieu, 75008 Paris, France
Le BB (Bouchon des Batignolles), 2 rue Lemercier, 75017 Paris, France
La cantine de Zoé, 136 rue du Faubourg poissonnière, 75010 Paris, France
Chez Rutabaga, 16 rue des Petits Champs, 75002 Paris, France
Les caves populaires, 22 rue des Dames, 75017 Paris, France
Le Plomb du cantal, 3 rue Gaîté, 75014 Paris, France
Trois pièces cuisine, 101 rue des dames, 75017 Paris, France
La Brocante, 10 rue Rossini, 75009 Paris, France
Le Zinc, 61 avenue de la Motte Picquet, 75015 Paris, France
Chez Luna, 108 rue de Ménilmontant, 75020 Paris, France
Le bar Fleuri, 1 rue du Plateau, 75019 Paris, France
La Liberté, 196 rue du faubourg saint-antoine, 75012 Paris, France
La cantoche de Paname, 40 Boulevard Beaumarchais, 75011 Paris, France
Le Saint René, 148 Boulevard de Charonne, 75020 Paris, France
Café Clochette, 16 avenue Richerand, 75010 Paris, France
L'européen, 21 Bis Boulevard Diderot, 75012 Paris, France
NoMa, 39 rue Notre Dame de Nazareth, 75003 Paris, France
le lutece, 380 rue de vaugirard, 75015 Paris, France
O'Paris, 1 Rue des Envierges, 75020 Paris, France
Rivolux, 16 rue de Rivoli, 75004 Paris, France
Brasiloja, 16 rue Ganneron, 75018 Paris, France
Le café Monde et Médias, Place de la République, 75003 Paris, France
L'entrepôt, 157 rue Bercy 75012 Paris, 75012 Paris, France
Coffee Chope, 344Vrue Vaugirard, 75015 Paris, France
l'Eléphant du nil, 125 Rue Saint-Antoine, 75004 Paris, France
Le Parc Vaugirard, 358 rue de Vaugirard, 75015 Paris, France
Pari's Café, 174 avenue de Clichy, 75017 Paris, France
Le Comptoir, 354 bis rue Vaugirard, 75015 Paris, France
Café Varenne, 36 rue de Varenne, 75007 Paris, France
Melting Pot, 3 rue de Lagny, 75020 Paris, France
le Zango, 58 rue Daguerre, 75014 Paris, France
Chez Miamophile, 6 rue Mélingue, 75019 Paris, France
Institut des Cultures d'Islam, 19-23 rue Léon, 75018 Paris, France
Canopy Café associatif, 19 rue Pajol, 75018 Paris, France
Café rallye tournelles, 11 Quai de la Tournelle, 75005 Paris, France
Petits Freres des Pauvres, 47 rue de Batignolles, 75017 Paris, France
Brasserie le Morvan, 61 rue du château d'eau, 75010 Paris, France
Le Lucernaire, 53 rue Notre-Dame des Champs, 75006 Paris, France
L'Angle, 28 rue de Ponthieu, 75008 Paris, France
Le Café d'avant, 35 rue Claude Bernard, 75005 Paris, France
Café Dupont, 198 rue de la Convention, 75015 Paris, France
Le Sévigné, 15 rue du Parc Royal, 75003 Paris, France
L'Entracte, place de l'opera, 75002 Paris, France
Panem, 18 rue de Crussol, 75011 Paris, France
Au pays de Vannes, 34 bis rue de Wattignies, 75012 Paris, France
Le Lucernaire, 53 rue Notre-Dame des Champs, 75006 Paris, France
Le Café d'avant, 35 rue Claude Bernard, 75005 Paris, France
l'Eléphant du nil, 125 Rue Saint-Antoine, 75004 Paris, France
L'âge d'or, 26 rue du Docteur Magnan, 75013 Paris, France
Le Sévigné, 15 rue du Parc Royal, 75003 Paris, France
Le Comptoir, 354 bis rue Vaugirard, 75015 Paris, France
L'horizon, 93, rue de la Roquette, 75011 Paris, France
L'empreinte, 54, avenue Daumesnil, 75012 Paris, France
Café Victor, 10 boulevard Victor, 75015 Paris, France
Café Varenne, 36 rue de Varenne, 75007 Paris, France
Le Brigadier, 12 rue Blanche, 75009 Paris, France
Waikiki, 10 rue d"Ulm, 75005 Paris, France
Waikiki, 10 rue d"Ulm, 75005 Paris, France
Le Parc Vaugirard, 358 rue de Vaugirard, 75015 Paris, France
Pari's Café, 174 avenue de Clichy, 75017 Paris, France
Melting Pot, 3 rue de Lagny, 75020 Paris, France
le Zango, 58 rue Daguerre, 75014 Paris, France
Chez Miamophile, 6 rue Mélingue, 75019 Paris, France
Le café Monde et Médias, Place de la République, 75003 Paris, France
Café rallye tournelles, 11 Quai de la Tournelle, 75005 Paris, France
Brasserie le Morvan, 61 rue du château d'eau, 75010 Paris, France
L'entrepôt, 157 rue Bercy 75012 Paris, 75012 Paris, France

6
bonobo/examples/datasets/fablabs.py
View File

@ -87,13 +87,11 @@ def display(row):
graph = bonobo.Graph(
OpenDataSoftAPI(
dataset=API_DATASET, netloc=API_NETLOC, timezone='Europe/Paris'
),
OpenDataSoftAPI(dataset=API_DATASET, netloc=API_NETLOC, timezone='Europe/Paris'),
normalize,
bonobo.Filter(filter=lambda row: row.get('country') == 'France'),
bonobo.JsonWriter(path='fablabs.txt', ioformat='arg0'),
bonobo.Tee(display),
bonobo.JsonWriter(path='fablabs.txt'),
)
if __name__ == '__main__':

7
bonobo/examples/nodes/filter.py
View File

@ -9,13 +9,16 @@ class OddOnlyFilter(Filter):
@Filter
def MultiplesOfThreeOnlyFilter(self, i):
def multiples_of_three(i):
return not (i % 3)
graph = bonobo.Graph(
lambda: tuple(range(50)),
OddOnlyFilter(),
MultiplesOfThreeOnlyFilter(),
multiples_of_three,
print,
)
if __name__ == '__main__':
bonobo.run(graph)

3
bonobo/examples/nodes/slow.py
View File

@ -14,3 +14,6 @@ graph = bonobo.Graph(
pause,
print,
)
if __name__ == '__main__':
bonobo.run(graph)

6
bonobo/execution/node.py
View File

@ -3,7 +3,7 @@ from queue import Empty
from time import sleep
from bonobo.constants import INHERIT_INPUT, NOT_MODIFIED
from bonobo.errors import InactiveReadableError
from bonobo.errors import InactiveReadableError, UnrecoverableError
from bonobo.execution.base import LoopingExecutionContext
from bonobo.structs.bags import Bag
from bonobo.structs.inputs import Input
@ -93,6 +93,10 @@ class NodeExecutionContext(WithStatistics, LoopingExecutionContext):
except Empty:
sleep(self.PERIOD)
continue
except UnrecoverableError as exc:
self.handle_error(exc, traceback.format_exc())
self.input.shutdown()
break
except Exception as exc: # pylint: disable=broad-except
self.handle_error(exc, traceback.format_exc())

5
bonobo/execution/plugin.py
View File

@ -16,8 +16,9 @@ class PluginExecutionContext(LoopingExecutionContext):
self.wrapped.initialize()
def shutdown(self):
with recoverable(self.handle_error):
self.wrapped.finalize()
if self.started:
with recoverable(self.handle_error):
self.wrapped.finalize()
self.alive = False
def step(self):

42
bonobo/ext/console.py
View File

@ -2,7 +2,9 @@ import io
import sys
from contextlib import redirect_stdout
from colorama import Style, Fore
from colorama import Style, Fore, init
init(wrap=True)
from bonobo import settings
from bonobo.plugins import Plugin
@ -10,6 +12,13 @@ from bonobo.util.term import CLEAR_EOL, MOVE_CURSOR_UP
class IOBuffer():
"""
The role of IOBuffer is to overcome the problem of multiple threads wanting to write to stdout at the same time. It
works a bit like a videogame: there are two buffers, one that is used to write, and one which is used to read from.
On each cycle, we swap the buffers, and the console plugin handle output of the one which is not anymore "active".
"""
def __init__(self):
self.current = io.StringIO()
self.write = self.current.write
@ -23,12 +32,18 @@ class IOBuffer():
finally:
previous.close()
def flush(self):
self.current.flush()
class ConsoleOutputPlugin(Plugin):
"""
Outputs status information to the connected stdout. Can be a TTY, with or without support for colors/cursor
movements, or a non tty (pipe, file, ...). The features are adapted to terminal capabilities.
On Windows, we'll play a bit differently because we don't know how to manipulate cursor position. We'll only
display stats at the very end, and there won't be this "buffering" logic we need to display both stats and stdout.
.. attribute:: prefix
String prefix of output lines.
@ -40,17 +55,18 @@ class ConsoleOutputPlugin(Plugin):
self.counter = 0
self._append_cache = ''
self.isatty = sys.stdout.isatty()
self.iswindows = (sys.platform == 'win32')
self._stdout = sys.stdout
self.stdout = IOBuffer()
self.redirect_stdout = redirect_stdout(self.stdout)
self.redirect_stdout = redirect_stdout(self._stdout if self.iswindows else self.stdout)
self.redirect_stdout.__enter__()
def run(self):
if self.isatty:
if self.isatty and not self.iswindows:
self._write(self.context.parent, rewind=True)
else:
pass # not a tty
pass # not a tty, or windows, so we'll ignore stats output
def finalize(self):
self._write(self.context.parent, rewind=False)
@ -59,24 +75,28 @@ class ConsoleOutputPlugin(Plugin):
def write(self, context, prefix='', rewind=True, append=None):
t_cnt = len(context)
buffered = self.stdout.switch()
for line in buffered.split('\n')[:-1]:
print(line + CLEAR_EOL, file=sys.stderr)
if not self.iswindows:
buffered = self.stdout.switch()
for line in buffered.split('\n')[:-1]:
print(line + CLEAR_EOL, file=sys.stderr)
alive_color = Style.BRIGHT
dead_color = (Style.BRIGHT + Fore.BLACK) if self.iswindows else Fore.BLACK
for i in context.graph.topologically_sorted_indexes:
node = context[i]
name_suffix = '({})'.format(i) if settings.DEBUG else ''
name_suffix = '({})'.format(i) if settings.DEBUG.get() else ''
if node.alive:
_line = ''.join(
(
' ', Style.BRIGHT, '+', Style.RESET_ALL, ' ', node.name, name_suffix, ' ',
' ', alive_color, '+', Style.RESET_ALL, ' ', node.name, name_suffix, ' ',
node.get_statistics_as_string(), Style.RESET_ALL, ' ',
)
)
else:
_line = ''.join(
(
' ', Fore.BLACK, '-', ' ', node.name, name_suffix, ' ', node.get_statistics_as_string(),
' ', dead_color, '-', ' ', node.name, name_suffix, ' ', node.get_statistics_as_string(),
Style.RESET_ALL, ' ',
)
)
@ -100,7 +120,7 @@ class ConsoleOutputPlugin(Plugin):
print(MOVE_CURSOR_UP(t_cnt + 2), file=sys.stderr)
def _write(self, graph_context, rewind):
if settings.PROFILE:
if settings.PROFILE.get():
if self.counter % 10 and self._append_cache:
append = self._append_cache
else:

4
bonobo/ext/jupyter/plugin.py
View File

@ -1,11 +1,11 @@
import logging
from bonobo.ext.jupyter.widget import BonoboWidget
from bonobo.plugins import Plugin
try:
import IPython.core.display
except ImportError as e:
import logging
logging.exception(
'You must install Jupyter to use the bonobo Jupyter extension. Easiest way is to install the '
'optional "jupyter" dependencies with «pip install bonobo[jupyter]», but you can also install a '

4
bonobo/ext/opendatasoft.py
View File

@ -13,13 +13,13 @@ def path_str(path):
class OpenDataSoftAPI(Configurable):
dataset = Option(str, required=True)
dataset = Option(str, positional=True)
endpoint = Option(str, default='{scheme}://{netloc}{path}')
scheme = Option(str, default='https')
netloc = Option(str, default='data.opendatasoft.com')
path = Option(path_str, default='/api/records/1.0/search/')
rows = Option(int, default=500)
limit = Option(int, default=None)
limit = Option(int, required=False)
timezone = Option(str, default='Europe/Paris')
kwargs = Option(dict, default=dict)

22
bonobo/logging.py
View File

@ -8,19 +8,22 @@ from colorama import Fore, Style
from bonobo import settings
from bonobo.util.term import CLEAR_EOL
iswindows = (sys.platform == 'win32')
def get_format():
yield '{b}[%(fg)s%(levelname)s{b}][{w}'
yield '{b}][{w}'.join(('%(spent)04d', '%(name)s'))
yield '{b}]'
yield ' %(fg)s%(message)s{r}'
yield CLEAR_EOL
if not iswindows:
yield CLEAR_EOL
colors = {
'b': Fore.BLACK,
'w': Fore.LIGHTBLACK_EX,
'r': Style.RESET_ALL,
'b': '' if iswindows else Fore.BLACK,
'w': '' if iswindows else Fore.LIGHTBLACK_EX,
'r': '' if iswindows else Style.RESET_ALL,
}
format = (''.join(get_format())).format(**colors)
@ -28,7 +31,9 @@ format = (''.join(get_format())).format(**colors)
class Filter(logging.Filter):
def filter(self, record):
record.spent = record.relativeCreated // 1000
if record.levelname == 'DEBG':
if iswindows:
record.fg = ''
elif record.levelname == 'DEBG':
record.fg = Fore.LIGHTBLACK_EX
elif record.levelname == 'INFO':
record.fg = Fore.LIGHTWHITE_EX
@ -46,7 +51,10 @@ class Filter(logging.Filter):
class Formatter(logging.Formatter):
def formatException(self, ei):
tb = super().formatException(ei)
return textwrap.indent(tb, Fore.BLACK + ' | ' + Fore.WHITE)
if iswindows:
return textwrap.indent(tb, ' | ')
else:
return textwrap.indent(tb, Fore.BLACK + ' | ' + Fore.WHITE)
def setup(level):
@ -75,4 +83,4 @@ def get_logger(name='bonobo'):
getLogger = get_logger
# Setup formating and level.
setup(level=settings.LOGGING_LEVEL)
setup(level=settings.LOGGING_LEVEL.get())

11
bonobo/nodes/__init__.py
View File

@ -1,9 +1,8 @@
from bonobo.nodes.io import __all__ as _all_io
from bonobo.nodes.io import *
from bonobo.nodes.basics import __all__ as _all_basics
from bonobo.nodes.basics import *
from bonobo.nodes.basics import __all__ as _all_basics
from bonobo.nodes.filter import Filter
from bonobo.nodes.io import *
from bonobo.nodes.io import __all__ as _all_io
from bonobo.nodes.throttle import RateLimited
__all__ = _all_basics + _all_io + ['Filter']
__all__ = _all_basics + _all_io + ['Filter', 'RateLimited']

80
bonobo/nodes/basics.py
View File

@ -1,24 +1,22 @@
import functools
from pprint import pprint as _pprint
import itertools
from colorama import Fore, Style
from bonobo import settings
from bonobo.config import Configurable, Option
from bonobo.config.processors import ContextProcessor
from bonobo.constants import NOT_MODIFIED
from bonobo.structs.bags import Bag
from bonobo.util.objects import ValueHolder
from bonobo.util.term import CLEAR_EOL
from bonobo.constants import NOT_MODIFIED
__all__ = [
'identity',
'Limit',
'Tee',
'count',
'pprint',
'PrettyPrinter',
'Tee',
'arg0_to_kwargs',
'count',
'identity',
'kwargs_to_arg0',
'noop',
]
@ -73,7 +71,7 @@ def _count_counter(self, context):
class PrettyPrinter(Configurable):
def call(self, *args, **kwargs):
formater = self._format_quiet if settings.QUIET else self._format_console
formater = self._format_quiet if settings.QUIET.get() else self._format_console
for i, (item, value) in enumerate(itertools.chain(enumerate(args), kwargs.items())):
print(formater(i, item, value))
@ -87,48 +85,28 @@ class PrettyPrinter(Configurable):
)
pprint = PrettyPrinter()
pprint.__name__ = 'pprint'
def PrettyPrint(title_keys=('title', 'name', 'id'), print_values=True, sort=True):
from bonobo.constants import NOT_MODIFIED
from colorama import Fore, Style
def _pprint(*args, **kwargs):
nonlocal title_keys, sort, print_values
row = args[0]
for key in title_keys:
if key in row:
print(Style.BRIGHT, row.get(key), Style.RESET_ALL, sep='')
break
if print_values:
for k in sorted(row) if sort else row:
print(
' • ',
Fore.BLUE,
k,
Style.RESET_ALL,
' : ',
Fore.BLACK,
'(',
type(row[k]).__name__,
')',
Style.RESET_ALL,
' ',
repr(row[k]),
CLEAR_EOL,
)
yield NOT_MODIFIED
_pprint.__name__ = 'pprint'
return _pprint
def noop(*args, **kwargs): # pylint: disable=unused-argument
from bonobo.constants import NOT_MODIFIED
return NOT_MODIFIED
def arg0_to_kwargs(row):
"""
Transform items in a stream from "arg0" format (each call only has one positional argument, which is a dict-like
object) to "kwargs" format (each call only has keyword arguments that represent a row).
:param row:
:return: bonobo.Bag
"""
return Bag(**row)
def kwargs_to_arg0(**row):
"""
Transform items in a stream from "kwargs" format (each call only has keyword arguments that represent a row) to
"arg0" format (each call only has one positional argument, which is a dict-like object) .
:param **row:
:return: bonobo.Bag
"""
return Bag(row)

9
bonobo/nodes/io/base.py
View File

@ -1,5 +1,6 @@
from bonobo import settings
from bonobo.config import Configurable, ContextProcessor, Option, Service
from bonobo.errors import UnrecoverableValueError, UnrecoverableNotImplementedError
from bonobo.structs.bags import Bag
@ -9,7 +10,7 @@ class IOFormatEnabled(Configurable):
def get_input(self, *args, **kwargs):
if self.ioformat == settings.IOFORMAT_ARG0:
if len(args) != 1 or len(kwargs):
raise ValueError(
raise UnrecoverableValueError(
'Wrong input formating: IOFORMAT=ARG0 implies one arg and no kwargs, got args={!r} and kwargs={!r}.'.
format(args, kwargs)
)
@ -17,13 +18,13 @@ class IOFormatEnabled(Configurable):
if self.ioformat == settings.IOFORMAT_KWARGS:
if len(args) or not len(kwargs):
raise ValueError(
raise UnrecoverableValueError(
'Wrong input formating: IOFORMAT=KWARGS ioformat implies no arg, got args={!r} and kwargs={!r}.'.
format(args, kwargs)
)
return kwargs
raise NotImplementedError('Unsupported format.')
raise UnrecoverableNotImplementedError('Unsupported format.')
def get_output(self, row):
if self.ioformat == settings.IOFORMAT_ARG0:
@ -32,7 +33,7 @@ class IOFormatEnabled(Configurable):
if self.ioformat == settings.IOFORMAT_KWARGS:
return Bag(**row)
raise NotImplementedError('Unsupported format.')
raise UnrecoverableNotImplementedError('Unsupported format.')
class FileHandler(Configurable):

2
bonobo/nodes/io/csv.py
View File

@ -26,7 +26,7 @@ class CsvHandler(FileHandler):
"""
delimiter = Option(str, default=';')
quotechar = Option(str, default='"')
headers = Option(tuple)
headers = Option(tuple, required=False)
class CsvReader(IOFormatEnabled, FileReader, CsvHandler):

2
bonobo/nodes/io/pickle.py
View File

@ -17,7 +17,7 @@ class PickleHandler(FileHandler):
"""
item_names = Option(tuple)
item_names = Option(tuple, required=False)
class PickleReader(IOFormatEnabled, FileReader, PickleHandler):

55
bonobo/nodes/throttle.py Normal file
View File

@ -0,0 +1,55 @@
import threading
import time
from bonobo.config import Configurable, ContextProcessor, Method, Option
class RateLimitBucket(threading.Thread):
daemon = True
@property
def stopped(self):
return self._stop_event.is_set()
def __init__(self, initial=1, period=1, amount=1):
super(RateLimitBucket, self).__init__()
self.semaphore = threading.BoundedSemaphore(initial)
self.amount = amount
self.period = period
self._stop_event = threading.Event()
def stop(self):
self._stop_event.set()
def run(self):
while not self.stopped:
time.sleep(self.period)
for _ in range(self.amount):
self.semaphore.release()
def wait(self):
return self.semaphore.acquire()
class RateLimited(Configurable):
handler = Method()
initial = Option(int, positional=True, default=1)
period = Option(int, positional=True, default=1)
amount = Option(int, positional=True, default=1)
@ContextProcessor
def bucket(self, context):
print(context)
bucket = RateLimitBucket(self.initial, self.amount, self.period)
bucket.start()
print(bucket)
yield bucket
bucket.stop()
bucket.join()
def call(self, bucket, *args, **kwargs):
print(bucket, args, kwargs)
bucket.wait()
return self.handler(*args, **kwargs)

55
bonobo/settings.py
View File

@ -5,6 +5,10 @@ from bonobo.errors import ValidationError
def to_bool(s):
if s is None:
return False
if type(s) is bool:
return s
if len(s):
if s.lower() in ('f', 'false', 'n', 'no', '0'):
return False
@ -13,7 +17,18 @@ def to_bool(s):
class Setting:
def __init__(self, name, default=None, validator=None):
__all__ = {}
@classmethod
def clear_all(cls):
for setting in Setting.__all__.values():
setting.clear()
def __new__(cls, name, *args, **kwargs):
Setting.__all__[name] = super().__new__(cls)
return Setting.__all__[name]
def __init__(self, name, default=None, validator=None, formatter=None):
self.name = name
if default:
@ -21,15 +36,14 @@ class Setting:
else:
self.default = lambda: None
if validator:
self.validator = validator
else:
self.validator = None
self.validator = validator
self.formatter = formatter
def __repr__(self):
return '<Setting {}={!r}>'.format(self.name, self.value)
return '<Setting {}={!r}>'.format(self.name, self.get())
def set(self, value):
value = self.formatter(value) if self.formatter else value
if self.validator and not self.validator(value):
raise ValidationError('Invalid value {!r} for setting {}.'.format(value, self.name))
self.value = value
@ -38,21 +52,35 @@ class Setting:
try:
return self.value
except AttributeError:
self.value = self.default()
value = os.environ.get(self.name, None)
if value is None:
value = self.default()
self.set(value)
return self.value
def clear(self):
try:
del self.value
except AttributeError:
pass
# Debug/verbose mode.
DEBUG = to_bool(os.environ.get('DEBUG', 'f'))
DEBUG = Setting('DEBUG', formatter=to_bool, default=False)
# Profile mode.
PROFILE = to_bool(os.environ.get('PROFILE', 'f'))
PROFILE = Setting('PROFILE', formatter=to_bool, default=False)
# Quiet mode.
QUIET = to_bool(os.environ.get('QUIET', 'f'))
QUIET = Setting('QUIET', formatter=to_bool, default=False)
# Logging level.
LOGGING_LEVEL = logging.DEBUG if DEBUG else logging.INFO
LOGGING_LEVEL = Setting(
'LOGGING_LEVEL',
formatter=logging._checkLevel,
validator=logging._checkLevel,
default=lambda: logging.DEBUG if DEBUG.get() else logging.INFO
)
# Input/Output format for transformations
IOFORMAT_ARG0 = 'arg0'
@ -67,5 +95,8 @@ IOFORMAT = Setting('IOFORMAT', default=IOFORMAT_KWARGS, validator=IOFORMATS.__co
def check():
if DEBUG and QUIET:
if DEBUG.get() and QUIET.get():
raise RuntimeError('I cannot be verbose and quiet at the same time.')
clear_all = Setting.clear_all

11
bonobo/structs/graphs.py
View File

@ -1,3 +1,5 @@
from copy import copy
from bonobo.constants import BEGIN
@ -62,6 +64,15 @@ class Graph:
return self
def copy(self):
g = Graph()
g.edges = copy(self.edges)
g.named = copy(self.named)
g.nodes = copy(self.nodes)
return g
@property
def topologically_sorted_indexes(self):
"""Iterate in topological order, based on networkx's topological_sort() function.

18
bonobo/structs/inputs.py
View File

@ -77,6 +77,12 @@ class Input(Queue, Readable, Writable):
return Queue.put(self, data, block, timeout)
def _decrement_runlevel(self):
if self._runlevel == 1:
self.on_finalize()
self._runlevel -= 1
self.on_end()
def get(self, block=True, timeout=None):
if not self.alive:
raise InactiveReadableError('Cannot get() on an inactive {}.'.format(Readable.__name__))
@ -84,13 +90,7 @@ class Input(Queue, Readable, Writable):
data = Queue.get(self, block, timeout)
if data == END:
if self._runlevel == 1:
self.on_finalize()
self._runlevel -= 1
# callback
self.on_end()
self._decrement_runlevel()
if not self.alive:
raise InactiveReadableError(
@ -100,6 +100,10 @@ class Input(Queue, Readable, Writable):
return data
def shutdown(self):
while self._runlevel >= 1:
self._decrement_runlevel()
def empty(self):
self.mutex.acquire()
while self._qsize() and self.queue[0] == END:

6
bonobo/util/collections.py Normal file
View File

@ -0,0 +1,6 @@
import bisect
class sortedlist(list):
def insort(self, x):
bisect.insort(self, x)

116
bonobo/util/inspect.py Normal file
View File

@ -0,0 +1,116 @@
from collections import namedtuple
def isconfigurabletype(mixed):
"""
Check if the given argument is an instance of :class:`bonobo.config.ConfigurableMeta`, meaning it has all the
plumbery necessary to build :class:`bonobo.config.Configurable`-like instances.
:param mixed:
:return: bool
"""
from bonobo.config.configurables import ConfigurableMeta
return isinstance(mixed, ConfigurableMeta)
def isconfigurable(mixed):
"""
Check if the given argument is an instance of :class:`bonobo.config.Configurable`.
:param mixed:
:return: bool
"""
from bonobo.config.configurables import Configurable
return isinstance(mixed, Configurable)
def isoption(mixed):
"""
Check if the given argument is an instance of :class:`bonobo.config.Option`.
:param mixed:
:return: bool
"""
from bonobo.config.options import Option
return isinstance(mixed, Option)
def ismethod(mixed):
"""
Check if the given argument is an instance of :class:`bonobo.config.Method`.
:param mixed:
:return: bool
"""
from bonobo.config.options import Method
return isinstance(mixed, Method)
def iscontextprocessor(x):
"""
Check if the given argument is an instance of :class:`bonobo.config.ContextProcessor`.
:param mixed:
:return: bool
"""
from bonobo.config.processors import ContextProcessor
return isinstance(x, ContextProcessor)
def istype(mixed):
"""
Check if the given argument is a type object.
:param mixed:
:return: bool
"""
return isinstance(mixed, type)
ConfigurableInspection = namedtuple(
'ConfigurableInspection', [
'type',
'instance',
'options',
'processors',
'partial',
]
)
ConfigurableInspection.__enter__ = lambda self: self
ConfigurableInspection.__exit__ = lambda *exc_details: None
def inspect_node(mixed, *, _partial=None):
"""
If the given argument is somehow a :class:`bonobo.config.Configurable` object (either a subclass, an instance, or
a partially configured instance), then it will return a :class:`ConfigurableInspection` namedtuple, used to inspect
the configurable metadata (options). If you want to get the option values, you don't need this, it is only usefull
to perform introspection on a configurable.
If it's not looking like a configurable, it will raise a :class:`TypeError`.
:param mixed:
:return: ConfigurableInspection
:raise: TypeError
"""
if isconfigurabletype(mixed):
inst, typ = None, mixed
elif isconfigurable(mixed):
inst, typ = mixed, type(mixed)
elif hasattr(mixed, 'func'):
return inspect_node(mixed.func, _partial=(mixed.args, mixed.keywords))
else:
raise TypeError(
'Not a Configurable, nor a Configurable instance and not even a partially configured Configurable. Check your inputs.'
)
return ConfigurableInspection(
typ,
inst,
list(typ.__options__),
list(typ.__processors__),
_partial,
)

7
docs/changelog.rst
View File

@ -1,6 +1,13 @@
Changelog
=========
v.0.4.3 - 16 july 2017
::::::::::::::::::::::
* #113 - Add flush() method to IOBuffer (Vitalii Vokhmin)
* Dependencies updated.
* Minor project artifacts updated.
v.0.4.2 - 18 june 2017
::::::::::::::::::::::

2
docs/tutorial/tut02.rst
View File

@ -59,7 +59,7 @@ available in **Bonobo**'s repository:
.. code-block:: shell-session
$ curl https://raw.githubusercontent.com/python-bonobo/bonobo/master/bonobo/examples/datasets/coffeeshops.txt > `python -c 'import bonobo; print(bonobo.get_examples_path("datasets/coffeeshops.txt"))'`
$ curl https://raw.githubusercontent.com/python-bonobo/bonobo/master/bonobo/examples/datasets/coffeeshops.txt > `python3 -c 'import bonobo; print(bonobo.get_examples_path("datasets/coffeeshops.txt"))'`
.. note::

4
requirements-dev.txt
View File

@ -21,8 +21,8 @@ pygments==2.2.0
pytest-cov==2.5.1
pytest-sugar==0.8.0
pytest-timeout==1.2.0
pytest==3.1.2
python-dateutil==2.6.0
pytest==3.1.3
python-dateutil==2.6.1
pytz==2017.2
requests==2.18.1
six==1.10.0

4
requirements-docker.txt
View File

@ -1,6 +1,6 @@
-e .[docker]
appdirs==1.4.3
bonobo-docker==0.2.9
bonobo-docker==0.2.11
certifi==2017.4.17
chardet==3.0.4
colorama==0.3.9
@ -15,6 +15,6 @@ pyparsing==2.2.0
pytz==2017.2
requests==2.18.1
six==1.10.0
stevedore==1.23.0
stevedore==1.24.0
urllib3==1.21.1
websocket-client==0.44.0

4
requirements-jupyter.txt
View File

@ -1,7 +1,7 @@
-e .[jupyter]
appnope==0.1.0
bleach==2.0.0
decorator==4.0.11
decorator==4.1.1
entrypoints==0.2.3
html5lib==0.999999999
ipykernel==4.6.1
@ -26,7 +26,7 @@ pickleshare==0.7.4
prompt-toolkit==1.0.14
ptyprocess==0.5.2
pygments==2.2.0
python-dateutil==2.6.0
python-dateutil==2.6.1
pyzmq==16.0.2
qtconsole==4.3.0
simplegeneric==0.8.1

2
requirements.txt
View File

@ -12,5 +12,5 @@ pyparsing==2.2.0
pytz==2017.2
requests==2.18.1
six==1.10.0
stevedore==1.23.0
stevedore==1.24.0
urllib3==1.21.1

56
tests/config/test_configurables.py
View File

@ -2,12 +2,17 @@ import pytest
from bonobo.config.configurables import Configurable
from bonobo.config.options import Option
from bonobo.util.inspect import inspect_node
class NoOptConfigurable(Configurable):
pass
class MyConfigurable(Configurable):
required_str = Option(str, required=True)
required_str = Option(str)
default_str = Option(str, default='foo')
integer = Option(int)
integer = Option(int, required=False)
class MyHarderConfigurable(MyConfigurable):
@ -25,14 +30,20 @@ class MyConfigurableUsingPositionalOptions(MyConfigurable):
def test_missing_required_option_error():
with inspect_node(MyConfigurable()) as ni:
assert ni.partial
with pytest.raises(TypeError) as exc:
MyConfigurable()
MyConfigurable(_final=True)
assert exc.match('missing 1 required option:')
def test_missing_required_options_error():
with inspect_node(MyHarderConfigurable()) as ni:
assert ni.partial
with pytest.raises(TypeError) as exc:
MyHarderConfigurable()
MyHarderConfigurable(_final=True)
assert exc.match('missing 2 required options:')
@ -50,6 +61,10 @@ def test_extraneous_options_error():
def test_defaults():
o = MyConfigurable(required_str='hello')
with inspect_node(o) as ni:
assert not ni.partial
assert o.required_str == 'hello'
assert o.default_str == 'foo'
assert o.integer == None
@ -57,6 +72,10 @@ def test_defaults():
def test_str_type_factory():
o = MyConfigurable(required_str=42)
with inspect_node(o) as ni:
assert not ni.partial
assert o.required_str == '42'
assert o.default_str == 'foo'
assert o.integer == None
@ -64,6 +83,10 @@ def test_str_type_factory():
def test_int_type_factory():
o = MyConfigurable(required_str='yo', default_str='bar', integer='42')
with inspect_node(o) as ni:
assert not ni.partial
assert o.required_str == 'yo'
assert o.default_str == 'bar'
assert o.integer == 42
@ -71,6 +94,10 @@ def test_int_type_factory():
def test_bool_type_factory():
o = MyHarderConfigurable(required_str='yes', also_required='True')
with inspect_node(o) as ni:
assert not ni.partial
assert o.required_str == 'yes'
assert o.default_str == 'foo'
assert o.integer == None
@ -79,6 +106,10 @@ def test_bool_type_factory():
def test_option_resolution_order():
o = MyBetterConfigurable()
with inspect_node(o) as ni:
assert not ni.partial
assert o.required_str == 'kaboom'
assert o.default_str == 'foo'
assert o.integer == None
@ -86,3 +117,20 @@ def test_option_resolution_order():
def test_option_positional():
o = MyConfigurableUsingPositionalOptions('1', '2', '3', required_str='hello')
with inspect_node(o) as ni:
assert not ni.partial
assert o.first == '1'
assert o.second == '2'
assert o.third == '3'
assert o.required_str == 'hello'
assert o.default_str == 'foo'
assert o.integer is None
def test_no_opt_configurable():
o = NoOptConfigurable()
with inspect_node(o) as ni:
assert not ni.partial

74
tests/config/test_methods.py
View File

@ -1,7 +1,5 @@
import pytest
from bonobo.config import Configurable, Method, Option
from bonobo.errors import ConfigurationError
from bonobo.util.inspect import inspect_node
class MethodBasedConfigurable(Configurable):
@ -13,23 +11,73 @@ class MethodBasedConfigurable(Configurable):
self.handler(*args, **kwargs)
def test_one_wrapper_only():
with pytest.raises(ConfigurationError):
def test_multiple_wrapper_suppored():
class TwoMethods(Configurable):
h1 = Method(required=True)
h2 = Method(required=True)
class TwoMethods(Configurable):
h1 = Method()
h2 = Method()
with inspect_node(TwoMethods) as ci:
assert ci.type == TwoMethods
assert not ci.instance
assert len(ci.options) == 2
assert not len(ci.processors)
assert not ci.partial
@TwoMethods
def OneMethod():
pass
with inspect_node(OneMethod) as ci:
assert ci.type == TwoMethods
assert not ci.instance
assert len(ci.options) == 2
assert not len(ci.processors)
assert ci.partial
@OneMethod
def transformation():
pass
with inspect_node(transformation) as ci:
assert ci.type == TwoMethods
assert ci.instance
assert len(ci.options) == 2
assert not len(ci.processors)
assert not ci.partial
def test_define_with_decorator():
calls = []
@MethodBasedConfigurable
def Concrete(self, *args, **kwargs):
def my_handler(*args, **kwargs):
calls.append((args, kwargs, ))
Concrete = MethodBasedConfigurable(my_handler)
assert callable(Concrete.handler)
assert Concrete.handler == my_handler
with inspect_node(Concrete) as ci:
assert ci.type == MethodBasedConfigurable
assert ci.partial
t = Concrete('foo', bar='baz')
assert callable(t.handler)
assert len(calls) == 0
t()
assert len(calls) == 1
def test_late_binding_method_decoration():
calls = []
@MethodBasedConfigurable(foo='foo')
def Concrete(*args, **kwargs):
calls.append((args, kwargs, ))
assert callable(Concrete.handler)
t = Concrete('foo', bar='baz')
t = Concrete(bar='baz')
assert callable(t.handler)
assert len(calls) == 0
@ -43,7 +91,7 @@ def test_define_with_argument():
def concrete_handler(*args, **kwargs):
calls.append((args, kwargs, ))
t = MethodBasedConfigurable('foo', bar='baz', handler=concrete_handler)
t = MethodBasedConfigurable(concrete_handler, 'foo', bar='baz')
assert callable(t.handler)
assert len(calls) == 0
t()
@ -71,7 +119,7 @@ def test_inheritance_then_decorate():
pass
@Inheriting
def Concrete(self, *args, **kwargs):
def Concrete(*args, **kwargs):
calls.append((args, kwargs, ))
assert callable(Concrete.handler)

66
tests/config/test_methods_partial.py Normal file
View File

@ -0,0 +1,66 @@
from unittest.mock import MagicMock
from bonobo.config import Configurable, ContextProcessor, Method, Option
from bonobo.util.inspect import inspect_node
class Bobby(Configurable):
handler = Method()
handler2 = Method()
foo = Option(positional=True)
bar = Option(required=False)
@ContextProcessor
def think(self, context):
yield 'different'
def call(self, think, *args, **kwargs):
self.handler('1', *args, **kwargs)
self.handler2('2', *args, **kwargs)
def test_partial():
C = Bobby
# inspect the configurable class
with inspect_node(C) as ci:
assert ci.type == Bobby
assert not ci.instance
assert len(ci.options) == 4
assert len(ci.processors) == 1
assert not ci.partial
# instanciate a partial instance ...
f1 = MagicMock()
C = C(f1)
with inspect_node(C) as ci:
assert ci.type == Bobby
assert not ci.instance
assert len(ci.options) == 4
assert len(ci.processors) == 1
assert ci.partial
assert ci.partial[0] == (f1, )
assert not len(ci.partial[1])
# instanciate a more complete partial instance ...
f2 = MagicMock()
C = C(f2)
with inspect_node(C) as ci:
assert ci.type == Bobby
assert not ci.instance
assert len(ci.options) == 4
assert len(ci.processors) == 1
assert ci.partial
assert ci.partial[0] == (f1, f2, )
assert not len(ci.partial[1])
c = C('foo')
with inspect_node(c) as ci:
assert ci.type == Bobby
assert ci.instance
assert len(ci.options) == 4
assert len(ci.processors) == 1
assert not ci.partial

16
tests/config/test_services.py
View File

@ -3,7 +3,7 @@ import time
import pytest
from bonobo.config import Configurable, Container, Exclusive, Service
from bonobo.config import Configurable, Container, Exclusive, Service, requires
from bonobo.config.services import validate_service_name
@ -94,3 +94,17 @@ def test_exclusive():
'hello', '0 0', '0 1', '0 2', '0 3', '0 4', '1 0', '1 1', '1 2', '1 3', '1 4', '2 0', '2 1', '2 2', '2 3',
'2 4', '3 0', '3 1', '3 2', '3 3', '3 4', '4 0', '4 1', '4 2', '4 3', '4 4'
]
def test_requires():
vcr = VCR()
services = Container(output=vcr.append)
@requires('output')
def append(out, x):
out(x)
svcargs = services.args_for(append)
assert len(svcargs) == 1
assert svcargs[0] == vcr.append

20
tests/structs/test_graphs.py
View File

@ -71,3 +71,23 @@ def test_graph_topological_sort():
assert g.topologically_sorted_indexes.index(3) < g.topologically_sorted_indexes.index(4)
assert g[3] == sentinel.b1
assert g[4] == sentinel.b2
def test_copy():
g1 = Graph()
g2 = g1.copy()
assert g1 is not g2
assert len(g1) == 0
assert len(g2) == 0
g1.add_chain([])
assert len(g1) == 1
assert len(g2) == 0
g2.add_chain([], identity)
assert len(g1) == 1
assert len(g2) == 2

1
tests/test_basics.py
View File

@ -5,6 +5,7 @@ import pytest
import bonobo
from bonobo.config.processors import ContextCurrifier
from bonobo.constants import NOT_MODIFIED
from bonobo.util.inspect import inspect_node
def test_count():

19
tests/test_commands.py
View File

@ -1,3 +1,4 @@
import os
import runpy
import sys
from unittest.mock import patch
@ -10,10 +11,12 @@ from bonobo.commands import entrypoint
def runner_entrypoint(*args):
""" Run bonobo using the python command entrypoint directly (bonobo.commands.entrypoint). """
return entrypoint(list(args))
def runner_module(*args):
""" Run bonobo using the bonobo.__main__ file, which is equivalent as doing "python -m bonobo ..."."""
with patch.object(sys, 'argv', ['bonobo', *args]):
return runpy.run_path(__main__.__file__, run_name='__main__')
@ -70,6 +73,22 @@ def test_run_path(runner, capsys):
assert out[2].startswith('Baz ')
@all_runners
def test_install_requirements_for_dir(runner):
dirname = get_examples_path('types')
with patch('bonobo.commands.run._install_requirements') as install_mock:
runner('run', '--install', dirname)
install_mock.assert_called_once_with(os.path.join(dirname, 'requirements.txt'))
@all_runners
def test_install_requirements_for_file(runner):
dirname = get_examples_path('types')
with patch('bonobo.commands.run._install_requirements') as install_mock:
runner('run', '--install', os.path.join(dirname, 'strings.py'))
install_mock.assert_called_once_with(os.path.join(dirname, 'requirements.txt'))
@all_runners
def test_version(runner, capsys):
runner('version')

63
tests/test_settings.py Normal file
View File

@ -0,0 +1,63 @@
import logging
from os import environ
from unittest.mock import patch
import pytest
from bonobo import settings
TEST_SETTING = 'TEST_SETTING'
def test_to_bool():
assert not settings.to_bool('')
assert not settings.to_bool('FALSE')
assert not settings.to_bool('NO')
assert not settings.to_bool('0')
assert settings.to_bool('yup')
assert settings.to_bool('True')
assert settings.to_bool('yes')
assert settings.to_bool('1')
def test_setting():
s = settings.Setting(TEST_SETTING)
assert s.get() is None
with patch.dict(environ, {TEST_SETTING: 'hello'}):
assert s.get() is None
s.clear()
assert s.get() == 'hello'
s = settings.Setting(TEST_SETTING, default='nope')
assert s.get() is 'nope'
with patch.dict(environ, {TEST_SETTING: 'hello'}):
assert s.get() == 'nope'
s.clear()
assert s.get() == 'hello'
def test_default_settings():
settings.clear_all()
assert settings.DEBUG.get() == False
assert settings.PROFILE.get() == False
assert settings.QUIET.get() == False
assert settings.LOGGING_LEVEL.get() == logging._checkLevel('INFO')
with patch.dict(environ, {'DEBUG': 't'}):
settings.clear_all()
assert settings.LOGGING_LEVEL.get() == logging._checkLevel('DEBUG')
settings.clear_all()
def test_check():
settings.check()
with patch.dict(environ, {'DEBUG': 't', 'PROFILE': 't', 'QUIET': 't'}):
settings.clear_all()
with pytest.raises(RuntimeError):
settings.check()
settings.clear_all()