morepath / dectate

Decorator based advanced configuration engine.

Morepath: Python web microframework with super powers

Morepath is a Python web framework. An application consists of models. Each type of model is published on a URL path. Content is exposed to the web using views.

Documentation.

dectate's Issues

remove deprecated autocommit

Remove autocommit which was deprecated.

multiple actions with different config factories

If one action declares one factory in its config for a particular named config entry and another action declares another, that should be an error.

It should also be an error if there's an inconsistency between factory_arguments and action config, or an internal inconsistency between factory_arguments.

override composite directives in same app

If you have a composite directive that generates sub-actions and sub-actions can also be generated directly by a directive, then currently this results in a conflict. Investigate whether we could instead let the directly specified actions override the implicit sub-actions.

how to deal with keyword arguments for filters

Some directives use keyword arguments (**kw), which can then be stored as a dictionary on the action. So filter_name is insufficient - the true value is on a subobject.

a query tool mode to dump a fully rendered action with all parameters

Show full actions with all parameters like done in logging.

Remove deprecated auto_query_tool

Remove auto_query_tool which was deprecated.

make topological sort stable

It would be nice if topological sort were stable. I don't think it is now entirely, and this can lead to different order of execution of directives and thus different order of execution of registries. This can lead to subtle bugs, see #3

We should in this case also use this stable topological sort in Morepath itself.

finalize configuration with app instance

Is it possible to be able to register some kind of finalizers for directives that get called with the app instance as soon as an application is instantiated? This would allow some configuration to be done that really needs the app instance, like what we need for this issue:

morepath/morepath#158

functional style decorators proof of concept

Here is some proof of concept code I wrote that demonstrates how Dectate might work if we use a more functional way to define directives, instead of using the action subclasses we do now. The idea is to make the signature of the directive be the correct signature right away if we generated code or alternatively used a library like wrapt.

This does require us to give up on the current with statement as that allows you to define a subset of functions. This could be replaced by pointing people in the direction of functools.partial.

class App(object):
    registered = []

    @classmethod
    def commit(cls):
        for perform, config, obj in cls.registered:
            kw = {}
            for key, value in config.items():
                for name, factory in config.items():
                    c = getattr(cls, name, None)
                    if c is None:
                        c = factory()
                        setattr(cls, name, c)
                    kw[name] = c
            perform(obj, **kw)


def action(config):
    def get_directive(func):
        def wrapper(cls, *args, **kw):
            info = func(cls, *args, **kw)

            def register(obj):
                cls.registered.append(
                    (info.perform, get_directive.config, obj))
                return obj
            return register
        return classmethod(wrapper)
    get_directive.config = config
    return get_directive


class Info(object):
    def __init__(self, perform):
        self.perform = perform


class MyApp(App):
    @action(config={'r': dict})
    def mydecorator(cls, message):
        def perform(obj, r):
            r[message] = obj
        return Info(perform)


@MyApp.mydecorator('foo')
def myfunction():
    return "the function"


MyApp.commit()


assert MyApp.r == {'foo': myfunction}

better error if factory fails

If the factory construction fails, for instance due to a type error, we should give a more informative error so we can trace which Action has the problem.

mention importscan in docs

document config and dectate class attributes for App

Because code that uses the registries actually uses at least the config attribute.

a composite query may result in multiple results but generated a single place

a composite query can result in multiple actions as results but they're both generated in a single
place. How do we report them without duplicates?

test_directive_repr fails on Python 3

In test_directive.py, the value of repr(MyApp.foo) is:

<bound method DirectiveDirective.__call__.<locals>.method of 
<class 'dectate.tests.test_directive.test_directive_repr.<locals>.MyApp'>

make topological_sort part of the public API

While not part of the core mission of Dectate we should make topological_sort a part of the official API so that Morepath can import it. I've been wondering vaguely to put it in its own package but I think it's too small for that to bother.

make app subclassing sort like Python classes

Use the same trick as Reg to make sure app extends sorts like Python classes in an mro to handle the "diamond" extension pattern in the same way Python does.

Moved over here from:

morepath/morepath#104

Phase out autocommit

With reference to morepath#392, after completing #18 we should phase out autocommit. I suggest we do it in two steps:

1. NOP autocommit and have it raise a DeprecationWarning
2. Finally remove autocommit.

filter_convert and group_class

MountAction in Morepath has an app attribute and a path attribute. But does filter_action go to PathAction for information? And how then to filter_convert app?

query tool formats

Add a --format option to the query tool that lets you select different output format. The current format is Python exception traceback inspired, but we can also add one where the full arguments of the directive are displayed like the logging facility already can.

a batching orderable directive

In Morepath we have, for instance in case of predicates, a situation were take the directive's configuration, stuff it into a separate registry, and then in after take that registry, sort what's in it (by information given to the directive), and finally flow it into the "real" registry.

It strikes me that we could generalize this into Dectate: collected dectate actions of a class are sorted against a sorting algorithm first before they are actually performed.

Let's analyze the use cases in Morepath:

• with predicates we also have the predicate fallback directive, which affects the same registration state. So more is going on than just storing a directive and storing.
• with template directories there is the whole hacky configurable passing in. The actually sorting of template directories takes place in the template_loader action.
• with tween factory the sorting only happens at the last moment during runtime in App.publish.

The interactions between different directives make the above generalization proposal insufficient. Perhaps we can instead generalize the registries that are used to record and sort information something that is specified directly by a special kind of action. So perhaps we need a generalized configuration
object to support this use case. Something like:

def sort_tweens(tweens):
    ... topological sort tweens by over and under ...
    ... could generalize it some more by specifying on what attributes
    ... to do topological sort instead

@App.directive('tween_factory')
class TweenFactoryAction(dectate.OrderedAction):
    config = {
        'tween_registry': dectate.OrderedRegistry(sort_tweens)
    }

    def __init__(self, under=None, over=None, name=None):
        global tween_factory_id
        self.under = under
        self.over = over
        if name is None:
            name = u'tween_factory_%s' % tween_factory_id
            tween_factory_id += 1
        self.name = name

    def identifier(self, tween_registry):
        return self.name

   # default perform that stores action and obj in OrderedRegistry

# elsewhere

config.tween_registry.get_sorted_values()

Even so this won't work completely for predicates, where multiple directives work together to construct the information needed. But we could still use this by storing predicate fallbacks in a supplementary registry.

directive directive confusing

When you use the directive directive, you need to make sure that the module it is in is imported before you scan any other code. This can be surprising, as you might expect it works like any other Dectate directive, where import order is immaterial.

Perhaps this implies the directive directive shouldn't be a directive at all. Directives are really like methods. Perhap we should install them like methods on the App class, like this:

class App(dectate.App):
     foo = dectate.directive(FooAction)

If we make this the way to install directives, we would have no more import issues, as it's impossible to use a new directive without importing its app class first.

What do you think, @href, @taschini?

actions applied to app/configurable

There are a few cases where it is useful to have access to the app or configurable in an action. Normally I'd like to avoid it though. Would it be possible to have a special config variable that if named
'app' will pass it in (but not create it)?

use newer with_metaclass from six

It should be possible to use a newer with_metaclass from six as we don't rely on Venusian magic that can get confused anymore. Try it out.

separation between config.py and app.py

The separation of concerns between config.py and app.py is not ideal. We test them together too. So perhaps we should just merge them.

Update supported Python versions

• drop support for Python versions < 3.6
• add support for Python versions 3.7, 3.8 (if missing)
• fix linting errors (if present)

If you are a beginner, and need to help for this issue, especially during Hacktoberfest, please comment here or reach out to me via https://jugmac00.github.io/

Change commit signature

Using the commit function I kind of wished it worked like this:

commit(App)

Or

commit(App1, App2)

Instead of

commit([App])

Really, really minor point, but since committing a single app during tests seems to be the main use case for this function I think we should get rid of the array:

def commit(*apps):
    # ...

query tool should have auto mode

Similarly to autocommit, we should have auto_query_tool which automatically queries all apps known.

how to create a module-based API

Dectate is written around app class driven APIs such as used in Morepath. This gives you special features like inheritance and overrides. But what if you want to give a decorator-based API to something that already exists? If there is no clear "registry" concept you can associate with an app class those features of Dectate are not useful anyway. You can still use Dectate for the directive ordering and conflict detection and so on.

You can in this case expose a module API like this:

class HiddenApp(dectate.App):
    pass

some_directive = HiddenApp.some_directive
commit = HiddenApp.commit

We should document this and ideally offer some automation so that this is done by itself. Or would it be enough to simply expose HiddenApp as a pretend module you can use?

query tool should allow querying app without directive

Right now it errors out if the directive is not known, but if you throw in all App subclasses automatically that can lead to a problem. Instead it should just report nothing found.

commit method on App

We should implement a commit method on App. Morepath overrides this anyway to do the recursive commit for mounted apps. This method should be explicitly documented as being intended to support such scenarios (by overriding in subclasses), as opposed to the more low-level dectate.commit which only commits a single class.

See:

morepath/morepath#383

query system

Would it be possible to devise a generic query API for an action database that lets you figure out the state of configuration? It would need to be aware of subclassing at least.

It should know about the action arguments, which right now requires some magic that's only in Reg.

simplify doc generation

Dectate contains a sphinx extension to help generate dectate documentation. It's unmaintainable grotty sphinx plugin code. It takes care of two things:

• get the docstring from __init__ of the action.
• get the argument list correct (the __init__ arguments without self).

We can fix the docstring issue by modifying the __doc__ setting behavior in app.py, for instance on the no-more-directive-directive branch:

    action_factory_doc = action_factory.__doc__
    action_factory_init_doc = None
    if hasattr(action_factory, '__init__'):
        action_factory_init_doc = action_factory.__init__.__doc__
    method.__doc__ = ((action_factory_doc or '') +
                      (action_factory_init_doc or ''))

Unfortunately the argument list is harder to get right. But we recently started to generate functions in Reg with the right signature, and we could do that in Dectate too. The drawback is that this generated function won't be very pretty in the debugger but it's probably not very comprehensible anyway due to the deferred nature of directive execution. If we could generate the correct list of arguments for our wrapper function, we could do away with the sphinx extension entirely. We'd do a bit of function generation when we import Morepath, but that seems tolerable.