dfee / forge

forge (python signatures) for fun and profit

forge (python) signatures for fun and profit

forge is an elegant Python package for revising function signatures at runtime. This libraries aim is to help you write better, more literate code with less boilerplate.

Installation

forge is a Python-only package hosted on PyPI for Python 3.5+.

The recommended installation method is pip-installing into a virtualenv:

$ pip install python-forge

Example

Consider a library like requests that provides a useful API for performing HTTP requests. Every HTTP method has it's own function which is a thin wrapper around requests.Session.request. The code is a little more than 150 lines, with about 90% of that being boilerplate. Using forge we can get that back down to about 10% it's current size, while increasing the literacy of the code.

import forge
import requests

request = forge.copy(requests.Session.request, exclude='self')(requests.request)

def with_method(method):
    revised = forge.modify(
        'method', default=method, bound=True,
        kind=forge.FParameter.POSITIONAL_ONLY,
    )(request)
    revised.__name__ = method.lower()
    return revised

post = with_method('POST')
get = with_method('GET')
put = with_method('PUT')
delete = with_method('DELETE')
options = with_method('OPTIONS')
head = with_method('HEAD')
patch = with_method('PATCH')

So what happened? The first thing we did was create an alternate request function to replace requests.request that provides the exact same functionality but makes its parameters explicit:

# requests.get() looks like this:
assert forge.repr_callable(requests.get) == 'get(url, params=None, **kwargs)'

# our get() calls the same code, but looks like this:
assert forge.repr_callable(get) == (
    'get(url, params=None, data=None, headers=None, cookies=None, '
        'files=None, auth=None, timeout=None, allow_redirects=True, '
        'proxies=None, hooks=None, stream=None, verify=None, cert=None, '
        'json=None'
    ')'
)

Next, we built a factory function with_method that creates new functions which make HTTP requests with the proper HTTP verb. Because the method parameter is bound, it won't show up it is removed from the resulting functions signature. Of course, the signature of these generated functions remains explicit, let's try it out:

response = get('http://google.com')
assert 'Feeling Lucky' in response.text

You can review the alternate code (the actual implementation) by visiting the code for requests.api.

Project information

forge is released under the MIT license, its documentation lives at Read the Docs, the code on GitHub, and the latest release on PyPI. It’s rigorously tested on Python 3.6+ and PyPy 3.5+.

forge is authored by Devin Fee. Other contributors are listed under https://github.com/dfee/forge/graphs/contributors.

forge's Issues

Change signature depending on dict input

Awesome package! I am trying to use it but I can't figure out how to do the following thing (and I suspect I should be able to do so).

I'll give the sort version and a longer version of what I actually want to do.

Goal

# This is the function I have
def f(changing_params, fixed_params):
    params = {**changing_params, **fixed_params}
    return do_stuff(**params)

changing_params ['x', 'a', 'b']
fixed_params = ['c']

forge_please_fix_my_problems(f, changing_params, fixed_params)
# which returns the following function
def _forged_f(x, a, b, fixed_params):
    params = {'x': x, 'a': a, 'b': b, **fixed_params}
    return do_stuff(**params)

With some more background

Imagine I need to call the following function in one of my own functions

def do_stuff(x, a, b, c):
    return a * x**c + b

I want to create functions where some parameters are fixed and other changing, by modifying the following

def f(val, key, changing_params, fixed_params):
    params = {key: val, **changing_params, **fixed_params}
    return do_stuff(**params)

I want to create functions (from f) for the following "settings" that I will parse with functools.partial:

from itertools import product
def named_product(**items):
    names = items.keys()
    vals = items.values()
    return [dict(zip(names, res)) for res in product(*vals)]

fixed_params = dict(c=2)
combinations = dict(a=[0, 1], b=[0, 1])
variable_params = named_product(combinations)
print(variable_params)

[{'a': 0, 'b': 0},
 {'a': 0, 'b': 1},
 {'a': 1, 'b': 0},
 {'a': 1, 'b': 1}]

So here I want a bunch of functions that always have c=2, and one has a=0, b=0, the next a=0, b=1, etc.

Essentially I want to create the following functions:

from functools import partial
fs = [partial(f, key='x', changing_params=p, fixed_params=fixed_params)
        for p in changing_params]

But with the following signatures:

def _f(x, a=0, b=0, fixed_params):
    ....

def _f(x, a=0, b=1, fixed_params):
    ....

So in order to do that I want forge to generate me a function that does the following

magic_forge(f, key='x', changing_params=combinations.keys(), fixed_params=fixed_params)
=>
def _f(x, a, b, fixed_params):
    ....

I am sorry if this is needlessly complicated, but I did my best to keep it as minimal as possible.

Add PyPy 6.0.0 to Travis CI

Support for pypy/pypy3 v6.0+ python #9542 on travis-ci

Add `reflect` functionality to copy parameters from other callable.

"Copying parameters from another callable" can be achieved already using:

def foo(a, b, c=0):
    pass

@forge.sign(**forge.FSignature.from_callable(foo))
def bar(**kwargs):
    return foo(**kwargs)

assert foo.stringify_callable(bar) == 'bar(a, b, c=0)'

However, this syntax is somewhat inconvenient, and could instead be a function on its own, copy.

def foo(a, b, c=0):
    pass

@forge.copy(foo)
def bar(**kwargs):
    return foo(**kwargs)

assert foo.stringify_callable(bar) == 'bar(a, b, c=0)'

If a user wants to include only certain parameters:

def foo(a, b, c=0):
    pass

@forge.copy(foo, include=['a', 'c'])
def bar(**kwargs):
    return foo(**kwargs)

assert foo.stringify_callable(bar) == 'bar(a, c=0)'

Or, if a user wants to instead exclude certain parameters:

def foo(a, b, c=0):
    pass

@forge.copy(foo, exclude=['b'])
def bar(**kwargs):
    return foo(**kwargs)

assert foo.stringify_callable(bar) == 'bar(a, c=0)'

Composition of `copy`/`synthetize` revision with `modify` revision does not track _interface name_ correctly

Thank you for this great library and for the huge effort that has been made on documentation.

I'm experimenting a minor problem (I surely have workaround for it, but isn't it the whole point of forge to simplify our life?). I actually think that it is a bug, as it is in my opinion not the expected behavior. It concerns composition of revisions, and more precisely composition of copy (or synthetize) with modify. Doing so, it seems that the flattening process of the composition of revisions do not track the interface name of parameters correctly (at least, not as I expected).

MWE

Here is a minimal (non-)working example:

import forge
def f(a):
    pass
@forge.copy(f)
@forge.modify("b", name="a")
def g(b):
    pass

The expected result (in my opinion) would be that function g is well defined with signature g(a), but instead I obtain a TypeError with the message Missing requisite mapping to non-default positional or keyword parameter 'b'. I understand that the modify revision has not been considered when flattening.

Other tests

I did some tests and it appears that, contrary to modify (and probably other revisions), synthetize suffers the same issues. (This actually makes sense since, just as with copy, the signature set by synthetise is not based on the current signature of the function, contrary to the ones defined by modify and other revisions that alter the current one.) Here is the example:

@forge.modify("b", name="a")
@forge.modify("c", name="b")
def g(c):
    pass
#works fine: g well-defined is defined with signature g(a)

@forge.synthetize(*forge.fsignature(f))
@forge.modify("b", name="a")
def g(b):
    pass
#fails similarly as in the minimal example given above

Expected feature

In my opinion, copy should use the signature of the preceding revisions to build its correspondences with the copied signature, instead of directly look at the inner function. Hence, the above examples should work fine.

Thanks.

Solve the problem of the overzealous programmer

... who copied all the signatures. As the forge docs say the lazy programmer will use kwargs at times to avoid repeating the same signature again and again, make forwarding args simpler etc. But the stakhanovist programmer, with the same concerns this package was designed to solve, has copied the same signature all over again. I would like to refactor his code to define the common signature in a single place, without having to modify his code. For instance

def f(a):
    return a

def g(a):
    return -a

Should become

a_sig = Signature(a)
@apply_sig(a_sig)
def f():
     return a
def g():
    return -a

Or some such, this is fantasy code. Of course this makes more sense with a longer signature and more functions, but the idea should be clear. I am afraid this requires eval sorcery, but I thought I'd ask first. Thanks!

Naming conflict with positional arguments and others.

First of all, thanks for amazing library.

I found positional arguments conflict to others, and implementation tracked all parameter name using set.

>>> import forge
>>> sigs = [forge.pos('a'), forge.kwo('a')]
>>> def hello(*args, **kargs):
...     pass
...
>>> forge.sign(*sigs)(hello)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/yan/ITF/kfra/.venv/lib/python3.6/site-packages/forge/_revision.py", line 330, in __call__
    next_.validate()
  File "/Users/yan/ITF/kfra/.venv/lib/python3.6/site-packages/forge/_signature.py", line 1328, in validate
    format(current.name)
ValueError: Received multiple parameters with name 'a'

imho, user does not care about the name of positional argument since they could not use it like keyword parameter. So how about letting user use two same name for positional argument?

Thanks.

Request for use of `forge` name on PyPI

Hello!

I attempted to send an email to the email listed on your PyPI listing but didn't hear back.

Would you be willing to release this name to me on PyPI? I have a bit of code that I use in my typing library (Typical) which I'd like to break out into its own package, and I was hoping to use the name forge, as it describes the functionality of the library quite well, which will be a tool for dynamically writing and compiling (aka "forging") new code at runtime.

Publish python-forge on conda-forge

So I heard you like forging things...

I have some packages that use python-forge that I would like to publish on conda-forge so they can be installed with the conda package manager. To do that, I will need this package published there first.

I am willing to set up a conda recipe for this (see conda-forge/staged-recipes#14713), I'm just creating this issue for reference.