Comments (19)
I'll buy a beer to the first one fixing this.
from forbiddenfruit.
I was trying to redefine the __mul__
operator for functions so it's do function composition instead. I was trying with forbidden fruit, but it wouldn't work because of this error. But I came up with a different and satisfactory solution, in case someone needs inspiration, I put it here:
import types
__all__ = ['composible']
# This decorator makes function composible by '*' operator with other functions
def composible(f):
class ComposibleFunction:
def __init__(self, f):
self.f = f
def __call__(self, *a, **kw):
return self.f(*a, **kw)
def __mul__(self, g):
return self.__class__(lambda *a, **kw: self.f(g(*a, **kw)))
return ComposibleFunction(f)
# Make all builtin function composible:
for b in (b for b in dir(__builtins__)
if type(getattr(__builtins__,b)) in (type, types.BuiltinMethodType)):
exec('global '+b+'\n'+b + '=composible(__builtins__.' + b + ')')
__all__.append(b)
# New functions need to be decorated
@composible
def foo(x): return x+1
@composible
def bar(x): return x*2
# This will output 7:
print((foo * bar * sum)([1, 2]))
from forbiddenfruit.
There seems to be some "not going through the method dictionary" issue here. Because:
>>> (2).__add__(3)
6
So the __add__
method is being switched out. But in expression form, it's not being used.
from forbiddenfruit.
That's fantastic. I was working on this exact feature RIGHT NOW. My guess is that the parser might be calling things differently for built-in types!
from forbiddenfruit.
wow!
from forbiddenfruit.
@jonathaneunice thank you so much for your bug report. I think I understood what's going on. Let me try to explain what's going on here, then we can try to find a solution to this problem! :)
When the python is processing some code and finds a sum operation like this:
var1 + var2
The system that evaluates the code (Python/ceval.c) will basically call the PyNumber_Add
function, which is part of the python API that handles the object abstract protocols (the magic methods -- you find it in Objects/abstract.c
). This function, unlike the regular getattr
will check for things declared in the tp_as_number
type slot before trying to access the instance dict (our dear __dict__
attribute).
So, basically, we need to find a way to patch the type
data and replace the ob_type->tp_as_{number,sequence}[nb_add]
method.
tl;dr: Basically, magic methods are resolved in a different way compared to the instance or class methods. There's a C array that contains a pointer to the actual implementation. Wich means that it's not actually present on the __slots__
dictionary.
I'm not sure if I'm 100% right, but it's my best shot so far. Thank you!
from forbiddenfruit.
@gabrielfalcao I assume he would not entirely approve. ;-)
As JavaScript and Ruby demonstrate, there's no fundamental reason why the base classes of a dynamic language cannot be extended. In JavaScript, libraries like underscore.js are very popular. But Python seems governed with a less exuberant desire for unapproved extensions, and more of a "monkey-patching considered harmful" view of the world.
from forbiddenfruit.
My efforts to kill this issue are happening here: https://github.com/clarete/forbiddenfruit/tree/fix/4
from forbiddenfruit.
Well, I was being kinda silly here. I completely forgot about the code generation. Python does not execute everything every time. For example, given the following python file:
1+1
2+3
The python compiler will generate the following code:
1 0 LOAD_CONST 4 (2)
3 POP_TOP
9 4 LOAD_CONST 5 (5)
7 POP_TOP
8 LOAD_CONST 3 (None)
11 RETURN_VALUE
Which means that no matter what, the integer sum operation will happen before my patch gets any chance to be applied.
from forbiddenfruit.
I'm not hugely disappointed that operations on primitive numeric types cannot be entirely intercepted. I expected that, optimizations being what they are. The ability to monkey-patch basic data classes such as dict
, str
, list
, and set
is still quite good magic.
from forbiddenfruit.
Also, @jonathaneunice we still can change all other operations that are not optimized. So I'll keep this bug open and I'll write more tests addressing other cases. Thank you and let me know if you have any other ideas!
from forbiddenfruit.
Working on getting the segfault fixed on Python 3.x
from forbiddenfruit.
Do we have any progress on this issue? What's the current status? I'd be glad to help with it.
from forbiddenfruit.
Hi @rizo, thanks for asking. That's the last thing I did: feae79a
If you're interested in helping, I can definitely provide you all the details you need to keep the train moving! :D
from forbiddenfruit.
Just found my way here while trying to override slice
to use a sentinel object rather than None
for the value of start, stop, and step when the user doesn't pass anything (background).
I tried running the following with forbiddenfruit revision feae79a and it didn't work:
from forbiddenfruit import curse
def simpletest(self):
return 'works'
curse(slice, 'simpletest', simpletest)
assert slice(None).simpletest() == 'works'
def myslice__init__(self, *args, **kw):
print('myslice__init__') # never called
self.start = self.stop = self.step = 42
curse(slice, '__init__', myslice__init__)
print(eval('slice(None).start')) # doesn't work :(
class myslice:
def __init__(self, *args, **kw):
self.start = self.stop = self.step = 42
def myslice__new__(cls, *args, **kw):
print('myslice__new__') # never called
inst = myslice()
curse(slice, '__new__', classmethod(myslice__new__))
print(eval('slice(None).start')) # doesn't work :(
Is there something I'm missing, or is this expected to not work yet?
Thanks for reading, and thanks for releasing forbiddenfruit, it's super cool.
from forbiddenfruit.
I've found this issue when looking for ability to compose functions like f @ g @ h
. While cursing object
with redefined __matmul__
method doesn't work, there is still yet another satisfactory solution, inspired by @jachymb :
from forbiddenfruit import curse
def go(arg, *t):
s = t[0](*arg)
for f in t[1:]:
s = f(s)
return (s,) # Composability: everybody loves monoids!
curse(tuple,"go", go)
usage: (x,y,z).go(f, g, h).go(f1,f2)
turns into f2(f1(h(g(f(x,y,z)))))
(3,4).\
go(lambda x,y: x*y
,lambda x: x**2).go(print)
144
(None,)
Of course, one can also define composable functions like (some_long_chain).then(f, additional_args_for_f)
. Iterables may also come into play...
Happy functional programming! :)
from forbiddenfruit.
@tomyo I guess you owe a beer to @alendit :)
from forbiddenfruit.
Closed by #24
from forbiddenfruit.
I was trying to redefine the
__mul__
operator for functions so it's do function composition instead. I was trying with forbidden fruit, but it wouldn't work because of this error. But I came up with a different and satisfactory solution, in case someone needs inspiration, I put it here:import types __all__ = ['composible'] # This decorator makes function composible by '*' operator with other functions def composible(f): class ComposibleFunction: def __init__(self, f): self.f = f def __call__(self, *a, **kw): return self.f(*a, **kw) def __mul__(self, g): return self.__class__(lambda *a, **kw: self.f(g(*a, **kw))) return ComposibleFunction(f) # Make all builtin function composible: for b in (b for b in dir(__builtins__) if type(getattr(__builtins__,b)) in (type, types.BuiltinMethodType)): exec('global '+b+'\n'+b + '=composible(__builtins__.' + b + ')') __all__.append(b) # New functions need to be decorated @composible def foo(x): return x+1 @composible def bar(x): return x*2 # This will output 7: print((foo * bar * sum)([1, 2]))
Is there any lib implementing this wonderful?
from forbiddenfruit.
Related Issues (20)
- test fails with py3.8 and pytest HOT 3
- str.__mod__ method is not patchable! HOT 1
- Unable to perfectly re-curse to __add__ HOT 2
- Cursing int with __iter__ fails with an error HOT 1
- Possible simplification of patchable_builtin() HOT 2
- Provide wheel packages? HOT 8
- Exceptions raised in cursed-in dunder methods are not raised to calling code
- Crash on Windows if dunder methods cursed into `ctypes.c_int` raises an exception
- Fix simple typo: infomation -> information HOT 1
- Replacing __hash__ returns a key error HOT 1
- Cursing dunder methods fails with: KeyError: '<method_to_curse>' HOT 5
- Patch for str __add__ not working
- object has no attribute '__dict__' HOT 1
- how to use reverse with existing attributes HOT 2
- MacOS errors monkeypatching `object`
- Undiagnosed crash (Windows 11, Python 3.10)
- How to make the additional function can be autocompleted in VS Code? HOT 1
- Add `pyproject.toml` or implement wheel-based distribution
- Unable to curse __call__
- __init_subclass__ seems to not be implemented as a cursable magic method
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from forbiddenfruit.