Comments (9)
@jmdejong Hm, not sure what's going on. I don't see that with Python 2.7.15 on 64-bit linux (Fedora28, using Anaconda Python 2.7.15). I haven't tried 3.7 yet. Any clues?
from asteval.
I believe it's in the error handling of RecursionErrors.
After messing around a bit I think I the problem is in those lines:
https://github.com/newville/asteval/blob/cca7e6621be39974f0a566907d1edba3e4a09d52/asteval/asteval.py#L757-L762
and
https://github.com/newville/asteval/blob/cca7e6621be39974f0a566907d1edba3e4a09d52/asteval/asteval.py#L293-L300
About when it is deep enough in recursion the python interpreter will call a RecursionError
Somewhere after catching this error the program starts running away.
I haven't figured out yet how or why.
My recursion limit is 1000 by the way (for both python 2 and 3)
from asteval.
The problem is not infinite looping/recursion, it turns out to be the exponential growth of error messages!
https://github.com/newville/asteval/blob/cca7e6621be39974f0a566907d1edba3e4a09d52/asteval/asteval.py#L242
https://github.com/newville/asteval/blob/cca7e6621be39974f0a566907d1edba3e4a09d52/asteval/asteval.py#L248
https://github.com/newville/asteval/blob/cca7e6621be39974f0a566907d1edba3e4a09d52/asteval/asteval.py#L760-L762
on_call will call raise_exception with a custom message including the message of the exception that was caught.
raise_exception will add this message to the current messages.
Then it will and then re-raise with all current messages as message.
With each iteration of this the size of the current messages will double.
For 30 levels of nested error throwing this is already gigabytes of data.
A simple recursive function with an if
will hit the recursion limit at about python_recusion_limit/8
iterations, in my case around 120 iterations.
from asteval.
Hm, OK thanks for the update. I'm not sure why this is only now a problem... But I'm also not sure why it doubles with each step instead of growing linearly with step.
Perhaps we should limit the length of self.error_msg
? I don't know for sure that this will fix the problem. For sure there is supposed to be a RecursionError raised here....
from asteval.
Yes there is a RecursionError here. That's what originally triggers these exceptions.
Lines 761 an 762 were only implemented like this a month ago, that would explain why it wasn't seen before.
It could be possible that you and travis have a lower recursion depth limit.
If your limit would be 100 the evaluated code would only nest to about 20 levels I think. This results in only a few megabytes of error messages and most systems can deal with that
So what is happening is that raise_exception
is called, it raises an exception and the next level up the call stack it is caught, and then that exception handler calls raise_exception
again etc...
Line 242 comes down to:
self.error_msg = self.error_msg + msg
and the other lines come down to
msg = self.error_msg
So this in combination comes down to
self.error_msg = self.error_msg + self.error_msg
Limiting the length of self.error_msg
would probably solve the problem but it would also probably be an ugly solution.
I think my preferred solution would be to not use self.error_msg
at all. The error message can just bubble up through the exceptions then.
from asteval.
@jmdejong Um, I still cannot reproduce this problem, and I'm afraid that I do not completely understand the reports you are giving. I would much rather read the actual messages than read your interpretations of them. For example, you said the error message doubles with each step, but it sure looks to me like it should grow linearly.
The default recursion depth is 1000. With this script:
import sys
from asteval import Interpreter
aeval = Interpreter()
script = """
def tmp(x):
return tmp(x+1)
##
"""
aeval(script)
for rec_limit in (50, 100, 200, 500, 1000, 2000, 5000, 10000):
sys.setrecursionlimit(rec_limit)
aeval('tmp(33)')
msg = aeval.error_msg
if msg is not None:
print("rec limit=%d, length of error messge=%d" % (rec_limit, len(msg)))
and Python 2.7 I get:
RuntimeError
<>
^^^
Error running <Procedure tmp(x)>
rec limit=50, length of error messge=181
RuntimeError
<>
^^^
Error running <Procedure tmp(x)>
rec limit=100, length of error messge=555
RuntimeError
<>
^^^
maximum recursion depth exceeded
rec limit=200, length of error messge=1337
RuntimeError
<>
^^^
Error running <Procedure tmp(x)>
rec limit=500, length of error messge=3411
RuntimeError
<>
^^^
maximum recursion depth exceeded
rec limit=1000, length of error messge=7593
RuntimeError
<>
^^^
maximum recursion depth exceeded
rec limit=2000, length of error messge=16025
RuntimeError
<>
^^^
maximum recursion depth exceeded
rec limit=5000, length of error messge=37207
RuntimeError
<>
^^^
maximum recursion depth exceeded
rec limit=10000, length of error messge=79639
which looks pretty linear to me -- and it runs quickly. With Python 3.6.6, I get similar results.
Since I'm not able to reproduce the problem, I'm not sure what is going on.
from asteval.
@jmdejong Ah, sorry, I was getting 0.9.12 not the master branch, from some oddity of python eggs. With master-branch I can see the problem with Python 2.7.
from asteval.
I'm not sure why this is only now a problem...
Haven't looked into it much, but it could be related to #42 and #43
from asteval.
calling this closed...
from asteval.
Related Issues (20)
- asteval segfault HOT 5
- Missing dependency in Python 3.7 HOT 3
- Latest release (0.9.26) has importlib_metadata/importlib.metadata bug HOT 7
- Improve the propagation of Exception information (the bad name for a NameError) HOT 2
- List comprehension with multiple "for" loops results in NameError HOT 6
- max_time seems to be unsued HOT 2
- Support regex / glob modules HOT 2
- PyPI: also upload source tarball HOT 2
- power to a negative int HOT 4
- name 'str' is not defined HOT 1
- tests/test_asteval.py::TestEval::test_set_default_nodehandler fails if Python is built without Tkiner HOT 4
- ipmt throws deprecation warning in numpy 2.0.dev HOT 2
- The truth value of a Series is ambiguous HOT 3
- Variable unpacking does not work as expected HOT 8
- Supporting lambda expressions HOT 2
- Python 3.10: test_kaboom fails with IndexError HOT 4
- Import error with python 3.7 due to missing import.metadata HOT 5
- Backward incompatible change is not documented HOT 8
- Does not evaluate types like eval() HOT 6
- Returned value is repeated on next invocation of eval HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from asteval.