OverflowError in 1 Preprocess Data about code_search

[hamelsmu]

Just so I understand what is the downside of using Docker for this tutorial? The web of dependencies can be hard to manage with just requirements.txt

[Laksh47]

Like @hamelsmu suggests, using Docker is the best thing to do! However, this issue is just a floating point arithmetic issue. One can workaround it by editing the library file code_gen.py

Just change the line "import math" to "from mpmath import mp as math" @briandw
Refer: http://mpmath.org/

[hydeparksnow]

I changed the following line in function get_function_docstring_pairs, worked for me
#except (AssertionError, MemoryError, SyntaxError, UnicodeEncodeError):
to
except:

[hamelsmu]

Are you using the docker container associated with this tutorial? Also make sure you use the exact copy of the fastai library in this repo

[briandw]

I wasn't using the docker. I prefer to have the ability to reproduce the environment that the code runs on myself. Thanks.

[shengyfu]

@briandw, I got the same error as you did, how did you resolve the issue?

[briandw]

I didn't get it resolved. I've moved on to making my own project to train sentence vectors. Check it out if you are interested. It should be easy to run in a condo environment with Fastai and pytorch 0.4.1
https://github.com/briandw/SiameseULMFiT

[gauravkoradiya]

I also get same error :
RemoteTraceback Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\multiprocess\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\multiprocess\pool.py", line 44, in mapstar
return list(map(*args))
File "E:\Mindtree_IDP\general_utils.py", line 184, in get_function_docstring_pairs_list
return [get_function_docstring_pairs(b) for b in blob_list]
File "E:\Mindtree_IDP\general_utils.py", line 184, in
return [get_function_docstring_pairs(b) for b in blob_list]
File "E:\Mindtree_IDP\general_utils.py", line 167, in get_function_docstring_pairs
source = astor.to_source(f)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 52, in to_source
generator.visit(node)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\node_util.py", line 143, in visit
return visitor(node)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 320, in visit_FunctionDef
self.body(node.body)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 218, in body
self.write(*statements)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 168, in write
visit(item)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\node_util.py", line 143, in visit
return visitor(node)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 356, in visit_If
self.body(node.body)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 218, in body
self.write(*statements)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 168, in write
visit(item)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\node_util.py", line 143, in visit
return visitor(node)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 472, in visit_Return
self.conditional_write(' ', node.value)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 206, in conditional_write
self.write(*stuff)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 168, in write
visit(item)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\node_util.py", line 143, in visit
return visitor(node)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 627, in visit_Num
real = part(x.real if isinstance(x, complex) else x, imaginary=False)
File "C:\Users\GK\AppData\Local\Continuum\anaconda3\envs\mindtree\lib\site-packages\astor\code_gen.py", line 619, in part
if math.isinf(p):
OverflowError: int too large to convert to float
"""