hannes-brt / hebel Goto Github PK
View Code? Open in Web Editor NEWGPU-Accelerated Deep Learning Library in Python
License: GNU General Public License v2.0
GPU-Accelerated Deep Learning Library in Python
License: GNU General Public License v2.0
ubgpu@ubgpu:/github/hebel$ sudo pip install pyCUDA/github/hebel$
Requirement already satisfied (use --upgrade to upgrade): pyCUDA in /usr/local/lib/python2.7/dist-packages
Requirement already satisfied (use --upgrade to upgrade): decorator>=3.2.0 in /usr/local/lib/python2.7/dist-packages (from pyCUDA)
Requirement already satisfied (use --upgrade to upgrade): pytools>=2011.2 in /usr/local/lib/python2.7/dist-packages (from pyCUDA)
Requirement already satisfied (use --upgrade to upgrade): pytest>=2 in /usr/local/lib/python2.7/dist-packages (from pyCUDA)
Requirement already satisfied (use --upgrade to upgrade): appdirs>=1.4.0 in /usr/local/lib/python2.7/dist-packages (from pytools>=2011.2->pyCUDA)
Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python2.7/dist-packages (from pytools>=2011.2->pyCUDA)
Requirement already satisfied (use --upgrade to upgrade): py>=1.4.25 in /usr/local/lib/python2.7/dist-packages (from pytest>=2->pyCUDA)
ubgpu@ubgpu:
ubgpu@ubgpu:/github/hebel$/github/hebel$
ubgpu@ubgpu:
ubgpu@ubgpu:~/github/hebel$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
quit()
ubgpu@ubgpu:/github/hebel$/github/hebel$
ubgpu@ubgpu:
ubgpu@ubgpu:/github/hebel$ sudo pip install PyCUDA/github/hebel$
Requirement already satisfied (use --upgrade to upgrade): PyCUDA in /usr/local/lib/python2.7/dist-packages
Requirement already satisfied (use --upgrade to upgrade): decorator>=3.2.0 in /usr/local/lib/python2.7/dist-packages (from PyCUDA)
Requirement already satisfied (use --upgrade to upgrade): pytools>=2011.2 in /usr/local/lib/python2.7/dist-packages (from PyCUDA)
Requirement already satisfied (use --upgrade to upgrade): pytest>=2 in /usr/local/lib/python2.7/dist-packages (from PyCUDA)
Requirement already satisfied (use --upgrade to upgrade): appdirs>=1.4.0 in /usr/local/lib/python2.7/dist-packages (from pytools>=2011.2->PyCUDA)
Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python2.7/dist-packages (from pytools>=2011.2->PyCUDA)
Requirement already satisfied (use --upgrade to upgrade): py>=1.4.25 in /usr/local/lib/python2.7/dist-packages (from pytest>=2->PyCUDA)
ubgpu@ubgpu:
ubgpu@ubgpu:/github/hebel$/github/hebel$
ubgpu@ubgpu:
ubgpu@ubgpu:/github/hebel$/github/hebel$
ubgpu@ubgpu:
ubgpu@ubgpu:/github/hebel$ echo $PYTHONPATH/github/hebel$
/usr/local/lib/python2.7/dist-packages
ubgpu@ubgpu:
ubgpu@ubgpu:/github/hebel$/github/hebel$ python train_model.py examples/mnist_neural_net_shallow.yml
ubgpu@ubgpu:
Traceback (most recent call last):
File "train_model.py", line 39, in
run_from_config(yaml_src)
File "/home/ubgpu/github/hebel/hebel/config.py", line 41, in run_from_config
config = load(yaml_src)
File "/home/ubgpu/github/hebel/hebel/config.py", line 92, in load
proxy_graph = yaml.load(string, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/yaml/init.py", line 71, in load
return loader.get_single_data()
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 39, in get_single_data
return self.construct_document(node)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 48, in construct_document
for dummy in generator:
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 398, in construct_yaml_map
value = self.construct_mapping(node)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object
data = constructor(self, tag_suffix, node)
File "/home/ubgpu/github/hebel/hebel/config.py", line 318, in multi_constructor
mapping = loader.construct_mapping(node)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object
data = constructor(self, tag_suffix, node)
File "/home/ubgpu/github/hebel/hebel/config.py", line 318, in multi_constructor
mapping = loader.construct_mapping(node)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object
data = constructor(self, tag_suffix, node)
File "/home/ubgpu/github/hebel/hebel/config.py", line 323, in multi_constructor
classname = try_to_import(tag_suffix)
File "/home/ubgpu/github/hebel/hebel/config.py", line 251, in try_to_import
exec('import %s' % modulename)
File "", line 1, in
File "/home/ubgpu/github/hebel/hebel/layers/init.py", line 17, in
from .dummy_layer import DummyLayer
File "/home/ubgpu/github/hebel/hebel/layers/dummy_layer.py", line 17, in
from .hidden_layer import HiddenLayer
File "/home/ubgpu/github/hebel/hebel/layers/hidden_layer.py", line 25, in
from ..pycuda_ops import linalg
File "/home/ubgpu/github/hebel/hebel/pycuda_ops/linalg.py", line 32, in
from . import cublas
File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cublas.py", line 47, in
import cuda
File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cuda.py", line 35, in
from cudart import *
File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cudart.py", line 60, in
raise OSError('CUDA runtime library not found')
OSError: CUDA runtime library not found
Wrap Alex Krizhevsky's cuda-convnet kernels (https://code.google.com/p/cuda-convnet/).
Example script data_providers.py imports skdata by
from skdata.mnist.view import OfficialVectorClassification
It should be skdata.mnist.views, otherwise errors occur.
When running optimizer.run(100), an error occurred: global name 'hidden_inputs' is not defined in line 323 of ./hebel/hebel/models/neurals_net.py
Where to define the global variable 'hidden_inputs'? Thanks!
Heya
I stumbled across this project looking for some PyCUDA routines that operate on matrices per-row or per-column. It seems you have a bunch of handy routines for this, which is awesome, e.g. row-wise maximum, add_vec_to_mat etc.
Would you be willing to contribute them back to PyCUDA? a lot of these routines seem like they'd definitely be useful more widely. And perhaps offering the contribution might give the PyCUDA guys some inspiration or a kick in the arse to create a more general partial reductions API (like numpy's axis=0 arguments) and broadcasting behaviour for element-wise operations on GPUArrays? (I would attempt this myself but my CUDA-fu is weak)
Just a thought anyway. I would suggest it to them myself but the licencing is different (GPL vs MIT)
Cheers!
I have almost got it to work, but it is missing memory_pool. This is imported in the regression test.
ubgpu@ubgpu:/github/hebel$ echo $LD_LIBRARY_PATH/github/hebel$
/usr/local/cuda:/usr/local/cuda/bin:/usr/local/cuda/lib64:/home/ubgpu/torch/install/lib:/home/ubgpu/torch/install/lib
ubgpu@ubgpu:
ubgpu@ubgpu:/github/hebel$/github/hebel$ python train_model.py examples/mnist_neural_net_shallow.yml
ubgpu@ubgpu:
Traceback (most recent call last):
File "train_model.py", line 39, in
run_from_config(yaml_src)
File "/home/ubgpu/github/hebel/hebel/config.py", line 41, in run_from_config
config = load(yaml_src)
File "/home/ubgpu/github/hebel/hebel/config.py", line 92, in load
proxy_graph = yaml.load(string, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/yaml/init.py", line 71, in load
return loader.get_single_data()
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 39, in get_single_data
return self.construct_document(node)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 48, in construct_document
for dummy in generator:
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 398, in construct_yaml_map
value = self.construct_mapping(node)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object
data = constructor(self, tag_suffix, node)
File "/home/ubgpu/github/hebel/hebel/config.py", line 318, in multi_constructor
mapping = loader.construct_mapping(node)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object
data = constructor(self, tag_suffix, node)
File "/home/ubgpu/github/hebel/hebel/config.py", line 318, in multi_constructor
mapping = loader.construct_mapping(node)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object
data = constructor(self, tag_suffix, node)
File "/home/ubgpu/github/hebel/hebel/config.py", line 323, in multi_constructor
classname = try_to_import(tag_suffix)
File "/home/ubgpu/github/hebel/hebel/config.py", line 251, in try_to_import
exec('import %s' % modulename)
File "", line 1, in
File "/home/ubgpu/github/hebel/hebel/layers/init.py", line 17, in
from .dummy_layer import DummyLayer
File "/home/ubgpu/github/hebel/hebel/layers/dummy_layer.py", line 17, in
from .hidden_layer import HiddenLayer
File "/home/ubgpu/github/hebel/hebel/layers/hidden_layer.py", line 25, in
from ..pycuda_ops import linalg
File "/home/ubgpu/github/hebel/hebel/pycuda_ops/linalg.py", line 32, in
from . import cublas
File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cublas.py", line 47, in
import cuda
File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cuda.py", line 36, in
from cudadrv import *
File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cudadrv.py", line 233, in
_libcuda.cuPointerGetAttribute.restype = int
File "/usr/lib/python2.7/ctypes/init.py", line 378, in getattr
func = self.getitem(name)
File "/usr/lib/python2.7/ctypes/init.py", line 383, in getitem
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: python: undefined symbol: cuPointerGetAttribute
ubgpu@ubgpu:~/github/hebel$
at commit a7f4cbb
with python 2.7.6 I try:
'python train_model.py examples/mnist_neural_net_shallow.yml'
And i get the following
Traceback (most recent call last):
File "train_model.py", line 39, in <module>
run_from_config(yaml_src)
File "/Users/epic/Documents/git/hebel/hebel/config.py", line 41, in run_from_config
config = load(yaml_src)
File "/Users/epic/Documents/git/hebel/hebel/config.py", line 92, in load
proxy_graph = yaml.load(string, **kwargs)
File "/usr/local/lib/python2.7/site-packages/yaml/__init__.py", line 71, in load
return loader.get_single_data()
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 39, in get_single_data
return self.construct_document(node)
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 48, in construct_document
for dummy in generator:
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 398, in construct_yaml_map
value = self.construct_mapping(node)
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 208, in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 133, in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 90, in construct_object
data = constructor(self, tag_suffix, node)
File "/Users/epic/Documents/git/hebel/hebel/config.py", line 318, in multi_constructor
mapping = loader.construct_mapping(node)
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 208, in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 133, in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 90, in construct_object
data = constructor(self, tag_suffix, node)
File "/Users/epic/Documents/git/hebel/hebel/config.py", line 318, in multi_constructor
mapping = loader.construct_mapping(node)
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 208, in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 133, in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 90, in construct_object
data = constructor(self, tag_suffix, node)
File "/Users/epic/Documents/git/hebel/hebel/config.py", line 323, in multi_constructor
classname = try_to_import(tag_suffix)
File "/Users/epic/Documents/git/hebel/hebel/config.py", line 251, in try_to_import
exec('import %s' % modulename)
File "<string>", line 1, in <module>
File "/Users/epic/Documents/git/hebel/hebel/layers/__init__.py", line 17, in <module>
from .dummy_layer import DummyLayer
File "/Users/epic/Documents/git/hebel/hebel/layers/dummy_layer.py", line 17, in <module>
from .hidden_layer import HiddenLayer
File "/Users/epic/Documents/git/hebel/hebel/layers/hidden_layer.py", line 25, in <module>
from ..pycuda_ops import linalg
File "/Users/epic/Documents/git/hebel/hebel/pycuda_ops/linalg.py", line 32, in <module>
from . import cublas
File "/Users/epic/Documents/git/hebel/hebel/pycuda_ops/cublas.py", line 47, in <module>
import cuda
File "/Users/epic/Documents/git/hebel/hebel/pycuda_ops/cuda.py", line 35, in <module>
from cudart import *
File "/Users/epic/Documents/git/hebel/hebel/pycuda_ops/cudart.py", line 142, in <module>
_libcudart.cudaGetErrorString.restype = ctypes.c_char_p
AttributeError: 'NoneType' object has no attribute 'cudaGetErrorString'
Implement Autoencoders, including denoising autoencoders and contracting autoencoders.
Does this work on Python 3, Windows? I'm facing some installation issues.
when will you release the windows x64 project of this tool
Hi there,
I really appreciate Hebel. It was a good first step for me to "take the plunge" into using GPU.
I struggled a bit after going through the example (MNIST) script. In particular, it wasn't clear how to have the model predict new data (i.e., data you don't have targets for).
The first (small) stumble was what to with the DataProvider. I just put in dummy zero targets. Perhaps targets
could be an optional field somehow?
A more thorny issue was how to actually do the predictions. I couldn't for the life of me figure out how to feed the DataProvider data into the feed_forward without getting the error:
File "/usr/local/lib/python2.7/dist-packages/hebel/models/neural_net.py", line 422, in feed_forward
prediction=prediction))
File "/usr/local/lib/python2.7/dist-packages/hebel/layers/input_dropout.py", line 96, in feed_forward
return (input_data * (1 - self.dropout_probability),)
TypeError: unsupported operand type(s) for *: 'MiniBatchDataProvider' and 'float'
This was my original attempt:
# After loading in the data . . .
Xv = Xv.astype(np.float32)
yv = pd.get_dummies(yv).values.astype(np.float32)
valid_data = MiniBatchDataProvider(Xv, yv, batch_size=5000)
I finally resorted to useing a gpu array which worked:
from pycuda import gpuarray
valid_data = gpuarray.to_gpu(Xt)
y_pred = model.feed_forward(valid_data, return_cache=False, prediction=True).get()
The .get()
at the end of the last statement was also something I had to figure out going through code.
Having an example in the documentation would be helpful.
I guess you did:
`python setup.py sdist register
I am trying to compile in Mac OSX yosemite and it seems hebel is not running. i installed PyCUDA and other libraries needed but stuck at this error.
$ python hebel_test.py
Traceback (most recent call last):
File "hebel_test.py", line 18, in
hebel.init(0)
File "/Users/prabhubalakrishnan/Desktop/hebel/hebel/init.py", line 131, in init
from pycuda import gpuarray, driver, curandom
File "/Library/Python/2.7/site-packages/pycuda-2014.1-py2.7-macosx-10.10-intel.egg/pycuda/gpuarray.py", line 3, in
import pycuda.elementwise as elementwise
File "/Library/Python/2.7/site-packages/pycuda-2014.1-py2.7-macosx-10.10-intel.egg/pycuda/elementwise.py", line 34, in
from pytools import memoize_method
File "/Library/Python/2.7/site-packages/pytools-2014.3.5-py2.7.egg/pytools/init.py", line 5, in
from six.moves import range, zip, intern, input
ImportError: cannot import name intern
How to fix?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.