che-shr-cat / currennt Goto Github PK

View Code? Open in Web Editor NEW

16.0 16.0 5.0 49.07 MB

CURRENNT -- CUDA-enabled machine learning library for recurrent neural network

License: GNU General Public License v3.0

CMake 0.17% C++ 54.20% Cuda 34.20% C 10.13% Perl 1.15% Python 0.15%

currennt's People

Contributors

Stargazers

Watchers

Forkers

polceanum hdubey nd1511 ly-so emilpi

currennt's Issues

Can't compile with CUDA 10.0

Trying to compile https://github.com/che-shr-cat/currennt/ with CUDA 10.0:

git clone https://github.com/che-shr-cat/currennt/
mkdir build && cd build && cmake -DCUDA_cublas_device_LIBRARY=/usr/local/cuda/targets/x86_64-linux/lib/libcublas.so .. && make

yields

...
[ 61%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/optimizers/currennt_generated_Optimizer.cu.o
Scanning dependencies of target currennt
[ 65%] Building CXX object CMakeFiles/currennt.dir/currennt_lib/src/Configuration.cpp.o
[ 69%] Building CXX object CMakeFiles/currennt.dir/currennt_lib/src/NeuralNetwork.cpp.o
In file included from /usr/local/cuda-10.0/include/thrust/detail/reference.h:173:0,
                 from /usr/local/cuda-10.0/include/thrust/memory.h:25,
                 from /usr/local/cuda-10.0/include/thrust/detail/allocator/temporary_allocator.h:23,
                 from /usr/local/cuda-10.0/include/thrust/detail/temporary_array.h:40,
                 from /usr/local/cuda-10.0/include/thrust/system/cuda/detail/internal/copy_cross_system.h:41,
                 from /usr/local/cuda-10.0/include/thrust/system/cuda/detail/copy.h:99,
                 from /usr/local/cuda-10.0/include/thrust/system/detail/adl/copy.h:42,
                 from /usr/local/cuda-10.0/include/thrust/detail/copy.inl:22,
                 from /usr/local/cuda-10.0/include/thrust/detail/copy.h:90,
                 from /usr/local/cuda-10.0/include/thrust/detail/allocator/copy_construct_range.inl:21,
                 from /usr/local/cuda-10.0/include/thrust/detail/allocator/copy_construct_range.h:46,
                 from /usr/local/cuda-10.0/include/thrust/detail/contiguous_storage.inl:22,
                 from /usr/local/cuda-10.0/include/thrust/detail/contiguous_storage.h:161,
                 from /usr/local/cuda-10.0/include/thrust/detail/vector_base.h:29,
                 from /usr/local/cuda-10.0/include/thrust/host_vector.h:26,
                 from /home/misc/archive/currennt/currennt_lib/src/layers/../Types.hpp:26,
                 from /home/misc/archive/currennt/currennt_lib/src/layers/Layer.hpp:26,
                 from /home/misc/archive/currennt/currennt_lib/src/layers/InputLayer.hpp:26,
                 from /home/misc/archive/currennt/currennt_lib/src/NeuralNetwork.hpp:26,
                 from /home/misc/archive/currennt/currennt_lib/src/NeuralNetwork.cpp:23:
/usr/local/cuda-10.0/include/thrust/detail/reference.inl: In instantiation of ‘thrust::reference<Element, Pointer, Derived>::value_type thrust::reference<Element, Pointer, Derived>::strip_const_get_value(const System&) const [with System = thrust::cuda_cub::tag; Element = const char; Pointer = thrust::device_ptr<const char>; Derived = thrust::device_reference<const char>; thrust::reference<Element, Pointer, Derived>::value_type = char]’:
/usr/local/cuda-10.0/include/thrust/detail/reference.inl:105:31:   required from ‘thrust::reference<Element, Pointer, Derived>::value_type thrust::reference<Element, Pointer, Derived>::convert_to_value_type(System*) const [with System = thrust::cuda_cub::tag; Element = const char; Pointer = thrust::device_ptr<const char>; Derived = thrust::device_reference<const char>; thrust::reference<Element, Pointer, Derived>::value_type = char]’
/usr/local/cuda-10.0/include/thrust/detail/reference.inl:122:31:   required from ‘thrust::reference<Element, Pointer, Derived>::operator thrust::reference<Element, Pointer, Derived>::value_type() const [with Element = const char; Pointer = thrust::device_ptr<const char>; Derived = thrust::device_reference<const char>; thrust::reference<Element, Pointer, Derived>::value_type = char]’
/home/misc/archive/currennt/currennt_lib/src/NeuralNetwork.cpp:244:9:   required from ‘std::vector<std::vector<std::vector<float> > > NeuralNetwork<TDevice>::getOutputs() [with TDevice = Gpu]’
/home/misc/archive/currennt/currennt_lib/src/NeuralNetwork.cpp:267:16:   required from here
/usr/local/cuda-10.0/include/thrust/detail/reference.inl:137:73: error: void value not ignored as it ought to be
   return get_value(thrust::detail::derived_cast(non_const_system), m_ptr);
                                                                         ^
In file included from /usr/local/cuda-10.0/include/thrust/system/detail/generic/for_each.h:27:0,
                 from /usr/local/cuda-10.0/include/thrust/detail/for_each.inl:26,
                 from /usr/local/cuda-10.0/include/thrust/for_each.h:279,
                 from /usr/local/cuda-10.0/include/thrust/system/detail/generic/transform.inl:19,
                 from /usr/local/cuda-10.0/include/thrust/system/detail/generic/transform.h:105,
                 from /usr/local/cuda-10.0/include/thrust/detail/transform.inl:25,
                 from /usr/local/cuda-10.0/include/thrust/transform.h:724,
                 from /usr/local/cuda-10.0/include/thrust/system/detail/generic/copy.inl:23,
                 from /usr/local/cuda-10.0/include/thrust/system/detail/generic/copy.h:58,
                 from /usr/local/cuda-10.0/include/thrust/detail/copy.inl:21,
                 from /usr/local/cuda-10.0/include/thrust/detail/copy.h:90,
                 from /usr/local/cuda-10.0/include/thrust/detail/allocator/copy_construct_range.inl:21,
                 from /usr/local/cuda-10.0/include/thrust/detail/allocator/copy_construct_range.h:46,
                 from /usr/local/cuda-10.0/include/thrust/detail/contiguous_storage.inl:22,
                 from /usr/local/cuda-10.0/include/thrust/detail/contiguous_storage.h:161,
                 from /usr/local/cuda-10.0/include/thrust/detail/vector_base.h:29,
                 from /usr/local/cuda-10.0/include/thrust/host_vector.h:26,
                 from /home/misc/archive/currennt/currennt_lib/src/layers/../Types.hpp:26,
                 from /home/misc/archive/currennt/currennt_lib/src/layers/Layer.hpp:26,
                 from /home/misc/archive/currennt/currennt_lib/src/layers/InputLayer.hpp:26,
                 from /home/misc/archive/currennt/currennt_lib/src/NeuralNetwork.hpp:26,
                 from /home/misc/archive/currennt/currennt_lib/src/NeuralNetwork.cpp:23:
/usr/local/cuda-10.0/include/thrust/system/detail/generic/for_each.h: In instantiation of ‘void thrust::system::detail::generic::get_value(thrust::execution_policy<Derived>&, Pointer) [with DerivedPolicy = thrust::cuda_cub::tag; Pointer = thrust::device_ptr<const char>]’:
/usr/local/cuda-10.0/include/thrust/detail/reference.inl:137:19:   required from ‘thrust::reference<Element, Pointer, Derived>::value_type thrust::reference<Element, Pointer, Derived>::strip_const_get_value(const System&) const [with System = thrust::cuda_cub::tag; Element = const char; Pointer = thrust::device_ptr<const char>; Derived = thrust::device_reference<const char>; thrust::reference<Element, Pointer, Derived>::value_type = char]’
/usr/local/cuda-10.0/include/thrust/detail/reference.inl:105:31:   required from ‘thrust::reference<Element, Pointer, Derived>::value_type thrust::reference<Element, Pointer, Derived>::convert_to_value_type(System*) const [with System = thrust::cuda_cub::tag; Element = const char; Pointer = thrust::device_ptr<const char>; Derived = thrust::device_reference<const char>; thrust::reference<Element, Pointer, Derived>::value_type = char]’
/usr/local/cuda-10.0/include/thrust/detail/reference.inl:122:31:   required from ‘thrust::reference<Element, Pointer, Derived>::operator thrust::reference<Element, Pointer, Derived>::value_type() const [with Element = const char; Pointer = thrust::device_ptr<const char>; Derived = thrust::device_reference<const char>; thrust::reference<Element, Pointer, Derived>::value_type = char]’
/home/misc/archive/currennt/currennt_lib/src/NeuralNetwork.cpp:244:9:   required from ‘std::vector<std::vector<std::vector<float> > > NeuralNetwork<TDevice>::getOutputs() [with TDevice = Gpu]’
/home/misc/archive/currennt/currennt_lib/src/NeuralNetwork.cpp:267:16:   required from here
/usr/local/cuda-10.0/include/thrust/detail/static_assert.h:71:13: error: invalid application of ‘sizeof’ to incomplete type ‘thrust::detail::STATIC_ASSERTION_FAILURE<false>’
       sizeof(::thrust::detail::STATIC_ASSERTION_FAILURE< (bool)( B ) >)>\
             ^
/usr/local/cuda-10.0/include/thrust/system/detail/generic/for_each.h:49:3: note: in expansion of macro ‘THRUST_STATIC_ASSERT’
   THRUST_STATIC_ASSERT( (thrust::detail::depend_on_instantiation<InputIterator, false>::value) );
   ^~~~~~~~~~~~~~~~~~~~
CMakeFiles/currennt.dir/build.make:198: recipe for target 'CMakeFiles/currennt.dir/currennt_lib/src/NeuralNetwork.cpp.o' failed
make[2]: *** [CMakeFiles/currennt.dir/currennt_lib/src/NeuralNetwork.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/currennt.dir/all' failed
make[1]: *** [CMakeFiles/currennt.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Problem building on c++-4.9 or ubuntu 16

Hey,

Thanks for the port to recent CUDA.
I got an issue when compiling on a recent C++-4.9 or ubuntu 16.04 (I think it is related to C++):

$ cmake ..
$ cmake ..
-- The C compiler identification is GNU 4.9.3
-- The CXX compiler identification is GNU 4.9.3
-- Found CUDA: /usr/local/cuda (found version "7.5")
-- Boost version: 1.58.0
-- Build files have been written to: /home/personnels/rioultf/software/currennt/build.jet
$ make
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   program_options
--   system
--   filesystem
--   random
--   thread
--   chrono
--   date_time
--   atomic
-- Configuring done
-- Generating done
-- Build files have been written to: /home/personnels/rioultf/software/currennt/build.jet
[  3%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/optimizers/currennt_generated_SteepestDescentOptimizer.cu.o
/usr/include/string.h: In function ‘void* __mempcpy_inline(void*, const void*, size_t)’:
/usr/include/string.h:652:42: error: ‘memcpy’ was not declared in this scope
   return (char *) memcpy (__dest, __src, __n) + __n;
                                          ^
CMake Error at currennt_generated_SteepestDescentOptimizer.cu.o.cmake:266 (message):
  Error generating file
  /home/personnels/rioultf/software/currennt/build.jet/CMakeFiles/currennt.dir/currennt_lib/src/optimizers/./currennt_generated_SteepestDescentOptimizer.cu.o

Cheers,
FR.

Problem with compilation

This happens when I run make command. There is some way to fix it?

$ make
[ 4%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_CePostOutputLayer.cu.o
[ 8%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/optimizers/./currennt_generated_Optimizer.cu.o
/var/tmp/testCurrennt/currennt_lib/src/optimizers/Optimizer.cu(52): warning: controlling expression is constant

/var/tmp/testCurrennt/currennt_lib/src/optimizers/Optimizer.cu(54): warning: controlling expression is constant

/var/tmp/testCurrennt/currennt_lib/src/optimizers/Optimizer.cu(52): warning: controlling expression is constant

/var/tmp/testCurrennt/currennt_lib/src/optimizers/Optimizer.cu(54): warning: controlling expression is constant

[ 12%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/optimizers/./currennt_generated_SteepestDescentOptimizer.cu.o
[ 16%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/./currennt_generated_LayerFactory.cu.o
[ 20%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/helpers/./currennt_generated_cublas.cu.o
[ 24%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/helpers/./currennt_generated_Matrix.cu.o
[ 28%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_TrainableLayer.cu.o
[ 32%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_SseMaskPostOutputLayer.cu.o
[ 36%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_MulticlassClassificationLayer.cu.o
[ 40%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_SsePostOutputLayer.cu.o
[ 44%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_LstmLayer.cu.o
[ 48%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_BinaryClassificationLayer.cu.o
[ 52%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_SoftmaxLayer.cu.o
[ 56%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_FeedForwardLayer.cu.o
[ 60%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_WeightedSsePostOutputLayer.cu.o
[ 64%] Building NVCC (Device) object CMakeFiles/currennt.dir/currennt_lib/src/layers/./currennt_generated_RmsePostOutputLayer.cu.o
Scanning dependencies of target currennt
[ 68%] Building CXX object CMakeFiles/currennt.dir/currennt_lib/src/NeuralNetwork.cpp.o
/var/tmp/testCurrennt/currennt_lib/src/NeuralNetwork.cpp:39:94: error: redeclaration of ‘NeuralNetwork::NeuralNetwork(const helpers::JsonDocument&, int, int, int, int)’ may not have default arguments [-fpermissive]
int inputSizeOverride = -1, int outputSizeOverride = -1)
^
CMakeFiles/currennt.dir/build.make:182: recipe for target 'CMakeFiles/currennt.dir/currennt_lib/src/NeuralNetwork.cpp.o' failed
make[2]: *** [CMakeFiles/currennt.dir/currennt_lib/src/NeuralNetwork.cpp.o] Error 1
CMakeFiles/Makefile2:60: recipe for target 'CMakeFiles/currennt.dir/all' failed
make[1]: *** [CMakeFiles/currennt.dir/all] Error 2
Makefile:76: recipe for target 'all' failed
make: *** [all] Error 2

It seems "weighted_sse" is not implemented

Hi,

I try "weighted_sse" with the proper configuration (defined in README, a copy is at the end) but it returns unknown layer type:

` ./run.sh
Started in hybrid online/batch training mode.
Mini-batches (50 sequences each) will be shuffled during training.
Using input noise with a standard deviation of 0.1.
The trained network will be written to 'trained_network.jsn'.
Validation error will be calculated every 1 epochs.
Training will be stopped after 100 epochs or if there is no new lowest validatio n error within 20 epochs.
Utilizing the GPU for computations with 50 sequences in parallel.
Normal distribution with mean=0 and sigma=0.1. Random seed: 2659768064

Using device #0 (Tesla K80)
Reading network from 'network.jsn'... done.

Loading training set 'train_1_speaker.nc' ...
using cache file: /tmp/2df4-2bb1-1592-3678
... done.
Loaded fraction: 100%
Sequences: 500
Sequence lengths: 113..216
Total timesteps: 74334

Loading validation set 'val_1_speaker.nc' ...
using cache file: /tmp/69cf-4c42-5905-2550
... done.
Loaded fraction: 100%
Sequences: 102
Sequence lengths: 113..152
Total timesteps: 13878

Creating the neural network... FAILED: Invalid network file: Could not create la yer: Unknown layer type 'weighted_sse'
`
I am using the updated CURRENNT, currennt-0.2-rc1, which is said weighted_sse is implemented but there is no!!

PS.
From README:
"weighted_sse: Weighted Sum of Squared Error objective function
To be used with any output layer. Requires 2N targets for output layer of
size N. The targets and the weights are expected to be interleaved, i.e., any
vector of targets should be of the form (t1, w1, t2, w2, ..., tN, wN)."

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.