leriomaggio / deep-learning-keras-tensorflow Goto Github PK
View Code? Open in Web Editor NEWIntroduction to Deep Neural Networks with Keras and Tensorflow
License: MIT License
Introduction to Deep Neural Networks with Keras and Tensorflow
License: MIT License
UserWarning: Theano does not recognise this flag: lib.cnmem
warnings.warn('Theano does not recognise this flag: {0}'.format(key))
Hi, Valerio,
I am still new to anaconda and was trying to build the conda env on Mac terminal by
conda env create -f deep-learning-osx.yml
And got the error
NoPackagesFoundError: Package missing in current osx-64 channels:
- backports.shutil_get_terminal_size 1.0.0 py35_0
.
After reading this post (conda/conda#4994), I deleted some of the build numbers and/or version numbers, and also deleted backports.shutil_get_terminal_size. The environment was then created (I can share my version if needed). Check import/installed version passed. I am still at the beginning of your tutorial, and not sure if this will lead to future problems. Any help with this issue would be much appreciated!
Also, thank you very much for sharing this consice tutorial!
The readme says
conda create env -f deep-learning.yml
when it should be
conda env create -f deep-learning.yml
I guess you have video recorded?
may you share this video?
I do sequence classification using LSTM and The length of sequences are variable.
So I need to pad it with max-length, and I also set mask_zero flag = true.
My Network model like below:
http://deeplearning.net/tutorial/lstm.html
My code as below:
def LSTM_Classification():
max_features = 19694+1
batch_size = 32
embedding_dims = 32
nb_epoch = 3
path = "./dataset/SemEval_task3.pkl"
max_words = max_features
(X_train, y_train), (X_test, y_test) = guan_SemEval_task3.load_data(path)
x_new_train = padding_all(X_train)
y_new_train = np.array([(1 if y is 0 else 0, 1 if y is 1 else 0, 1 if y is 2 else 0) for y in y_train])
x_new_test = padding_all(X_test)
y_new_test = np.array([(1 if y is 0 else 0, 1 if y is 1 else 0, 1 if y is 2 else 0) for y in y_test])
print (x_new_train.shape)
print (y_new_train.shape)
print (x_new_test.shape)
print (y_new_test.shape)
print('Build model...')
model = Sequential()
model.add(Embedding(max_features, embedding_dims, dropout=0.2,mask_zero=True))
model.add(LSTM(32, dropout_W=0.2, dropout_U=0.2,return_sequences=True))
print(model.output_shape) #(None, None, 32)
model.add(TimeDistributed(Dense(1)))
print(model.output_shape) #(None, None, 1)
avgpool = Lambda(lambda x: K.mean(x, axis=1, keepdims=False), output_shape=lambda x: (x[0], x[2]))
avgpool.supports_masking = True
model.add(avgpool)
print(model.output_shape) #(None, 1)
#model.add(Dense(3))
dense3 = Dense(3)
dense3.supports_masking = True
model.add(dense3)
print(model.output_shape) #(None, 1)
model.add(Activation('softmax'))
print(model.output_shape) #(None, 1)
model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
model.summary()
early_stop = EarlyStopping(monitor='val_loss', patience=20, verbose=1)
model.fit(x_new_train, y_new_train,nb_epoch=nb_epoch,batch_size=batch_size,
validation_data=(x_new_test, y_new_test),callbacks=[early_stop])
print('Testing...')
score, acc = model.evaluate(x_new_test, y_new_test,batch_size=batch_size)
print('Test score:', score)
print('Test accuracy:', acc)
the output_shape of the model seem to be right. but when I run this code,
the error message as below:
(16541L, 849L)
(16541L, 3L)
(1976L, 849L)
(1976L, 3L)
Build model...
(None, None, 32)
(None, None, 1)
(None, 1)
(None, 3)
(None, 3)
embedding_5 (Embedding) (None, None, 32) 630240 embedding_input_5[0][0]
lstm_5 (LSTM) (None, None, 32) 8320 embedding_5[0][0]
timedistributed_5 (TimeDistribute(None, None, 1) 33 lstm_5[0][0]
lambda_5 (Lambda) (None, 1) 0 timedistributed_5[0][0]
dense_10 (Dense) (None, 3) 6 lambda_5[0][0]
Total params: 638599
Train...
Train on 16541 samples, validate on 1976 samples
Epoch 1/3
Traceback (most recent call last):
File "", line 1, in
runfile('D:/prj_python_spyder/test1/test1/programs/test_fasttext.py', wdir='D:/prj_python_spyder/test1/test1/programs')
File "C:\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 714, in runfile
execfile(filename, namespace)
File "C:\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 74, in execfile
exec(compile(scripttext, filename, 'exec'), glob, loc)
File "D:/prj_python_spyder/test1/test1/programs/test_fasttext.py", line 508, in
main();
File "D:/prj_python_spyder/test1/test1/programs/test_fasttext.py", line 498, in main
doLSTM_AVGPooling()
File "D:/prj_python_spyder/test1/test1/programs/test_fasttext.py", line 305, in doLSTM_AVGPooling
validation_data=(x_new_test, y_new_test),callbacks=[early_stop])
File "C:\Anaconda2\lib\site-packages\keras\models.py", line 597, in fit
sample_weight=sample_weight)
File "C:\Anaconda2\lib\site-packages\keras\engine\training.py", line 1107, in fit
callback_metrics=callback_metrics)
File "C:\Anaconda2\lib\site-packages\keras\engine\training.py", line 825, in _fit_loop
outs = f(ins_batch)
File "C:\Anaconda2\lib\site-packages\keras\backend\theano_backend.py", line 657, in call
return self.function(*inputs)
File "C:\Anaconda2\lib\site-packages\theano\compile\function_module.py", line 879, in call
storage_map=getattr(self.fn, 'storage_map', None))
File "C:\Anaconda2\lib\site-packages\theano\gof\link.py", line 325, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "C:\Anaconda2\lib\site-packages\theano\compile\function_module.py", line 866, in call
self.fn() if output_subset is None else\
ValueError: Input dimension mis-match. (input[0].shape[1] = 32, input[1].shape[1] = 849)
Apply node that caused the error: Elemwise{mul,no_inplace}(InplaceDimShuffle{x,0}.0, Elemwise{Cast{float32}}.0)
Toposort index: 350
Inputs types: [TensorType(float32, row), TensorType(float32, matrix)]
Inputs shapes: [(1L, 32L), (32L, 849L)]
Inputs strides: [(128L, 4L), (3396L, 4L)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Sum{axis=[1], acc_dtype=float64}(Elemwise{mul,no_inplace}.0)]]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
The 849 of the max length I pad in sequences.
Could someone please help to fix this error ? I have got stuck several days.
Thanks a lot.
what is the value of sentence_matrix and i am unable to import data_helpers.. can you please help me
When I use the deep-learning.yml
file in pydatait branch, I get the error:
Could not find a version that satisfies the requirement pygpu==0.2.1 (from versions: )
No matching distribution found for pygpu==0.2.1
I noticed pygpu
is not required in setup recipe under branch master for example, should I exclude it?
Is it only required by Theano?
I succeeded in setting it all up, installing things by hands incrementally (keras
first).
I paste below my .yml
file, just in case (ah, I switched to tensorflow
as keras
backend, since Theano
still gives problems).
Content of deep_learning_keras.yml
:
name: deep_learning_keras
channels:
- conda-forge
- defaults
dependencies:
- blas=1.1=openblas
- bleach=1.5.0=py36_0
- ca-certificates=2017.1.23=0
- certifi=2017.1.23=py36_0
- cycler=0.10.0=py36_0
- dbus=1.10.10=3
- decorator=4.0.11=py36_0
- entrypoints=0.2.2=py36_1
- expat=2.1.0=2
- fontconfig=2.12.1=4
- freetype=2.7=1
- gettext=0.19.7=1
- glib=2.51.4=0
- gmp=6.1.2=0
- gst-plugins-base=1.8.0=0
- gstreamer=1.8.0=1
- h5py=2.7.0=np112py36_0
- hdf5=1.8.17=10
- html5lib=0.999=py36_0
- icu=58.1=1
- ipykernel=4.6.0=py36_0
- ipython=5.3.0=py36_0
- ipython_genutils=0.2.0=py36_0
- ipywidgets=6.0.0=py36_0
- jinja2=2.9.5=py36_0
- jpeg=9b=0
- jsonschema=2.5.1=py36_0
- jupyter=1.0.0=py36_0
- jupyter_client=5.0.1=py36_0
- jupyter_console=5.1.0=py36_0
- jupyter_core=4.3.0=py36_0
- keras=2.0.2=py36_1
- libffi=3.2.1=3
- libgpuarray=0.6.2=np112py36_0
- libiconv=1.14=4
- libpng=1.6.28=0
- libsodium=1.0.10=0
- libxcb=1.12=1
- libxml2=2.9.4=4
- line_profiler=2.0=py36_0
- mako=1.0.6=py36_0
- markupsafe=0.23=py36_1
- matplotlib=2.0.0=np112py36_3
- mistune=0.7.4=py36_0
- nbconvert=5.1.1=py36_1
- nbformat=4.2.0=py36_0
- ncurses=5.9=10
- nose=1.3.7=py36_2
- notebook=5.0.0=py36_0
- numpy=1.12.1=py36_blas_openblas_200
- openblas=0.2.19=1
- openssl=1.0.2k=0
- pandas=0.19.2=np112py36_1
- pandoc=1.19.2=0
- pandocfilters=1.4.1=py36_0
- pcre=8.39=0
- pexpect=4.2.1=py36_0
- pickleshare=0.7.3=py36_0
- pip=9.0.1=py36_0
- prompt_toolkit=1.0.14=py36_0
- ptyprocess=0.5.1=py36_0
- pygments=2.2.0=py36_0
- pyparsing=2.2.0=py36_0
- pyqt=5.6.0=py36_0
- python=3.6.1=0
- python-dateutil=2.6.0=py36_0
- pytz=2017.2=py36_0
- pyyaml=3.12=py36_1
- pyzmq=16.0.2=py36_1
- qt=5.6.2=1
- qtconsole=4.3.0=py36_0
- readline=6.2=0
- scikit-learn=0.18.1=np112py36_blas_openblas_200
- scipy=0.19.0=np112py36_blas_openblas_200
- setuptools=33.1.1=py36_0
- simplegeneric=0.8.1=py36_0
- sip=4.18=py36_1
- six=1.10.0=py36_1
- sqlite=3.13.0=1
- terminado=0.6=py36_0
- testpath=0.3=py36_0
- theano=0.9.0=py36_0
- tk=8.5.19=1
- tornado=4.4.3=py36_0
- traitlets=4.3.2=py36_0
- wcwidth=0.1.7=py36_0
- webencodings=0.5=py36_0
- wheel=0.29.0=py36_0
- widgetsnbextension=2.0.0=py36_0
- xorg-libxau=1.0.8=3
- xorg-libxdmcp=1.1.2=3
- xz=5.2.2=0
- yaml=0.1.6=0
- zeromq=4.2.1=1
- zlib=1.2.11=0
- libgfortran=3.0.0=1
- mkl=2017.0.1=0
- pip:
- appdirs==1.4.3
- ipython-genutils==0.2.0
- jupyter-client==5.0.1
- jupyter-console==5.1.0
- jupyter-core==4.3.0
- line-profiler==2.0
- packaging==16.8
- prompt-toolkit==1.0.14
- protobuf==3.2.0
- pygpu==0.6.2
- tensorflow-gpu==1.0.1
Hi All,
I put this up on theano-users google group but I wanted to put it up here too:
Using the deep-learning.yml install, I was able to install anaconda, tensorflow, cuda, keras and theano and others. I already had a cuda 7.5 working but I noticed that the deep-learning yml installs cuda 7.0 which is fine. I followed the instructions and I can import standard python packages like numpy, scipy pandas, etc. However, I am having trouble importing theano, via keras. When I import keras, it gives this error:
import keras
Using Theano backend.
Traceback (most recent call last):
File "", line 1, in
File "/home/nkiruka_ubuntu/anaconda3/envs/deep-learning/lib/python3.5/site-packages/keras/init.py", line 2, in
from . import backend
File "/home/nkiruka_ubuntu/anaconda3/envs/deep-learning/lib/python3.5/site-packages/keras/backend/init.py", line 58, in
from .theano_backend import *
File "/home/nkiruka_ubuntu/anaconda3/envs/deep-learning/lib/python3.5/site-packages/keras/backend/theano_backend.py", line 1, in
import theano
File "/home/nkiruka_ubuntu/anaconda3/envs/deep-learning/lib/python3.5/site-packages/theano/init.py", line 103, in
import theano.sandbox.cuda
File "/home/nkiruka_ubuntu/anaconda3/envs/deep-learning/lib/python3.5/site-packages/theano/sandbox/cuda/init.py", line 697, in
use(device=config.device, force=config.force_device, test_driver=False)
File "/home/nkiruka_ubuntu/anaconda3/envs/deep-learning/lib/python3.5/site-packages/theano/sandbox/cuda/init.py", line 516, in use
device = int(device[3:])
ValueError: invalid literal for int() with base 10: ' # switch to cpu if no GPU is available on your machine'
Now, I know my GPU works because I can run the samples. My theano.rc looks like this:
[global]
floatX = float32
device = gpu # switch to cpu if no GPU is available on your machine
[nvcc]
fastmath = True
[lib]
cnmem=.90
[cuda]
root = /usr/local/cuda-7.5/
If I remove the line device=gpu by commenting it out, then I get this error:
import keras
Using Theano backend.
Traceback (most recent call last):
File "", line 1, in
File "/home/nkiruka_ubuntu/anaconda3/envs/deep-learning/lib/python3.5/site-packages/keras/init.py", line 2, in
from . import backend
File "/home/nkiruka_ubuntu/anaconda3/envs/deep-learning/lib/python3.5/site-packages/keras/backend/init.py", line 58, in
from .theano_backend import *
File "/home/nkiruka_ubuntu/anaconda3/envs/deep-learning/lib/python3.5/site-packages/keras/backend/theano_backend.py", line 1, in
import theano
File "/home/nkiruka_ubuntu/anaconda3/envs/deep-learning/lib/python3.5/site-packages/theano/init.py", line 95, in
if hasattr(theano.tests, "TheanoNoseTester"):
AttributeError: module 'theano' has no attribute 'tests'
Can anyone direct me as to where I am going wrong?
is it possible to make the vgg19 model complete missing parts of an image according to its category ?
thanks
My model is defined as follows:
def create_model():
#input
input_layer=Input(shape=(max_squence_len,),name="x_seq")
embedding_layer=Embedding(input_dim=max_token_num,output_dim=embedding_dim) (input_layer)
#conv layers
convs=[]
filter_sizes=[2,3,4]
for fs in filter_sizes:
l_conv=Conv1D(filters=50,kernel_size=fs,activation="relu")(embedding_layer)
l_pool=MaxPooling1D(pool_size=2)(l_conv)
l_pool=Flatten()(l_pool)
convs.append(l_pool)
merge=concatenate(convs,axis=1)
out=Dropout(0.5)(merge)
output=Dense(32, activation='relu')(out)
output=Dense(units=31, activation='softmax')(output)
model=Model([input_layer],output)
adam = optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)
model.compile(loss="categorical_crossentropy",optimizer=adam,metrics=['accuracy'])
model=KerasClassifier(build_fn=self.create_model,batch_size=64,epochs=25)
batch_size = [10, 20, 40, 60, 80, 100]
epochs = [5,10,15,20,25,30]
param_grid = dict(batch_size=batch_size, epochs=epochs)
grid = GridSearchCV(estimator=model, param_grid=param_grid,cv=5)
grid.fit(x_train,y_train)
print("best_score:",grid.best_score_,"best_param:",grid.best_params_)
But the program throws an error:TypeError: cannot deepcopy this pattern object
can you help me solve this problem?
Hello!
I have a dataset of images in which the edges are manually labeled with different colors. I am planning to use deep learning on these training images. Then, I am thinking to test the model so that in the new input images, the model will predict those edges automatically.
I want to configure the "device" in theanorc to "cpu“ but it raised a value error.
And I removed the theanorc file, then it works.
In 1.1 Introduction - Deep Learning and ANN.ipynb there's this line:
change = output_deltas[k] * self.ah[j]
and change is then being used to the update the output weights.
self.wo[j][k] += N*change + M*self.co[j][k]
Why are the activations from the hidden layer being used to tweak the weights? shouldn't the weight changes be only based on the weights and the relative change of the loss?
Great looking tutorial. Just a small typo in the environment setup of README.md
.
conda create env ...
should be condo env create ...
Another issue I ran into while import matplotlib at 1.1 Introduction
import matplotlib
---> 24 from matplotlib.backends import _macosx
25
26
RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are Working with Matplotlib in a virtual enviroment see 'Working with Matplotlib in Virtual environments' in the Matplotlib FAQ
Got it to work using brew install python --framework
(https://github.com/gldnspud/virtualenv-pythonw-osx). Thank you for your attention.
Hi,
Just a quick question, I'm trying to fine tune the VGG16 as suggested in the notebook 2.4. However, I got an error: The shape of the input to "Flatten" is not fully defined (got (None, None, 512). Make sure to pass a complete "input_shape" or "batch_input_shape" argument to the first layer in your model.
. Not sure if anyone can give me a hint on this. My code is shown below:
from keras.applications import VGG16
model = VGG16(include_top=False)
for layer in model.layers:
layer.trainable = False
top_model = Sequential()
top_model.add(Flatten(input_shape=model.output_shape[1:]))
top_model.add(Dense(768, activation='sigmoid'))
top_model.add(Dropout(0.0))
top_model.add(Dense(768, activation='sigmoid'))
top_model.add(Dropout(0.0))
top_model.add(Dense(n_labels, activation='softmax'))
model.add(top_model)
train_model(model,
(X_train_gte5, y_train_gte5),
(X_test_gte5, y_test_gte5), nb_classes)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.