toni-heittola / dcase2019_task1_baseline Goto Github PK

Description

Error when running python task1c.py. Task 1A worked just fine and produced similar results.

Expected Results

Expected Task 1C to download the dataset (it did), extract features, normalize, train, test and evaluate.

Actual Results

root@1e8de6cd35a5:~/dcase2019_task1_baseline# python task1c.py 
[I] DCASE2019 / Task1C -- Acoustic scene classification
[I] 
Download packages        :  62%|#####7   | 8/13 [19:08<14:35, 175.03s/it^
[E] Dataset errors:                                                                                                                         
FOLD[1]          TRAIN SET     Empty set                                                                                                    
FOLD[1]          TEST SET      Empty set
FOLD[1]          EVAL SET    Empty set              	(datasets.py:1326)
NoneType: None
[E] Uncaught exception 	(logging.py:221)
Traceback (most recent call last):
  File "task1c.py", line 1105, in <module>
    sys.exit(main(sys.argv))
  File "task1c.py", line 125, in main
    data_path=param.get_path('path.dataset'),
  File "/mnt/dsp/miniconda3/lib/python3.6/site-packages/dcase_util/datasets/datasets.py", line 688, in initialize
    self.check_metadata()
  File "/mnt/dsp/miniconda3/lib/python3.6/site-packages/dcase_util/datasets/datasets.py", line 1327, in check_metadata
    raise ValueError(message)
ValueError: Dataset errors:
FOLD[1]          TRAIN SET     Empty set
FOLD[1]          TEST SET      Empty set
FOLD[1]          EVAL SET    Empty set   

Versions

Linux-4.15.0-39-generic-x86_64-with-debian-buster-sid
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34) 
[GCC 7.3.0]
NumPy 1.16.2
SciPy 1.2.1
Matplotlib 3.0.2
librosa 0.6.2
keras 2.2.4
tensorflow 1.12.0
dcase_util 0.2.5
sed_eval 0.2.1

PS: dcase_util reports version 0.2.5, but the latest version (0.2.6) is installed.

I'm not sure if this is officially supported already, but since there are leaderboard results at the Kaggle site, I suppose it should work. Default mode seems to work as expected. I used the latest version and setup a new environment from scratch, just to be on the safe side.

The download of the leaderboard dataset did not start, and I tried to add it manually to the default path, but to no avail. I attached the log, maybe this helps.

OS: WIN10
Environment:
_tflow_select 2.1.0 gpu
absl-py 0.7.1 py37_0
astor 0.7.1 py37_0
attrs 19.1.0 py37_1
audioread 2.1.8 pypi_0 pypi
backcall 0.1.0 py37_0
blas 1.0 mkl
bleach 3.1.0 py37_0
ca-certificates 2019.1.23 0
certifi 2019.3.9 py37_0
cffi 1.12.3 pypi_0 pypi
chardet 3.0.4 pypi_0 pypi
colorama 0.4.1 py37_0
cudatoolkit 10.0.130 0
cudnn 7.3.1 cuda10.0_0
cycler 0.10.0 pypi_0 pypi
dcase-util 0.2.10 pypi_0 pypi
decorator 4.4.0 py37_1
defusedxml 0.6.0 py_0
entrypoints 0.3 py37_0
future 0.17.1 pypi_0 pypi
gast 0.2.2 py37_0
grpcio 1.16.1 py37h351948d_1
h5py 2.9.0 py37h5e291fa_0
hdf5 1.10.4 h7ebc959_0
icc_rt 2019.0.0 h0cc432a_1
icu 58.2 ha66f8fd_1
idna 2.8 pypi_0 pypi
intel-openmp 2019.3 203
ipykernel 5.1.0 py37h39e3cac_0
ipython 7.5.0 py37h39e3cac_0
ipython_genutils 0.2.0 py37_0
ipywidgets 7.4.2 py37_0
jedi 0.13.3 py37_0
jinja2 2.10.1 py37_0
joblib 0.13.2 pypi_0 pypi
jpeg 9b hb83a4c4_2
jsonschema 3.0.1 py37_0
jupyter 1.0.0 py37_7
jupyter_client 5.2.4 py37_0
jupyter_console 6.0.0 py37_0
jupyter_core 4.4.0 py37_0
keras 2.2.4 pypi_0 pypi
keras-applications 1.0.7 py_0
keras-preprocessing 1.0.9 py_0
kiwisolver 1.1.0 pypi_0 pypi
libpng 1.6.37 h2a8f88b_0
libprotobuf 3.7.1 h7bd577a_0
librosa 0.6.3 pypi_0 pypi
libsodium 1.0.16 h9d3ae62_0
llvmlite 0.29.0 pypi_0 pypi
m2w64-gcc-libgfortran 5.3.0 6
m2w64-gcc-libs 5.3.0 7
m2w64-gcc-libs-core 5.3.0 7
m2w64-gmp 6.1.0 2
m2w64-libwinpthread-git 5.0.0.4634.697f757 2
markdown 3.1 py37_0
markupsafe 1.1.1 py37he774522_0
matplotlib 3.1.0 pypi_0 pypi
mistune 0.8.4 py37he774522_0
mkl 2019.3 203
mkl_fft 1.0.12 py37h14836fe_0
mkl_random 1.0.2 py37h343c172_0
mock 3.0.5 py37_0
msys2-conda-epoch 20160418 1
nbconvert 5.5.0 py_0
nbformat 4.4.0 py37_0
notebook 5.7.8 py37_0
numba 0.44.0 pypi_0 pypi
numpy 1.16.4 py37h19fb1c0_0
numpy-base 1.16.4 py37hc3f5095_0
openssl 1.1.1c he774522_1
pandoc 2.2.3.2 0
pandocfilters 1.4.2 py37_1
parso 0.4.0 py_0
pickleshare 0.7.5 py37_0
pip 19.1.1 py37_0
prometheus_client 0.6.0 py37_0
prompt_toolkit 2.0.9 py37_0
protobuf 3.7.1 py37h33f27b4_0
pycparser 2.19 pypi_0 pypi
pydot-ng 2.0.0 pypi_0 pypi
pygments 2.4.0 py_0
pyparsing 2.4.0 pypi_0 pypi
pyqt 5.9.2 py37h6538335_2
pyreadline 2.1 py37_1
pyrsistent 0.14.11 py37he774522_0
python 3.7.3 h8c8aaf0_1
python-dateutil 2.8.0 py37_0
python-magic 0.4.15 pypi_0 pypi
pywinpty 0.5.5 py37_1000
pyyaml 5.1 pypi_0 pypi
pyzmq 18.0.0 py37ha925a31_0
qt 5.9.7 vc14h73c81de_0
qtconsole 4.5.1 py_0
requests 2.22.0 pypi_0 pypi
resampy 0.2.1 pypi_0 pypi
scikit-learn 0.21.2 pypi_0 pypi
scipy 1.2.1 py37h29ff71c_0
sed-eval 0.2.1 pypi_0 pypi
send2trash 1.5.0 py37_0
setuptools 41.0.1 py37_0
sip 4.19.8 py37h6538335_0
six 1.12.0 py37_0
soundfile 0.10.2 pypi_0 pypi
sqlite 3.28.0 he774522_0
tensorboard 1.13.1 py37h33f27b4_0
tensorflow 1.13.1 gpu_py37h83e5d6a_0
tensorflow-base 1.13.1 gpu_py37h871c8ca_0
tensorflow-estimator 1.13.0 py_0
tensorflow-gpu 1.13.1 h0d30ee6_0
termcolor 1.1.0 py37_1
terminado 0.8.2 py37_0
testpath 0.4.2 py37_0
tornado 6.0.2 py37he774522_0
tqdm 4.32.1 pypi_0 pypi
traitlets 4.3.2 py37_0
urllib3 1.25.3 pypi_0 pypi
validators 0.13.0 pypi_0 pypi
vc 14.1 h0510ff6_4
vs2015_runtime 14.15.26706 h3a45250_4
wcwidth 0.1.7 py37_0
webencodings 0.5.1 py37_1
werkzeug 0.15.2 py_0
wheel 0.33.4 py37_0
widgetsnbextension 3.4.2 py37_0
wincertstore 0.2 py37_0
winpty 0.4.3 4
zeromq 4.3.1 h33f27b4_3
zlib 1.2.11 h62dcd97_3

task1a.log

Hi,
I have a problem with my implementation. I try to implement a RNN and the parameters which i use are shown below.

So, i get this error: Input 0 is incompatible with layer lstm_1: expected ndim=3, found ndim=4.

As you can see at the image below, I reshaped my X_train and X_validation array, but this error still exists.

Can you help me?

PS I use DCASE 2018 implementation, sorry about that. But I think it doesn't matter about this error.

Thanks in advance!

-Athanasiou Giannos

Description

Error when running python task1*.py. I tested Tasks A and C.
This is probably a problem with some API change on newer versions of Keras.

Forcing

if len(keras_model.input_shape) == 4:
    input_data = numpy.expand_dims(input_data, 3)

fixed the problem, but hurts the code generalization.

Expected Results

Expected Task test the trained system. Feature extraction, normalization and learning run just fine.

Actual Results

root@1e8de6cd35a5:~/dcase2019_task1_baseline# python task1a.py 
[I] DCASE2019 / Task1A -- Acoustic scene classification
[I] 
[I] General information
[I] ========================================
[I]   Parameter set
[I]     Set ID                          : dcase2019_baseline 
[I]     Set description                 : DCASE2019 baseline / Development setup 
[I]   Application
[I]     Overwrite                       : False 
[I]     Dataset                         : TAU-urban-acoustic-scenes-2019-development 
[I]     Active folds                    : [1] 
[I] 
[I]   DONE       
[I] 
[I] Feature Extraction
[I] ========================================
[I]   DONE       [0:00:02.002628] 
[I] 
[I] Feature Normalization
[I] ========================================
[I]   Fold [1]
[I]   DONE       [0:00:00.000101] 
[I] 
[I] Learning
[I] ========================================
[I]   Fold [1]
[I]   DONE       [0:00:00.000096] 
[I] 
[I] Testing
[I] ========================================
[I]   Fold [1]
[I]   === Keras setup ===
[I]     BLAS library                    : MKL (Threads[8], MKL_CBWR[COMPATIBLE]) 
[I]     Backend                         : tensorflow 
[I]     Tensorflow
2019-03-12 16:19:17.177174: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-03-12 16:19:17.177210: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-03-12 16:19:17.177223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0 
2019-03-12 16:19:17.177233: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N 
2019-03-12 16:19:17.177423: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/device:GPU:0 with 11376 MB memory) -> physical GPU (device: 0, name: TITAN X (Pascal), pci bus id: 0000:09:00.0, compute capability: 6.1)
[I]       Device                        : cuda0 
[I]   DONE       
[I] 
[E] Uncaught exception 	(logging.py:221)
Traceback (most recent call last):
  File "task1a.py", line 1226, in <module>
    sys.exit(main(sys.argv))
  File "task1a.py", line 234, in main
    overwrite=overwrite
  File "task1a.py", line 974, in do_testing
    if keras_model.get_config()[0]['config']['data_format'] == 'channels_first':
KeyError: 0

Versions

Linux-4.15.0-39-generic-x86_64-with-debian-buster-sid
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34) 
[GCC 7.3.0]
NumPy 1.16.2
SciPy 1.2.1
Matplotlib 3.0.2
librosa 0.6.2
keras 2.2.4
tensorflow 1.12.0
dcase_util 0.2.7
sed_eval 0.2.1