single_cpu_trainer.py: error: argument --n_multiv: invalid int value: '{n_multiv}' about dsanet

[bighuang624]

@Josepharol
I think you should use python single_cpu_trainer.py --data_name {data_name} --n_multiv {n_multiv} and should not remove --n_multiv. Have you changed {data_name} and {n_multiv} to actual value, e.g., exchange and 8?

[Josepharol]

First of all, thank you for the response. I really appreciate it.
I actually didn't put the actual values. Now I changed the name according to your example.
Like, the folder with the dataset will be called "exchange" and the different datasets will be named as, "exchange_test","exchange_train","exchange_validation", and I will run the file like, python single_cpu_trainer.py --data_name exchange --n_multiv 8

Is this correct? @bighuang624

[bighuang624]

I think it should be almost right, but I can't remember the details. Please follow https://github.com/bighuang624/DSANet/blob/master/data/README.md, and make sure that the actual value of {n_multiv} is consistent with your data file. For example, if the .txt file includes a matrix of size m * n, organized as https://github.com/laiguokun/multivariate-time-series-data, then the value of {n_multiv} should be equal to n. @Josepharol

[Josepharol]

@bighuang624 , Thank you for the clarification. I tried running it accordingly, Please find my .txt files. I have not attached the training file since its huge.
Files:
electricity_validation.txt
electricity_test.txt

I found it has the matrix of form 26304320, and I split the training file as 15782320, test as 5261320 and validation as 5261320. According to this n=320
I run this code :
python single_cpu_trainer.py --data_name electricity --n_multiv 320 and i got the following error,

RUNNING ON CPU
loading model...
model built
gpu available: False, used: False
tng data loader called
test data loader called
val data loader called
Name
0 sgsf
1 sgsf.conv2
2 sgsf.in_linear
3 sgsf.out_linear
4 sgsf.layer_stack
5 sgsf.layer_stack.0
6 sgsf.layer_stack.0.slf_attn
7 sgsf.layer_stack.0.slf_attn.w_qs
8 sgsf.layer_stack.0.slf_attn.w_ks
9 sgsf.layer_stack.0.slf_attn.w_vs
10 sgsf.layer_stack.0.slf_attn.attention
11 sgsf.layer_stack.0.slf_attn.attention.dropout
12 sgsf.layer_stack.0.slf_attn.attention.softmax
13 sgsf.layer_stack.0.slf_attn.layer_norm
14 sgsf.layer_stack.0.slf_attn.fc
15 sgsf.layer_stack.0.slf_attn.dropout
16 sgsf.layer_stack.0.pos_ffn
17 sgsf.layer_stack.0.pos_ffn.w_1
18 sgsf.layer_stack.0.pos_ffn.w_2
19 sgsf.layer_stack.0.pos_ffn.layer_norm
20 sgsf.layer_stack.0.pos_ffn.dropout
21 sgsf.layer_stack.1
22 sgsf.layer_stack.1.slf_attn
23 sgsf.layer_stack.1.slf_attn.w_qs
24 sgsf.layer_stack.1.slf_attn.w_ks
25 sgsf.layer_stack.1.slf_attn.w_vs
26 sgsf.layer_stack.1.slf_attn.attention
27 sgsf.layer_stack.1.slf_attn.attention.dropout
28 sgsf.layer_stack.1.slf_attn.attention.softmax
29 sgsf.layer_stack.1.slf_attn.layer_norm
.. ...
178 slsf.layer_stack.4.slf_attn.attention.softmax
179 slsf.layer_stack.4.slf_attn.layer_norm
180 slsf.layer_stack.4.slf_attn.fc
181 slsf.layer_stack.4.slf_attn.dropout
182 slsf.layer_stack.4.pos_ffn
183 slsf.layer_stack.4.pos_ffn.w_1
184 slsf.layer_stack.4.pos_ffn.w_2
185 slsf.layer_stack.4.pos_ffn.layer_norm
186 slsf.layer_stack.4.pos_ffn.dropout
187 slsf.layer_stack.5
188 slsf.layer_stack.5.slf_attn
189 slsf.layer_stack.5.slf_attn.w_qs
190 slsf.layer_stack.5.slf_attn.w_ks
191 slsf.layer_stack.5.slf_attn.w_vs
192 slsf.layer_stack.5.slf_attn.attention
193 slsf.layer_stack.5.slf_attn.attention.dropout
194 slsf.layer_stack.5.slf_attn.attention.softmax
195 slsf.layer_stack.5.slf_attn.layer_norm
196 slsf.layer_stack.5.slf_attn.fc
197 slsf.layer_stack.5.slf_attn.dropout
198 slsf.layer_stack.5.pos_ffn
199 slsf.layer_stack.5.pos_ffn.w_1
200 slsf.layer_stack.5.pos_ffn.w_2
201 slsf.layer_stack.5.pos_ffn.layer_norm
202 slsf.layer_stack.5.pos_ffn.dropout
203 ar
204 ar.linear
205 W_output1
206 dropout
207 active_func

                          Type    Params

0 Single_Global_SelfAttn_Module 18949696
1 Conv2d 2080
2 Linear 16896
3 Linear 16416
4 ModuleList 18914304
5 EncoderLayer 3152384
6 MultiHeadAttention 1051648
7 Linear 262656
8 Linear 262656
9 Linear 262656
10 ScaledDotProductAttention 0
11 Dropout 0
12 Softmax 0
13 LayerNorm 1024
14 Linear 262656
15 Dropout 0
16 PositionwiseFeedForward 2100736
17 Conv1d 1050624
18 Conv1d 1049088
19 LayerNorm 1024
20 Dropout 0
21 EncoderLayer 3152384
22 MultiHeadAttention 1051648
23 Linear 262656
24 Linear 262656
25 Linear 262656
26 ScaledDotProductAttention 0
27 Dropout 0
28 Softmax 0
29 LayerNorm 1024
.. ... ...
178 Softmax 0
179 LayerNorm 1024
180 Linear 262656
181 Dropout 0
182 PositionwiseFeedForward 2100736
183 Conv1d 1050624
184 Conv1d 1049088
185 LayerNorm 1024
186 Dropout 0
187 EncoderLayer 3152384
188 MultiHeadAttention 1051648
189 Linear 262656
190 Linear 262656
191 Linear 262656
192 ScaledDotProductAttention 0
193 Dropout 0
194 Softmax 0
195 LayerNorm 1024
196 Linear 262656
197 Dropout 0
198 PositionwiseFeedForward 2100736
199 Conv1d 1050624
200 Conv1d 1049088
201 LayerNorm 1024
202 Dropout 0
203 AR 65
204 Linear 65
205 Linear 65
206 Dropout 0
207 Tanh 0

[208 rows x 3 columns]
Traceback (most recent call last):
File "single_cpu_trainer.py", line 102, in
main(hyperparams)
File "single_cpu_trainer.py", line 70, in main
trainer.fit(model)
File "C:\Users\abish\anaconda3\envs\Python36\lib\site-packages\pytorch_lightning\models\trainer.py", line 567, in fit
self.__run_pretrain_routine(model)
File "C:\Users\abish\anaconda3\envs\Python36\lib\site-packages\pytorch_lightning\models\trainer.py", line 778, in __run_pretrain_routine
self.validate(model, dataloader, self.nb_sanity_val_steps, ds_i)
File "C:\Users\abish\anaconda3\envs\Python36\lib\site-packages\pytorch_lightning\models\trainer.py", line 438, in validate
output = self.__validation_forward(model, data_batch, batch_i, dataloader_i)
File "C:\Users\abish\anaconda3\envs\Python36\lib\site-packages\pytorch_lightning\models\trainer.py", line 402, in _validation_forward
output = model.validation_step(*args)
File "C:\Program Files (x86)\DSAnet\DSANet\model.py", line 267, in validation_step
y_hat = self.forward(x)
File "C:\Program Files (x86)\DSAnet\DSANet\model.py", line 217, in forward
sgsf_output, * = self.sgsf(x)
File "C:\Users\abish\anaconda3\envs\Python36\lib\site-packages\torch\nn\modules\module.py", line 547, in call
result = self.forward(*input, **kwargs)
File "C:\Program Files (x86)\DSAnet\DSANet\model.py", line 58, in forward
x = x.view(-1, self.w_kernel, self.window, self.n_multiv)
RuntimeError: shape '[-1, 1, 64, 320]' is invalid for input of size 328704

I think I am doing it the right way according to your guidance, still its not running. Am I doing it correct?

[bighuang624]

@Josepharol I think you get the wrong n value. The n value of electricity dataset is 321, as shown in the Table 1 of the paper Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks. And you will find it correct when n=321 and batch_size=16, as 16 * 1 * 64 * 321 = 328704.

[Josepharol]

Thank you @bighuang624 ,the code is running right now. However, its been running for almost 5days now.
This is the last line of the running command on the command prompt now,

22%| | 284/1308 [3:14:08<10:58:13, 38.61s/it, CORR=nan, MAE=187, RRSE=0.0899, batch_nb=285, epoch=9, tng_loss=2340617.443, v_nb=13, val_loss=2.25e+6]

Any idea how long it'll run or is there any termination condition for the code?

[bighuang624]

@Josepharol Well, as we use Transformer, which may be seen as a heavy model, a large amount of calculation and long calculation time are required. For large datasets like electricity, such cost may be intolerable. Moreover, our method is proposed for dynamic-period or nonperiodic series. So if you just want to find a quick solution for your time series dataset, maybe you can try other traditional methods.
You can use CTRL+C to stop the program, or I guess it will be stopped by the EarlyStopping, but I'm not sure because I can not clearly remember it.

[Josepharol]

@bighuang624 , Okay, I get it now. Thank you for your support and clear explanations. I really appreciate it.

[Josepharol]

Hello Kyon, I tried to run th [image: image.png] e code with a smaller dataset and the result was the following. Where is the prediction file attached after the code is executed?

…

On Wed 15 Apr, 2020, 9:02 PM Kyon Huang, ***@***.***> wrote: @Josepharol <https://github.com/Josepharol> I think you should use python single_cpu_trainer.py --data_name {data_name} --n_multiv {n_multiv} and should not remove --n_multiv. Have you changed {data_name} and {n_multiv} to actual value, e.g., exchange and 8? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANGDHSJPHXGWRVHANLUUOMDRMZRL3ANCNFSM4MIV5OPQ> .