Tensor shape or OOM error for smoke3_mov200_f400 training about deep-fluids

[byungsook]

You can try the following code:

python main.py --is_train=False --load_path=AE_MODEL_DIR  --arch=ae --z_num=16 --max_epoch=20 --filter=64 --is_3d=True --dataset=smoke3_mov200_f400 --res_x=48 --res_y=72 --res_z=48 --test_batch_size=5

python main.py --arch=nn --code_path=AE_MODEL_DIR --w_size=30 --z_num=16 --filters=512 --max_epoch=200 --batch_size=1024 --is_3d=True --dataset=smoke3_mov200_f400 --res_x=48 --res_y=72 --res_z=48

python main.py --is_train=False --load_path=NN_MODEL_DIR --arch=nn --code_path=AE_MODEL_DIR --w_size=30 --z_num=16 --filters=512 --max_epoch=200 --batch_size=1024 --is_3d=True --dataset=smoke3_mov200_f400 --res_x=48 --res_y=72 --res_z=48

python main.py --is_train=False --load_path=AE_MODEL_DIR --code_path=NN_MODEL_DIR --arch=ae --z_num=16 --max_epoch=20 --filter=64 --is_3d=True --dataset=smoke3_mov200_f400 --res_x=48 --res_y=72 --res_z=48 --test_batch_size=5

Let me know if it works.

[Kshitij-Khode]

Sorry. I copy/pasted the wrong command in my initial post. I've updated it with the correct command now.

I used the following command (when dealing with OOM error),

python main.py --is_train=False --load_path=log/smoke3_mov200_f400/1213_021319_ae_tag/  --arch=ae --z_num=16 --max_epoch=20 --filter=64 --is_3d=True --dataset=smoke_mov200_f400 --res_x=48 --res_y=72 --res_z=48 --test_batch_size=1

Which is similar to what you suggested

python main.py --is_train=False --load_path=AE_MODEL_DIR  --arch=ae --z_num=16 --max_epoch=20 --filter=64 --is_3d=True --dataset=smoke3_mov200_f400 --res_x=48 --res_y=72 --res_z=48 --test_batch_size=5

I'm still getting OOM error. Did you use --filter=64? Or was a reduced value used in the training? Any other suggestions?

[Kshitij-Khode]

I was accidentally limiting the max memory allocation for tensorflow. I removed the limit and I'm now able to generate latent code set.

Thanks for updating run.bat with the relevant commands though. Really appreciate it.