What's the content of path:'../Gesture-Generation-from-Trimodal-Context/double_feat' ? about seeg

[akira-l]

I used to extract beat data offline and save it in this folder.
You can also extract the beat signal online following the 'proc_audio' function in this class.

[QyDing02]

It appears that the proc_audio function is not functioning properly, as nothing is being saved to the ../Gesture-Generation-from-Trimodal-Context/double_feat folder. I would greatly appreciate it if you could shed some light on what might be happening. Here is the complete error message:

2023-03-22 19:55:39,728: PyTorch version: 1.10.2
2023-03-22 19:55:39,728: CUDA version: 11.3
2023-03-22 19:55:39,728: 8 GPUs, default cuda:0
2023-03-22 19:55:39,730: {'GAN_noise_size': 0,
 'batch_size': 256,
 'config': 'config/multimodal_context_beat_ep200.yml',
 'discriminator_lr_weight': 0.2,
 'dropout_prob': 0.3,
 'epochs': 200,
 'eval_net_path': 'output/train_h36m_gesture_autoencoder/gesture_autoencoder_checkpoint_best.bin',
 'freeze_wordembed': False,
 'hidden_size': 300,
 'input_context': 'audio',
 'learning_rate': 0.0005,
 'loader_workers': 16,
 'loss_gan_weight': 5.0,
 'loss_kld_weight': 0.1,
 'loss_reg_weight': 0.05,
 'loss_regression_weight': 500.0,
 'loss_warmup': 10,
 'mean_dir_vec': [[0.0154009],                  [-0.9690125],                  [-0.0884354],                  [-0.0022264],                  [-0.8655276],                  [0.4342174],                  [-0.0035145],                  [-0.8755367],                  [-0.4121039],                  [-0.9236511],                  [0.3061306],                  [-0.0012415],                  [-0.5155854],                  [0.8129665],                  [0.0871897],                  [0.2348464],                  [0.1846561],                  [0.8091402],                  [0.9271948],                  [0.2960011],                  [-0.013189],                  [0.5233978],                  [0.8092403],                  [0.0725451],                  [-0.2037076],                  [0.1924306],                  [0.8196916]],
 'mean_pose': [[3.06e-05],               [0.0004946],               [0.0008437],               [0.0033759],               [-0.2051629],               [-0.0143453],               [0.0031566],               [-0.3054764],               [0.0411491],               [0.0029072],               [-0.4254303],               [-0.001311],               [-0.1458413],               [-0.1505532],               [-0.0138192],               [-0.2835603],               [0.0670333],               [0.0107002],               [-0.2280813],               [0.112117],               [0.2087789],               [0.1523502],               [-0.1521499],               [-0.0161503],               [0.291909],              [0.0644232],               [0.0040145],               [0.2452035],               [0.1115339],               [0.2051307]],
 'model': 'multimodal_context',
 'model_save_path': 'output/train_beat_with-cat_emb_with-lrelu_200ep_dis-add-layer_2023-03-22-19_55_39',
 'motion_resampling_framerate': 15,
 'n_layers': 4,
 'n_poses': 34,
 'n_pre_poses': 4,
 'name': 'multimodal_context',
 'pose_representation': '3d_vec',
 'random_seed': -1,
 'save_result_video': True,
 'subdivision_stride': 10,
 'test_data_path': ['data/ted_dataset/lmdb_test'],
 'train_data_path': ['data/ted_dataset/lmdb_train'],
 'val_data_path': ['data/ted_dataset/lmdb_val'],
 'wordembed_dim': 300,
 'wordembed_path': 'data/fasttext/crawl-300d-2M-subword.bin',
 'z_type': 'speaker'}
2023-03-22 19:55:39,736: Reading data 'data/ted_dataset/lmdb_train'...
2023-03-22 19:55:39,736: Found the cache data/ted_dataset/lmdb_train_cache
2023-03-22 19:55:39,740: Reading data 'data/ted_dataset/lmdb_val'...
2023-03-22 19:55:39,740: Found the cache data/ted_dataset/lmdb_val_cache
2023-03-22 19:55:39,741: Reading data 'data/ted_dataset/lmdb_test'...
2023-03-22 19:55:39,741: Found the cache data/ted_dataset/lmdb_test_cache
2023-03-22 19:55:39,741:   building a language model...
2023-03-22 19:55:39,741:     loaded from data/ted_dataset/vocab_cache.pkl
Traceback (most recent call last):
  File "scripts/train.py", line 568, in <module>
    main({'args': _args})
  File "scripts/train.py", line 563, in main
    pose_dim=pose_dim, speaker_model=train_dataset.speaker_model)
  File "scripts/train.py", line 136, in train_epochs
    val_metrics = evaluate_testset(test_data_loader, beat_generator, loss_fn, embed_space_evaluator, args)
  File "scripts/train.py", line 301, in evaluate_testset
    for iter_idx, data in enumerate(test_data_loader):
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
    return self._process_data(data)
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
    data.reraise()
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/_utils.py", line 434, in reraise
    raise exception
FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/mnt/SSD_3TB/xxx/code/SEEG/scripts/data_loader/lmdb_data_loader.py", line 260, in __getitem__
    aux_key + '.pt'))
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/serialization.py", line 594, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/serialization.py", line 230, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/serialization.py", line 211, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '../Gesture-Generation-from-Trimodal-Context/double_feat/IGadg8xHVPI_1968_2010.pt'

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
  File "/home/xxx/software/miniconda3/envs/SEEG/lib/python3.6/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
    _error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 1091338) is killed by signal: Terminated.

[akira-l]

proc_audio function is used for producing beat data.
You can save the beat data offline and read saved features everytimes as line262-263.