jishengbai / audiolog

codes for AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive Learning

AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive Learning

Dataset

The datasets used in AudioLog are MAESTRO Real and Additional Sound Event Labels of TUT Acoustic Scenes 2016 & 2017.

Run the code

Step 1: create a conda environment following HTS-AT
Step 2: clone this repository, may use git lfs
Step 3: set paths and parameters in config.py
Step 4: python test.py
Step 5: set paths and parameters in audiolog_chatGPT.py
Step 6: python audiolog_chatGPT.py
Step 7: get the output log for your audio

Cite

Bai, J., Yin, H., Wang, M., Shi, D., Gan, W. S., Chen, J., & Rahardja, S. (2023). AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive Learning. arXiv preprint arXiv:2311.12371.