nicozwy / aigtd-survey

The official GitHub page for the survey paper of AIGTD entitled "Survey on Recent Advances in AI-Generated Text Detection"

Survey on Recent Advances in AI-Generated Text Detection

Collection of papers and resources for AIGTD

The papers are organized according to our AIGTD survey.

Note: GitHub is being used as a resource library, and we will continue updating it to refine and perfect this survey.

News and Updates

2024.1 Start the project

Table of Contents

• Tackle Classifier-Training
• Tackle Intrinsic-Attributes
• Tackle Information-Embedding
• Popular-Dataset-collection
• Citation

Tackle Classifier-Training

1. Chuck Rosenberg, Martial Hebert, and Henry Schneiderman. Semisupervised self-training of object detection models. In Proceedings of the Seventh IEEE Workshops on Application of Computer Vision, pages 29–36, 2005. [paper]

2. Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 761–769, 2016. [paper]

3. Beliz Gunel, Jingfei Du, Alexis Conneau, and Veselin Stoyanov. Supervised contrastive learning for pre-trained language model finetuning. In Proceedings of the International Conference on Learning Representations, pages 1–15, 2021. [paper]

Feature Analysis

Structural-based Analysis

1. Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Hang Pu, Yu Lan, and Chao Shen. Coco: Coherence-enhanced machine-generated text detection under data limitation with contrastive learning. arXiv preprint arXiv:2212.10341, 2022. [paper]

Partial Access

1. Yi Xu, Jie Hu, Zhiqiao Gao, and Jinpeng Chen. Ucl-ast: Active self-training with uncertainty-aware clouded logits for few-shot text classification. In Proceedings of the 2022 IEEE 34th International Conference on Tools with Artificial Intelligence (ICTAI), pages 1390–1395. IEEE, 2022. [paper]

2. Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang, and Xipeng Qiu. Seqxgpt: Sentence-level ai-generated text detection. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1144–1156, 2023. [paper]

Network Reconstruction

1. Guanhua Huang, Yuchen Zhang, Zhe Li, Yongjian You, Mingze Wang, and Zhouwang Yang. Are ai-generated text detectors robust to adversarial perturbations? arXiv preprint arXiv:2406.01179, 2024. [paper]

Probability and Statistics

Probability-based Model

1. Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng, and Tat-Seng Chua. Llmdet: A third party large language models generated text detection tool. In Findings of the Association for Computational Linguistics: EMNLP, pages 2113–2133, 2023. [paper]

2. Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. Ghostbuster: Detecting text ghostwritten by large language models. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, pages 1702–1717, 2024. [paper]

Deep Learning

Positive Unlabeled

1. Yuchuan Tian, Hanting Chen, Xutao Wang, Zheyuan Bai, Qinghua Zhang, Ruifeng Li, Chao Xu, and Yunhe Wang. Multiscale positive unlabeled detection of ai-generated texts. In Proceedings of the International Conference on Learning Representations, 2024. [paper]

Adversarial Training

1. Ying Zhou, Ben He, and Le Sun. Humanizing machine-generated content: Evading ai-text detection through adversarial attack. In Proceedings of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation, pages 8427–8437, 2024. [paper]

2. Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. Radar: Robust ai-text detection via adversarial learning. In Proceedings of the Advances in Neural Information Processing Systems, 36:15077–15095, 2023. [paper]

Transfer Training

1. Eric Chu, Jacob Andreas, Stephen Ansolabehere, and Deb Roy. Language models trained on media diets can predict public opinion. arXiv preprint arXiv:2303.16779, 2023. [paper]

2. Hans WA Hanley and Zakir Durumeric. Machine-made media: Monitoring the mobilization of machine-generated articles on misinformation and mainstream news websites. In Proceedings of the International AAAI Conference on Web and Social Media, volume 18, pages 542–556, 2024. [paper]

3. Amrita Bhattacharjee, Tharindu Kumarage, Raha Moraffah, and Huan Liu. Conda: Contrastive domain adaptation for ai-generated text detection. In Proceedings of the International Joint Conference on Natural Language Processing and the Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, pages 598610, 2023. [paper]

BERT-based

1. Hao Wang, Jianwei Li, and Zhengyu Li. Ai-generated text detection and classification based on bert deep learning algorithm. arXiv preprint arXiv:2405.16422, 2024. [paper]

Name	Black box	White box
GCN [paper]	✔️
Logits as waves [paper]		✔️
SeqXGPT [paper]		✔️
SCRN [paper]		✔️
Proxy perplexity [paper]	✔️
Ghostbuster [paper]	✔️
MPU [paper]		✔️
RADAR [paper]	✔️
conDA [paper]	✔️
BERT-based [paper]		✔️

Tackle Intrinsic-Attributes

1. Nathan Benaich and Ian Hogarth. State of ai report. London, UK, 2020. [paper]
2. Yuhong Mo, Hao Qin, Yushan Dong, Ziyi Zhu, and Zhenglin Li. Large language model (llm) ai text generation detection based on transformer deep learning algorithm. International Journal of Engineering and Management Research, 14(2):154–159, 2024. [paper]
3. Rongsheng Wang, Haoming Chen, Ruizhe Zhou, Han Ma, Yaofei Duan, Yanlan Kang, Songhua Yang, Baoyu Fan, and Tao Tan. Llm-detector: Improving ai-generated chinese text detection with open-source llm instruction tuning. arXiv preprint arXiv:2402.01158, 2024. [paper]
4. Farhad Pourpanah, Moloud Abdar, Yuxuan Luo, Xinlei Zhou, Ran Wang, Chee Peng Lim, Xi-Zhao Wang, and QM Jonathan Wu. A review of generalized zero-shot learning methods. IEEE transactions on pattern analysis and machine intelligence, 45(4):4051–4070, 2022. [paper]
5. Wei Wang, Vincent W Zheng, Han Yu, and Chunyan Miao. A survey of zero-shot learning: Settings, methods, and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1–37, 2019. [paper]

Feature Extraction

Logarithmic Ranking

1. Jinyan Su, Terry Zhuo, Di Wang, and Preslav Nakov. Detectllm: Leveraging log rank information for zero-shot detection of machine generated text. In Findings of the Association for Computational Linguistics: EMNLP, pages 12395–12412, 2023. [paper]

N-gram with BScore

1. Xianjun Yang, Wei Cheng, Yue Wu, Linda Petzold, William Yang Wang, and Haifeng Chen. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt-generated text. In Proceedings of the International Conference on Learning Representations, pages 1–26, 2024. [paper]

Internal Dimension

1. Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, Sergey Nikolenko, Evgeny Burnaev, Serguei Barannikov, and Irina Piontkovskaya. Intrinsic dimension estimation for robust detection of ai-generated texts. In Proceedings of the Advances in Neural Information Processing Systems, 36, 2024. [paper]

Probability-based

Conditional Probability

1. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the International Conference on Machine Learning, pages 24950–24962. PMLR, 2023. [paper]
2. Shengchao Liu, Xiaoming Liu, Yichen Wang, Zehua Cheng, Chengzhengxu Li, Zhaohan Zhang, Yu Lan, and Chao Shen. Does∖textsc {DetectGPT} fully utilize perturbation? selective perturbation on model-based contrastive learning detector would be better. arXiv preprint arXiv:2402.00263, 2024. [paper]

Probability Curvature

1. Niloofar Mireshghallah, Justus Mattern, Sicun Gao, Reza Shokri, and Taylor Berg-Kirkpatrick. Smaller language models are better black-box machine-generated text detectors. arXiv preprint arXiv:2305.09859, 2023. [paper]
2. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the International Conference on Machine Learning, pages 24950–24962. PMLR, 2023. [paper]
3. Xianjun Yang, Wei Cheng, Yue Wu, Linda Petzold, William Yang Wang, and Haifeng Chen. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt-generated text. In Proceedings of the International Conference on Learning Representations, pages 1–26, 2024. [paper]
4. Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-detectgpt: Efficient zero-shot detection of machine generated text via conditional probability curvature. In Proceedings of the International Conference on Learning Representations, pages 1–23, 2024. [paper]

Distribution Difference

1. Shuhai Zhang, Feng Liu, Jiahao Yang, Yifan Yang, Changsheng Li, Bo Han, and Mingkui Tan. Detecting machine-generated texts by multi-population aware optimization for maximum mean discrepancy. arXiv preprint arXiv:2402.16041, 2024. [paper]

Epidemic Model

BERT-powered

1. Utsho Chakraborty, Jaydeep Gheewala, Sheshang Degadwala, Dhairya Vyas, and Mukesh Soni. Safeguarding authenticity in text with bert-powered detection of ai-generated content. In Proceedings of the 2024 International Conference on Inventive Computation Technologies (ICICT), pages 34–37. IEEE, 2024 [paper]

ChatGPT

1. David M Markowitz, Jeffrey T Hancock, and Jeremy N Bailenson. Linguistic markers of inherently false ai communication and intentionally false human communication: Evidence from hotel reviews. Journal of Language and Social Psychology, 43(1):63–82, 2024. [paper]

Model Mixing

1. Yuhong Mo, Hao Qin, Yushan Dong, Ziyi Zhu, and Zhenglin Li. Large language model (llm) ai text generation detection based on transformer deep learning algorithm. International Journal of Engineering and Management Research, 14(2):154–159, 2024. [paper]

Name	Black box	White box
LRR [paper]		✔️
N-Gram [paper]	✔️
Inter Dimension [paper]	✔️
DetectGPT [paper] [paper]	✔️
OPT-125M [paper]	✔️
Divergence [paper]		✔️
Curvature [paper]	✔️	✔️
MMD [paper]	✔️
BERT-powered [paper]	✔️
ChatGPT [paper]	✔️
Mixing [paper]		✔️

Tackle Information-Embedding

1. Mercan Topkara, Cuneyt M Taskiran, and Edward J Delp III. Natural language watermarking. In Security, Steganography, and Watermarking of Multimedia Contents VII, volume 5681, pages 441–452. SPIE, 2005. [paper]

2. Umut Topkara, Mercan Topkara, and Mikhail J Atallah. The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions. In Proceedings of the 8th workshop on Multimedia and security, pages 164–174, 2006. [paper]

3. Xi Yang, Jie Zhang, Kejiang Chen, Weiming Zhang, Zehua Ma, Feng Wang, and Nenghai Yu. Tracing text provenance via context-aware lexical substitution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11613–11621, 2022. [paper]

4. Xi Yang, Kejiang Chen, Weiming Zhang, Chang Liu, Yuang Qi, Jie Zhang, Han Fang, and Nenghai Yu. Watermarking text generated by black-box language models. arXiv preprint arXiv:2305.08883, 2023. [paper]

5. Wenjie Qu, Dong Yin, Zixin He, Wei Zou, Tianyang Tao, Jinyuan Jia, and Jiaheng Zhang. Provably robust multi-bit watermarking for ai-generated text via error correction code. arXiv preprint arXiv:2401.16820, 2024. [paper]

Training-free

Logits Deviation

1. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In Proceedings of the International Conference on Machine Learning, pages 17061–17084. PMLR, 2023. [paper]

2. Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. Provable robust watermarking for ai-generated text. In Proceedings of the International Conference on Learning Representations, pages 1–35, 2024. [paper]

Hash-based

1. Abe Bohan Hou, Jingyu Zhang, Tianxing He, Yichen Wang, YungSung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. Semstamp: A semantic watermark with paraphrastic robustness for text generation. arXiv preprint arXiv:2310.03991, 2023. [paper]

2. Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang. Dipmark: A stealthy, efficient and resilient watermark for large language models. In Proceedings of the International Conference on Learning Representations, pages 1–27, 2024. [paper]

3. Abe Bohan Hou, Jingyu Zhang, Yichen Wang, Daniel Khashabi, and Tianxing He. k-semstamp: A clustering-based semantic watermark for detection of machine-generated text. arXiv preprint arXiv:2402.11399, 2024. [paper]

Message Decoding

1. Xuandong Zhao, Lei Li, and Yu-Xiang Wang. Permute-and-flip: An optimally robust and watermarkable decoder for llms. arXiv preprint arXiv:2402.05864, 2024. [paper]

2. Scott Aaronson, Jiahui Liu, Qipeng Liu, Mark Zhandry, and Ruizhe Zhang. New approaches for quantum copy-protection. In Proceedings of the Advances in Cryptology–CRYPTO 2021: 41st Annual International Cryptology Conference, CRYPTO 2021, Virtual Event, August 16–20, 2021, Proceedings, Part I 41, pages 526–555. Springer, 2021. [paper]

Training-based

Message Encoding

1. Han Fang, Zhaoyang Jia, Hang Zhou, Zehua Ma, and Weiming Zhang. Encoded feature enhancement in watermarking network for distortion in real scenes. IEEE Transactions on Multimedia, 2022. [paper]

2. Ruisi Zhang, Shehzeen Samarah Hussain, Paarth Neekhara, and Farinaz Koushanfar. Remark-llm: A robust and efficient watermarking framework for generative large language models. In Proceedings of the USENIX Security Symposium, 2024. [paper]

Information Capacity

Multi-bit

1. KiYoon Yoo, Wonhyuk Ahn, Jiho Jang, and Nojun Kwak. Robust multi-bit natural language watermarking through invariant features. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2092–2115, 2023. [paper]

2. Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. Three bricks to consolidate watermarks for large language models. In 2023 IEEE International Workshop on Information Forensics and Security (WIFS), pages 1–6. IEEE, 2023. [paper]

3. Massieh Kordi Boroujeny, Ya Jiang, Kai Zeng, and Brian Mark. Multi-bit distortion-free watermarking for large language models. arXiv preprint arXiv:2402.16578, 2024. [paper]

Popular-Dataset-collection

Datasets	Size	Data Description
TuringBench [paper]	200,000	News articles
....

Citation

If you find this project useful in your research or work, please consider citing it.

Acknowledgements

Your contributions will be acknowledged.

Github Flavored Markdown