18907305772 / kca

Knowledge Verification to Nip Hallucination in the Bud

Hi there 👋

My name is Fanqi Wan (万凡琦). You can call me Fanqi.

• 🌱 I am a second-year MS student (expected to graduate in 2025) at Sun Yat-sen University, advised by Prof. Xiaojun Quan. Before this, I received my Bachelor's degree (2018-2022, automation) from Xi'an Jiaotong University.
• 👯 I am currently conducting my internship at Tencent AI Lab (2023.03-Now), where I am mentored by Dr. Xinting Huang and Dr. Wei Bi.
• 🤔 My main research interests focused on deep learning for natural language generation. Previously, my work primarily focused on dialogue systems. After the emergence of large language models (LLMs), my research direction shifted towards instruction-tuning (e.g., developing LLMs for specific domains, mitigating hallucinations of LLMs) and model fusion (e.g., combining the capabilities of multiple structurally diverse LLMs).
• 📫 How to reach me: E-mail

View my homepage.

kca's Issues

请问有中文的prompt

prompt_bank下有三个英文的prompt, 请问有对应的效果不错的中文prompt吗?

Reproducing LIMAEval results

Hi,
thank you for the interesting work!

I am trying to reproduce the results for LLaMA-2-7b on LIMAEval for the discard method.
I ran the evaluation script after generating with the release model KCA_Llama_2_7B_Discarding_Tuning using the default setting which calls gpt-4.

My results were slightly different from the result reported in the paper (30.95).

"hallucination_judge": {
"all_scores": 0.3220338983050847,
"error_cnt": 0
,}

I initially thought this was caused by the API model update.
However, the snapshot of gpt-4 points to gpt-4-0613 according to the OpenAI documentation.

Do you have a guess at why this might be happening?

For the record, on MS MACRO the ROUGE scores are also slightly off compared to Table 2

{
"ROUGE-1": 31.27,
"ROUGE-2": 20.0,
"ROUGE-L": 27.57,
"ROUGE-Lsum": 27.7
}

Thanks!

请问有中文的prompt吗

prompt_bank下有三个英文的prompt, 请问有对应的效果不错的中文prompt吗?