shehabshalan / generating_and_analyzing_tweet_style_disinformation_with_llms

Generating And Analyzing Tweet-Style Disinformation with LLMs - Master Thesis

This repo contains the latest results from the experiments conducted to generate tweet-style disinformation using state-of-the-art prompting techniques including both closed and open source models.

Notebook content

The code can be found in disinformation_generation.ipynb. The notebook contains three sections:

• Data preparation: where the disinformation claims collected from fact-checking websites are added.
• Disinformation generation: code responsible for generating disinformation using various jailbreaking techniques across three different models namely GPT-3.5 Turbo, Vicuna, and WizardLM.
• Analysis: code responsible for analyzing the generated disinformation.

How to run

• Create a new virtual environment and install the required packages. pip install -r requirements.txt.
• Create a .env file and add API keys from OpenAI and TogtherAI. See example.env for reference.

Generated Datasets:

• responses.csv: contains the generated responses from the models.
• responses_annotated.csv: contains the generated responses from the models with the corresponding labels manually annotated.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.