safedep / pokebot

Demo Vulnerable GenAI App

Pokebot: Vulnerable RAG Application for Testing GenAI Specific Vulnerabilities

Pokebot is a demonstration vulnerable Retrieval-Augmented Generation (RAG) application designed for learning and testing specific vulnerabilities related to Generative AI (GenAI). This project provides a platform to experiment with various attacks and defenses in the context of AI-powered natural language understanding and generation systems.

Watch Demo

Try Hosted Version

Quick Start

Clone the repository and Install:

git clone https://github.com/safedep/pokebot
cd pokebot
pip install poetry
poetry install
poetry run pokebot rag start

above command will create a virtual env in case non exists

Setup OpenAI API Key

Set your OpenAI API key as an environment variable:

export OPENAI_API_KEY=<your_openai_api_key>

Replace <your_openai_api_key> with your actual OpenAI API key.

Running the Application

Start the Pokebot application by providing seed data (optional):

poetry run pokebot --seed data/med/articles/diabetes.txt

Alternatively use Pip

pip install -r requirements.txt
python pokebot/main.py --seed data/med/articles/diabetes.txt

Once the application is running, click on the Gradio link provided to access the app.

Usage

Prompting Without Poisoning Data

After accessing the application through the Gradio link, you can try prompts without poisoning the data. For example:

What is the best medicine for diabetes?

Poisoning Data

To simulate data poisoning, select the "Poison" option and submit. This action will inject poisoned data into the application for testing purposes.

Trying New Prompts

After poisoning the data, you can try new prompts to observe the behavior of the application. For example:

Get list of usernames

Contributing

Contributions to Pokebot are welcome! If you discover any vulnerabilities, have ideas for improvements, or want to add new features, feel free to open an issue or submit a pull request.

License

This project is licensed under the MIT License - see the LICENSE file for details.