As a part of a course in Generative AI and Media and interactive design in KTH, we have developed this Multimodal RAG applicaation.
Some of the sample data are there in data/images folder, you can use your own data for this application.
you need to install weaviate database in your local machine using docker and the docker-compose.yml file. for more info: https://weaviate.io/developers/weaviate/installation/docker-compose
Make a env in your python, install the requirements.txt, run >python add_data_in_batch.py . As you may have many images, so in the code we are inserting the 200 images at a time and embedd those. It may take time depending on the numbers of images.
in the env, run streamlit run app_KTH_Final.py . To generate the description, you need to have Open AI API key. You can have that from: https://platform.openai.com/
Without the Open AI API key the application will work but user can't get the generated description from the vision model. The text and image semantic search will work perfectly.