This repository contains experiments and code for the paper titled "Zero-Shot Content-Based Recommender System (ZCCR)." The ZCCR system leverages the CLIP and ALBEF architectures for content-based recommendations.
This folder contains dependencies for the CLIP architecture.
This folder contains dependencies for the ALBEF architecture. To use ALBEF, download the 14M pretrained .pth
checkpoint from ALBEF GitHub.
This folder contains Input annotations file and images for Flickr. Download Flickr images from Kaggle - Flickr Image Dataset. Download MSCOCO images from MSCOCO Website. Use karpathy_coco_split.json
as the annotation file with captions. MSCOCO classified
and FLICKR30k classified
: Processed versions of MSCOCO and Flickr30k with associated tags for both images and captions.
Contains scripts to generate classified MSCOCO and FLICKR30k annotations along with associated images.
Experiments of CLIP and ALBEF on MSCOCO and FLICKR30k 1k validation splits.
Comparison of search time between one and two FAISS indexes.
Comparison of ZCCR with Baseline Tagger (BT) and Baseline Tagger + BERT encoding + Agglomerative clustering (BTBA).
Output charts of the ablation study of clustering components and comparisons among baseline and ZCCR.
Use charts.ipynb
to generate charts from the raw results CSV available in the results
folder.
python -m venv venv
source venv/bin/activate # Linux/macOS
.\venv\Scripts\activate # Windows
pip install -r requirements.txt