fedasaro62 / zccr-exp

Zero-Shot Content-Based Recommender System (ZCCR)

This repository contains experiments and code for the paper titled "Zero-Shot Content-Based Recommender System (ZCCR)." The ZCCR system leverages the CLIP and ALBEF architectures for content-based recommendations.

Folder Structure

1. clip

This folder contains dependencies for the CLIP architecture.

2. albef

This folder contains dependencies for the ALBEF architecture. To use ALBEF, download the 14M pretrained .pth checkpoint from ALBEF GitHub.

3. data

This folder contains Input annotations file and images for Flickr. Download Flickr images from Kaggle - Flickr Image Dataset. Download MSCOCO images from MSCOCO Website. Use karpathy_coco_split.json as the annotation file with captions. MSCOCO classified and FLICKR30k classified: Processed versions of MSCOCO and Flickr30k with associated tags for both images and captions.

4. preprocessing

Contains scripts to generate classified MSCOCO and FLICKR30k annotations along with associated images.

5. retrieval

Experiments of CLIP and ALBEF on MSCOCO and FLICKR30k 1k validation splits.

6. search-time

Comparison of search time between one and two FAISS indexes.

7. tagger

Comparison of ZCCR with Baseline Tagger (BT) and Baseline Tagger + BERT encoding + Agglomerative clustering (BTBA).

8. charts

Output charts of the ablation study of clustering components and comparisons among baseline and ZCCR.

Generating Charts

Use charts.ipynb to generate charts from the raw results CSV available in the results folder.

Instructions for Setup

1. Create a Virtual Environment

python -m venv venv

2. Activate the Virtual Environment

source venv/bin/activate # Linux/macOS
.\venv\Scripts\activate  # Windows

3. Install Dependencies

pip install -r requirements.txt