fabiocarrara / visual-sentiment-analysis

PyTorch port of models for Visual Sentiment Analysis pre-trained on the T4SA dataset.

Cross-Media Learning for Image Sentiment Analysis in the Wild

This repo contains the PyTorch-converted models for visual sentiment analysis trained on the T4SA (Twitter for Sentiment Analysis) dataset presented in [1].

[1] Vadicamo, L., Carrara, F., Cimino, A., Cresci, S., Dell'Orletta, F., Falchi, F. and Tesconi, M., 2017.
    Cross-media learning for image sentiment analysis in the wild.
    In Proceedings of the IEEE International Conference on Computer Vision Workshops (pp. 308-317).

Usage

0. Install Requirements: PyTorch

1. Download the pretrained models:

   ./download_models.sh

2. Use the predict.py script to make predictions on images. Example:

   python predict.py images_list.txt --model vgg19_finetuned_all --batch-size 64 > predictions.csv

   The output file contains three columns representing the probability of each image belonging respectively to the negative, neutral, and positive classes in this order.

Converting the original Caffe models

We adopted MMdnn to convert caffe models to PyTorch. We recommend using the pre-built Docker image:

docker pull mmdnn/mmdnn:cpu.small

First, download the original models available at http://t4sa.it and extract them following this folder structure:

original-models/
├── hybrid_finetuned_all/
│   ├── deploy.prototxt
│   ├── mean.binaryproto
│   ├── snapshot_iter_34560.caffemodel
│   └── ...
├── hybrid_finetuned_fc6+/
│   ├── <same as above>
│   └── ...
├── vgg19_finetuned_all/
│   ├── <same as above>
│   └── ...
└── vgg19_finetuned_fc6+/
    ├── <same as above>
    └── ...

Then, run convert_models.sh:

docker run --rm -it -v $(pwd):/workspace -w /workspace mmdnn/mmdnn:cpu.small bash ./convert_models.sh

visual-sentiment-analysis's Issues

Inquiries about the model operation

Hi the author of the great model,

I am a graduate student new to computational political communication and IT technology. I am very interested in your models, and am willing to acknowledge them in my research.
Still, I am a newcomer to the research practice and understandably encountered some problems.
I am doing it on Windows 10 Enterprise.

With the help of Google, I have successfully installed Git Bash and changed the directory to where the download_models.sh is in. I also think that I have successfully downloaded the models via the link (though I failed in using ./download_models.sh).

• However, I encountered some difficulties when trying to implement the second sample code.
  
  I have set the directory, but in Git Bash, it returns ModuleNotFoundError: No module named 'torch'
  

• Meanwhile, I also tried to operate the code in PyCharm Professional 2020.01. I only added Line 11 and Line 12 to ensure the directory is where the image_list.txt in.
  
  
  It also returns ERROR: predict.py: error: the following arguments are required: image_list.
  While I am trying to Google for help, I suspect that this may because 1) There should be something (should it be links or paths leading to someplace locally or on web?); 2) I placed the downloaded models in the wrong directory.

I would really appreciate it if you could kindly help me to solve this problem.
Starting to Learn these techs really needs a strong heart and warm help from know-hows.
Many thanks for your invaluable guidance!