yxtay / recommender-tensorflow

Recommendation Models in TensorFlow

Recommendation Models in TensorFlow

This repository attempts to implement models for recommendation engines in TensorFlow using the Estimator API with feature columns.

The trainer module in this repository also allows for distributed model training and evaluation on Google Cloud Platform. Please refer to distributed.

Models

• Linear classifer: linear.py
• DNN classifier: deep.py
• Linear & DNN classifier: linear_deep.py
• DeepFM: deep_fm.py

DeepFM

Model Parameters

• categorical_columns: categorical feature columns input
• numeric_columns: numeric feature columns input
• use_linear: flag to include linear structure of model (default: True)
• use_mf: flag to include factorisation machine structure of model (default: True)
• use_dnn: flag to include deep structure of model (default: True)
• embedding_size: embedding size of latent factors (default: 4)
• hidden_units: layer sizes of hidden units of deep structure (default: [16, 16])
• activation_fn: activation function of deep structure (default: tf.nn.relu)
• dropout: dropout rate of deep structure (default: 0)
• optimizer: learning optimiser (default: "Adam")
• learning_rate: learning rate (default: 0.001)

Setup

# clone repo
git clone [email protected]:yxtay/recommender-tensorflow.git && cd recommender-tensorflow

# create conda environment
conda env create -f=environment.yml

# activate environment
source activate dl

Download & Process Data

The MovieLens 100K Dataset is used for demonstration purpose. The following script downloads the data, processes and enriches it with a few basic features and serialises it to csv.

# downloads and processes movielens 100k dataset
python -m src.data.ml_100k local

Usage

python -m src.data.ml_100k local -h

usage: ml_100k.py local [-h] [--url URL] [--dest DEST] [--log-path LOG_PATH]

optional arguments:
  -h, --help           show this help message and exit
  --url URL            url of MovieLens 100k data (default:
                       http://files.grouplens.org/datasets/movielens/ml-
                       100k.zip)
  --dest DEST          destination directory for downloaded and extracted
                       files (default: data)
  --log-path LOG_PATH  path of log file (default: main.log)

Train & Evaluate DeepFM

Usage

python -m trainers.deep_fm -h

usage: deep_fm.py [-h] [--train-csv TRAIN_CSV] [--test-csv TEST_CSV]
                  [--job-dir JOB_DIR] [--restore] [--exclude-linear]
                  [--exclude-mf] [--exclude-dnn]
                  [--embedding-size EMBEDDING_SIZE]
                  [--hidden-units HIDDEN_UNITS [HIDDEN_UNITS ...]]
                  [--dropout DROPOUT] [--batch-size BATCH_SIZE]
                  [--train-steps TRAIN_STEPS]

optional arguments:
  -h, --help            show this help message and exit
  --train-csv TRAIN_CSV
                        path to the training csv data (default: data/ml-
                        100k/train.csv)
  --test-csv TEST_CSV   path to the test csv data (default: data/ml-
                        100k/test.csv)
  --job-dir JOB_DIR     job directory (default: checkpoints/deep_fm)
  --restore             whether to restore from job_dir
  --exclude-linear      flag to exclude linear component (default: False)
  --exclude-mf          flag to exclude mf component (default: False)
  --exclude-dnn         flag to exclude dnn component (default: False)
  --embedding-size EMBEDDING_SIZE
                        embedding size (default: 4)
  --hidden-units HIDDEN_UNITS [HIDDEN_UNITS ...]
                        hidden layer specification (default: [16, 16])
  --dropout DROPOUT     dropout rate (default: 0.1)
  --batch-size BATCH_SIZE
                        batch size (default: 32)
  --train-steps TRAIN_STEPS
                        number of training steps (default: 20000)

Tensorboard

You may inspect model training metrics with Tensorboard.

tensorboard --logdir checkpoints/

Other Models Available

# linear model
python -m trainers.linear

# deep model
python -m trainers.deep

# wide & deep model
python -m trainers.linear_deep

Distributed

For distributed model training and evaluation, please refer to distributed.

References

• Harper F. M., & Konstan, J. A. (2015). The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems, 5(4), Article 19, 19 pages. DOI=http://dx.doi.org/10.1145/2827872.
• Cheng, H. T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., ... Shah, H. (2016). Wide & Deep Learning for Recommender Systems. arXiv:1606.07792 [cs.LG].
• Guo, H., Tang, R., Ye, Y., Li, Z., & He, X. (2017). DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. arXiv:1703.04247 [cs.IR].

recommender-tensorflow's Issues

one field has many values, like people's interest, how to resolve this?

question about fm

Does line91 in <recommender-tensorflow/src/models/deep_fm.py> means that 'mf_logit' is a predicted value of our label with the mf method? But I can not see the method of FM in this section of code? Why
0.5 * tf.reduce_sum(sum_square - square_sum, 1, keepdims=True) represents for the predicted value?