huashen218 / convxai

CSCW 2023 Demo paper: Conversational AI Explanations to Support Human-AI Scientific Writing

This repository includes code for the ConvXAI paper accepted in CSCW 2023 Demo:

“ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing

Bibtex for citations:

@article{shen2023convxai,
  title={ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing},
  author={Shen, Hua and Huang, Chieh-Yang and Wu, Tongshuang and Huang, Ting-Hao'Kenneth'},
  journal={arXiv preprint arXiv:2305.09770},
  year={2023}
}

Quick Code Link

• ConvXAI Unified API Notebook
• Train your own XAI_Question_Classifier with ChatGPT

Table of Contents

• ConvXAI
  • Table of Contents
  • Prerequisites
    • Installation
    • MongoDB setup
    • Config files setup
    • Check pretrained data and models
  • How to run ConvXAI
    • Run the server
    • Run the client
    • Browse UI to interact

Prerequisites

Installation

Create a convxai virtual environment, activate the environment, and install the libraries for ConvXAI as below.

$conda create --name convxai python=3.7
$conda activate convxai
$pip install -r requirements.txt
$conda install -c conda-forge git-lfs  # to support github large file upload
$conda install -c conda-forge jsonnet  # to support allennlp package

MongoDB setup

ConvXAI system is built upon MongoDB database. Please install MongoDB on your node and ensure you have the database access to connect and manage the data. Then refer to the Config Files Setup section to set up mongodb_config.yml.

Config files setup

Set up the configs files of ConvXAI under path convxai/configs:

• mongodb_config.yml: You can either deploy both server and client in the same machine setting mongo_host: localhost, or you can deply them on two machines and set your client machine's IP address as mongo_host, e.g., mongo_host: "157.230.188.155"".

mongo_host: localhost
mongo_db_name: convxai

• configs.yml

Set up the path for both scientific writing models and the pre-trained checkpoints of conversational XAI models.

Scientific writing models: ConvXAI involves a SciBERT-based writing structure model (i.e., diversity model) and a GPT-based writing style model (i.e., quality model).

The diversity model and quality model are both accessible from the Huggingface Hub and will be downloaded with below script.

scientific_writing:
    diversity_model_dir: "huashen218/convxai-diversity-model"
    quality_model_dir: "huashen218/convxai-quality-model"

Conversational XAI models: Please specify the path_of_convxai/ in the checkpoints_root_dir shown below. For instance, a user clone the convxai repo under /home/huashen/workspace/projects/ path, then the path_of_convxai is /home/hqs5468/hua/workspace/projects/convxai.

conversational_xai:
    intent_model: "path_of_convxai/checkpoints/xai_models/xai_user_intent_model/intent-deberta-v3-xsmall"
    checkpoints_root_dir: "path_of_convxai/checkpoints/xai_models/"
    xai_example_dir:
        xai_emample_embeddings_dir:
            ACL: "xai_example_embeddings/diversity_model_ACL_embeddings.h5"
            CHI: "xai_example_embeddings/diversity_model_CHI_embeddings.h5"
            ICLR: "xai_example_embeddings/diversity_model_ICLR_embeddings.h5"
        xai_emample_texts_dir:
            ACL: "xai_example_embeddings/diversity_model_ACL_texts.h5"
            CHI: "xai_example_embeddings/diversity_model_CHI_texts.h5"
            ICLR: "xai_example_embeddings/diversity_model_ICLR_texts.h5"
    xai_writing_aspect_prediction_dir: "/xai_writing_aspect_prediction"
    xai_counterfactual_dir: "xai_writing_aspect_prediction/"

• service_config.yml: You can keep this file unchanged unless you want to change the relative paths or the class names inside of service_config.yml.

Check pretrained data and models

You can skip this step if you are going to use the default datasets and models of ConvXAI system, because ConvXAI repository is self-contained. It includes:

• CIA dataset: collects paper abstracts from 2018-2022 in CHI, ICLR and ACL conferences. CIA dataset is for finetuning GPT-based model to generate scientific style quality scores. Data path is: data/CIA.
• XAI models: contains pretrained checkpoints supporting conversational XAI modules to generate AI comments and explanations on-the-fly. Particularly, the checkpoints/ include:
  • xai_writing_aspect_prediction/: enables xai_models to generate AI comments related to the submitted paper's apsect label distribution.
  • xai_example_embeddings/: saves embeddings from CIA datasets to enable xai_models to generate example-based explanations. The method is NN_DOT method described in this paper.
  • xai_counterfactual_explainer_models/: contains MiCE counterfactual model pre-trained on our writing structure model.

You can also train your own writing and XAI models from scratch. Please refer to the ConvXAI Tutorial for details.

How to Run ConvXAI

You can deploy the ConvXAI server (i.e., deep learning server for writing and XAI models) and client (i.e., UI web service) either on the same node OR on two different nodes.

Then please run server and client on two different terminals as described below.

Run the server:

One terminal runs the server with: $bash path_of_convxai/convxai/runners/main_server.sh.

Please specify the path_of_convxai/ inside the main_server.sh shown below. You can also change --port if needed.

#!/usr/bin/env bash
set -x;
set -e;
export PYTHONPATH=path_of_convxai/
RUN_SERVICE_DIR="path_of_convxai/convxai";
CUDA_VISIBLE_DEVICES=0 python $RUN_SERVICE_DIR/services/run_server/run.py \
                        --config-path $RUN_SERVICE_DIR/configs/service_config.yml \
                        --port 10020;

Run the client:

The other terminal runs the client with: $bash path_of_convxai/convxai/runners/main_client.sh. Please specify the path_of_convxai/ similarly.

#!/usr/bin/env bash
set -x;
set -e;
export PYTHONPATH=path_of_convxai/
RUN_SERVICE_DIR="/path_of_convxai/convxai/services/web_service";
python $RUN_SERVICE_DIR/web_server.py

Browse UI to interact:

Then check the client terminal output, such as - * Running on http://157.230.188.155:8080/ (Press CTRL+C to quit), to open the browser link to interact with ConvXAI user interface.

Have fun chatting with ConvXAI robot for improving your paper writing!