minisoco / composition_styles

Infusing Definiteness into Randomness: Rethinking Composition Styles for Deep Image Matting

This repository includes the official implementation of triplet-style composition & quadruplet-style composition, presented in our paper:
Infusing Definiteness into Randomness: Rethinking Composition Styles for Deep Image Matting
Proceedings of AAAI Conference on Artificial Intelligence (AAAI 2023)

🏃Author

👤 Zixuan Ye1, Yutong Dai2, Chaoyi Hong1, Zhiguo Cao1, Hao Lu1
🏠1 Huazhong University of Science and Technology, China
🏠2 Australian Institute for Machine Learning, The University of Adelaide, Australia

📑More Information

[paper][video]

🔆Highlights

📘The first to delve into the data generation flow and demonstrate that careful treatment can improve the performance significantly.
📘Explain the problem in NCF and propose a Reasonable Combination of Foregrounds (RCF)
📘Introduce triplet-style composition which builds the relation of source foregrounds and combined foreground.
📘Reveal the property of twin foregrounds and introduce quadruplet-style composition.

✔️Instructions

Our composition styles can be used in any deep matting models. To use our composition styles in your project, you only need follow the steps below:
1️⃣ We need to renew the sample set each epoch, therefore follow the generate_index.ipynb to obtain the foreground list, background list for each epoch. Order list can be used to control whether relevant samples will appear in the same batch or just the same sample set.
2️⃣Use the dataloader.py to generate the dataset with the selected composition styles. The changes are made from L478 to L612.
3️⃣Modify your own training code to load the sample set with generated indexes each epoch.(The code below is an example)

for epoch in range(start_epoch, cfg.TRAIN.num_epochs):
    if cfg.DATASET.composition_style is not None:
        back_list = np.load(file = 'backlist.npy')
        fore_list = np.load(file = 'forelist.npy')
        order_list = np.load(file = 'orderlist.npy') 
        backlist = back_list[epoch]
        forelist = fore_list[epoch]
        trainset = dataset(cfg, phase='train', test_scale='origin', crop_size=cfg.TRAIN.crop_size, back_list = backlist, fore_list = forelist, order_list=order_list)

📚Reported Results

Effectiveness on four baselines

---

Reference

IndexNet Matting
GCA Matting
A2U Matting
Matteformer