shgo / gamtl

Group LASSO with Asymmetric Structure Estimation for Multi-Task Learning

GAMTL

Main track of the 28th International Joint Conference on Artificial Intelligence - IJCAI 2019

PDF Open Access

Title: Group LASSO with Asymmetric Structure Estimation for Multi-Task Learning

Authors: Saullo Oliveira, André Gonçalves, Fernando Von Zuben.

Abstract: Group LASSO is a widely used regularization that imposes sparsity considering groups of covariates. When used in Multi-Task Learning (MTL) formulations, it makes an underlying assumption that if one group of covariates is not relevant for one or a few tasks, it is also not relevant for all tasks, thus implicitly assuming that all tasks are related. This implication can easily lead to negative transfer if this assumption does not hold for all tasks. Since for most practical applications we hardly know a priori how the tasks are related, several approaches have been conceived in the literature to (i) properly capture the transference structure, (ii) improve interpretability of the tasks interplay, and (iii) penalize potential negative transfer. Recently, the automatic estimation of asymmetric structures inside the learning process was capable of effectively avoiding negative transfer. Our proposal is the first attempt in the literature to conceive a Group LASSO with asymmetric transference formulation, looking for the best of both worlds in a framework that admits the overlap of groups. The resulting optimization problem is solved by an alternating procedure with fast methods. We performed experiments using synthetic and real datasets to compare our proposal with state-of-the-art approaches, evidencing the promising predictive performance and distinguished interpretability of our proposal. The real case study involves the prediction of cognitive scores for Alzheimer's disease progression assessment.

For replication instructions see Reproducing.

Date: 06/2019

License: GNU General Public License v3.0

How to Cite

@inproceedings{Oliveira2019,
  title     = {Group LASSO with Asymmetric Structure Estimation for Multi-Task Learning},
  author    = {Oliveira, Saullo H. G. and Gonçalves, André  R. and Von Zuben, Fernando J.},
  booktitle = {Proceedings of the Twenty-Eighth International Joint Conference on
               Artificial Intelligence, {IJCAI-19}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},             
  pages     = {3202--3208},
  year      = {2019},
  month     = {7},
  doi       = {10.24963/ijcai.2019/444},
  url       = {https://doi.org/10.24963/ijcai.2019/444},
}

Acknowledgements

We acknowledge the grants #141881/2015-1 and #307228/2018-5 from the Brazilian National Council for Scientific and Technological Development (CNPq), grant #2013/07559-3 from São Paulo Reseach Foundation (FAPESP), and the Coordination for the Improvement of Higher Education Personnel (CAPES).

gamtl's Issues

Remove experimental formulations

Remove experimental formulations and settings for GAMTL.
Here are some examples:

• Cross cost:
  classification.py:147
  regression: 125
• GAMTL2

Add headers to files with more detailed information

Headers on every file should follow a pattern and contain information such as:
author, license, etc.

Remove unused files

Some files are not necessary for replication and should be removed from repository:

• real_datasets.py
• gamtl_debug.py
• gamtl2.py
• mtsgl.py

Translate Portuguese comments / documentation to English

Files below present comments and print messages in Portuguese-BR:

• exp_var_m_art.py
• exp_var_m.py
• test_environment.py
• datasets.py
• hyper_params.py:450
• plot_print.py
• amtl.py

All comments should be in English, and following the same pattern.