A idea about future: Transfer learning based on Deep forest ? about deep-forest

[xuyxu]

Hi @learningelectric, thanks for your ideas. Is there any paper on using forest for transfer learning?

[learningelectric]

Dear @xuyxu .

Thank you very much for your prompt reply!

Firstly, since the deep-forest is a very powerful decision tree model, if the decision tree or random forest can be migrated[1-3], then the deep-forest should also be able to be migrated in theory. Of course, this is just an assumption of mine, and may be wrong and difficult. But in my current study, deep forests are competitive with neural networks or traditional machine learning algorithms (It will be submitted to the IEEE journal this year). However, in practical engineering, if the deep forest model also has the ability of transfer learning, it will be very competitive in practical engineering in the future.

This is just a immature suggest. And please don't mind if I offend.

References:
[1] L. Minvielle, M. Atiq, S. Peignier and M. Mougeot, "Transfer Learning on Decision Tree with Class Imbalance," 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), 2019, pp. 1003-1010, doi: 10.1109/ICTAI.2019.00141.
[2] S. Jiang, H. Mao, Z. Ding and Y. Fu, "Deep Decision Tree Transfer Boosting," in IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 2, pp. 383-395, Feb. 2020, doi: 10.1109/TNNLS.2019.2901273.
[3] N. Segev, M. Harel, S. Mannor, K. Crammer and R. El-Yaniv, "Learn on Source, Refine on Target: A Model Transfer Learning Framework with Random Forests," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 9, pp. 1811-1824, 1 Sept. 2017, doi: 10.1109/TPAMI.2016.2618118.

[IncubatorShokuhou]

I believe that what we really need may be an incremental learning method (i.e. partial_fit). With this method, we can not only implement tranffer learning, but distributed trainning (with dask and rabit) will also be possible.

[xuyxu]

The incremental version of deep forest is not a trivial task, and may be out of the scope of the current design. You are feel free to work on it, or even write a research paper accordingly.

[learningelectric]

OK, thank you very much!