ethanhe42 / awesome-architecture-search

A curated list of awesome architecture search resources

Awesome Architecture Search

A curated list of awesome architecture search and hyper-parameter optimization resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning and awesome-deep-learning-papers.

Hyper-parameter optimization has always been a popular field in the Machine Learning community, architecture search just emerges as a rising star in recent years. These are some of the awesome resources!

Table of Contents

• Architecture Search
  • Reinforcement Learning
  • Evolutionary Algorithm
  • Others
• Hyper-parameter Search

Architecture Search

Reinforcement Learning

• Neural Architecture Search with Reinforcement Learning [pdf]
  • Barret Zoph and Quoc V. Le. ICLR'17
• Designing Neural Network Architectures Using Reinforcement Learning [pdf] [code]
  • Bowen Baker, Otkrist Gupta, Nikhil Naik, Ramesh Raskar. ICLR'17
• Efficient Architecture Search by Network Transformation [pdf] [code]
  • Han Cai, Tianyao Chen, Weinan Zhang, Yong Yu, Jun Wang. AAAI'18
• Learning Transferable Architectures for Scalable Image Recognition [pdf] [nasnet]
  • Barret Zoph, Vijay Vasudevan, Jonathan Shlens, Quoc V. Le. Arxiv 1707
• Practical Network Blocks Design with Q-Learning [pdf]
  • Zhao Zhong, Junjie Yan, Cheng-Lin Liu. AAAI'18
• Efficient Neural Architecture Search via Parameter Sharing [pdf] [code (not official)] [code (official)]
  • Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, Jeff Dean. Arxiv 1802

Evolutionary Algorithm

• Large-Scale Evolution of Image Classifiers [pdf]
  • Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Jie Tan, Quoc Le, Alex Kurakin. ICML'17
• Genetic CNN [pdf] [code]
  • Lingxi Xie and Alan Yuille. ICCV'17
• Hierarchical Representations for Efficient Architecture Search [pdf]
  • Hanxiao Liu, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, Koray Kavukcuoglu. ICLR'18
• Regularized Evolution for Image Classifier Architecture Search [pdf]
  • Esteban Real, Alok Aggarwal, Yanping Huang, Quoc V Le. Arxiv 1802

Others

• AMC: AutoML for Model Compression and Acceleration on Mobile Devices [pdf]
  • Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, Song Han ECCV'2018
• DeepArchitect: Automatically Designing and Training Deep Architectures [pdf] [code]
  • Renato Negrinho and Geoff Gordon. Arxiv 1704
• SMASH: One-Shot Model Architecture Search through HyperNetworks [pdf] [code]
  • Andrew Brock, Theodore Lim, J.M. Ritchie, Nick Weston. ICLR'18
• Simple and efficient architecture search for Convolutional Neural Networks [pdf]
  • Thomas Elsken, Jan-Hendrik Metzen, Frank Hutter. Invited to ICLR'18 Workshop
• Progressive Neural Architecture Search [pdf]
  • Chenxi Liu, Barret Zoph, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy. Arxiv 1712
• DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures [pdf (ICLR'18 Workshop)] [pdf (ECCV)]
  • Jin-Dong Dong, An-Chieh Cheng, Da-Cheng Juan, Wei Wei, Min Sun. ECCV'18
• Neural Architecture Search with Bayesian Optimisation and Optimal Transport [pdf]
  • Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, Eric Xing. Arxiv 1802
• Path-Level Network Transformation for Efficient Architecture Search [pdf] [code]
  • Han Cai, Jiacheng Yang, Weinan Zhang, Song Han, Yong Yu. ICML'18
• Effective Building Block Design for Deep Convolutional Neural Networks using Search [pdf]
  • Jayanta K Dutta, Jiayi Liu, Unmesh Kurup, Mohak Shah. Arxiv 1801
• DARTS: Differentiable Architecture Search [pdf] [code]
  • Hanxiao Liu, Karen Simonyan, Yiming Yang. Arxiv 1806

Hyper-Parameter Search

• Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning Curves [pdf] [code]
  • Tobias Domhan, Jost Tobias Springenberg, Frank Hutter. IJCAI'15
• Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization [pdf]
  • Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, Ameet Talwalkar. ICLR'17
• Learning Curve Prediction with Bayesian Neural Networks [pdf]
  • Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, Frank Hutter. ICLR'17
• Accelerating Neural Architecture Search using Performance Prediction [pdf]
  • Bowen Baker, Otkrist Gupta, Ramesh Raskar, Nikhil Naik. Invited to ICLR'18 Workshop
• Hyperparameter Optimization: A Spectral Approach [pdf] [code]
  • Elad Hazan, Adam Klivans, Yang Yuan. NIPS DLTP Workshop 2017
• Population Based Training of Neural Networks [pdf]
  • Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, Koray Kavukcuoglu. Arxiv 1711

Contributing

Please help contribute this list by contacting me or add pull request

Markdown format:

- Paper Name [[pdf]](link) [[code]](link)
  - Author 1, Author 2, Author 3. *Conference'Year*

License

To the extent possible under law, Mark Dong has waived all copyright and related or neighboring rights to this work.