rohanasthana / dinas

Code for the paper- Multi-conditioned Graph Diffusion for Neural Architecture Search (https://openreview.net/forum?id=5VotySkajV)

Multi-conditioned Graph Diffusion for Neural Architecture Search

Rohan Asthana, Joschua Conrad, Youssef Dawoud, Maurits Ortmanns, Vasileios Belagiannis

This repository contains the code for the paper titled "Multi-conditioned Graph Diffusion for Neural Architecture Search" [link].

Abstract

Neural architecture search automates the design of neural network architectures usually by exploring a large and thus complex architecture search space. To advance the architecture search, we present a graph diffusion-based NAS approach that uses discrete conditional graph diffusion processes to generate high-performing neural network architectures. We then propose a multi-conditioned classifier-free guidance approach applied to graph diffusion networks to jointly impose constraints such as high accuracy and low hardware latency. Unlike the related work, our method is completely differentiable and requires only a single model training. In our evaluations, we show promising results on six standard benchmarks, yielding novel and unique architectures at a fast speed, i.e. less than 0.2 seconds per architecture. Furthermore, we demonstrate the generalisability and efficiency of our method through experiments on ImageNet dataset.

Getting Started

To get started with the DiNAS project, follow these steps:

1. Clone the repository: git clone https://github.com/rohanasthana/DiNAS.git
2. Load the conda environment 'environment.yml' using the command conda env create -f environment.yml
3. Run the training process: python main_reg_free.py --dataset nasbench101

• nasbench101: for the NAS-Bench-101 benchmark
• nasbench201: for the NAS-Bench-201 benchmark
• nasbench301: for the NAS-Bench-301 benchmark
• nasbenchNLP: for the NAS-Bench-NLP benchmark
• nasbenchHW: for the NAS-Bench-HW benchmark

Pre-trained models

The pre-trained models for the NAS-Bench-101, NAS-Bench-201, NAS-Bench-301 and NAS-Bench-NLP benchmarks can be found in Google Drive: https://drive.google.com/drive/folders/1a9CpJDWAe5MMe1hU-JvKf9njhWyUT3yw?usp=sharing

Cite this paper

@article{
asthana2024multiconditioned,
title={Multi-conditioned Graph Diffusion for Neural Architecture Search},
author={Rohan Asthana and Joschua Conrad and Youssef Dawoud and Maurits Ortmanns and Vasileios Belagiannis},
journal={Transactions on Machine Learning Research},
issn={2835-8856},
year={2024},
url={https://openreview.net/forum?id=5VotySkajV},
note={}
}

License

This project is licensed under the MIT License.

dinas's Issues

ImportError: cannot import name 'get_cell_based_tiny_net' from 'models' (unknown location)

Hi, I tried to deploy the DiNAS but I got this error message:

Traceback (most recent call last):                                                                                    
 File "/home/DiNAS/main_reg_free.py", line 25, in <module>
    from src.datasets.nasbench301dataset_reg_free import NASBench301Dataset,NASBench301DatasetInfos, NASBench301DataModule
  File "/home/DiNAS/src/datasets/nasbench301dataset_reg_free.py", line 15, in <module>
    from src.NASBench301_self import Dataset                                                                          
 File "/home/DiNAS/src/NASBench301_self.py", line 114, in <module>
    from src.procedures_darts import TENAS                                                                             
File "/home/src/procedures_darts.py", line 25, in <module>
    from models import get_cell_based_tiny_net, get_search_spaces
ImportError: cannot import name 'get_cell_based_tiny_net' from 'models' (unknown location)

It found that the src/models/__init__.py file in github repository is an empty file. Maybe you forgot to upload the code there?

btw, the Google Drive link is in the readme.md is invalid as well. Could you please update the link?

Thanks for your help in advance: )