Skip to main navigation Skip to search Skip to main content

AD-DARTS: Adaptive Dropout for Differentiable Architecture Search

  • Ziwei Zheng
  • , Le Yang
  • , Liejun Wang
  • , Fan Li
  • Xi'an Jiaotong University
  • Tsinghua University
  • Xinjiang University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

Although Differentiable Architecture Search (DARTS) has achieved promising performance in many machine learning tasks, it still suffers from a problem during searching: due to those different operations in candidate set may need different levels of optimization, directly handling them with the same training scheme will make DARTS in favor of networks with fast convergence, resulting in a performance drop correspondingly. This problem will become more serious at the later searching stages. In this paper, we propose an adaptive dropout method for DARTS (AD-DARTS), which zeros the output of each operation with a probability according to structure parameters which can be considered as the variable representing the difficulty-level training such a candidate operation, thus serving to balance the training procedures for different operations. The operations with more parameters can be trained more adequately to strengthen the characterization ability of the network. Our analysis further shows that the proposed AD-DARTS are also with high search stability. The proposed method effectively solves the aforementioned problem and can achieve better performance compared with other baselines based on DRATS on CIFAR-10, CIFAR-100, and ImageNet.

Original languageEnglish
Title of host publicationArtificial Intelligence - 1st CAAI International Conference, CICAI 2021, Proceedings
EditorsLu Fang, Yiran Chen, Guangtao Zhai, Jane Wang, Ruiping Wang, Weisheng Dong
PublisherSpringer Science and Business Media Deutschland GmbH
Pages115-126
Number of pages12
ISBN (Print)9783030930486
DOIs
StatePublished - 2021
Event1st CAAI International Conference on Artificial Intelligence, CICAI 2021 - Hangzhou, China
Duration: 5 Jun 2021 → 6 Jun 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13070 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference1st CAAI International Conference on Artificial Intelligence, CICAI 2021
Country/TerritoryChina
CityHangzhou
Period5/06/21 → 6/06/21

Keywords

  • Adaptive dropout
  • Differentiable neural architecture search
  • Search stability

Fingerprint

Dive into the research topics of 'AD-DARTS: Adaptive Dropout for Differentiable Architecture Search'. Together they form a unique fingerprint.

Cite this