Glossary

Neural Architecture Search (NAS)

Discover how Neural Architecture Search (NAS) automates neural network design for optimized performance in object detection, AI, and more.

Train YOLO models simply
with Ultralytics HUB

Learn more

Neural Architecture Search (NAS) is an automated technique within the field of machine learning (ML) focused on designing the optimal structure, or architecture, of neural networks (NNs). Instead of relying on human experts to manually design network layouts through trial and error, NAS employs algorithms to explore a vast space of possible architectures and identify the most effective ones for a given task and dataset. This automation accelerates the development process and can uncover novel, high-performing architectures that might not be intuitively obvious to human designers, optimizing for metrics like accuracy, speed (inference latency), or computational efficiency, which is crucial for deploying models on edge AI devices.

How Neural Architecture Search Works

The fundamental process of NAS involves three main components: a search space, a search strategy, and a performance estimation strategy. The search space defines the set of possible network architectures that can be designed, essentially outlining the building blocks (like different types of convolution or activation functions) and how they can be connected. The search strategy guides the exploration of this space, using methods ranging from random search and reinforcement learning to evolutionary algorithms. Finally, the performance estimation strategy evaluates how well a candidate architecture performs, often involving training the network partially or fully on a dataset and measuring its performance, although techniques like weight sharing or performance predictors are used to speed this up, as detailed in research from Google AI. Efficiently managing these experiments can be facilitated by platforms like Weights & Biases or Ultralytics HUB.

Key Benefits of NAS

Automating architecture design with NAS provides significant advantages:

  • Optimized Performance: Finds architectures tailored for specific tasks like object detection or image classification, often surpassing manually designed ones in accuracy or efficiency.
  • Reduced Development Time: Automates the time-consuming and often intuition-driven process of architecture design.
  • Discovery of Novel Architectures: Uncovers unconventional yet effective network structures that human designers might overlook.
  • Efficiency for Specific Hardware: Can optimize architectures for deployment constraints, such as low inference latency on mobile devices (Edge AI) or specific accelerators like TPUs.

Applications in AI and Machine Learning

NAS has proven valuable across various deep learning (DL) domains:

1. Optimized Object Detection Models

A prominent example is YOLO-NAS, developed by Deci AI using NAS technology. This model specifically targeted limitations in previous Ultralytics YOLO versions by incorporating quantization-friendly blocks found through NAS. This resulted in models offering a superior balance between accuracy and latency, making them highly effective for real-time applications such as AI in automotive solutions and smart traffic management, even after model quantization to formats like INT8 for efficient deployment. Further information on quantization techniques can be found in resources like the NVIDIA TensorRT documentation or the Ultralytics guide on model deployment options. Ultralytics provides support for various object detection models, including YOLO-NAS.

2. Medical Image Analysis

In healthcare, NAS is used to design custom Convolutional Neural Networks (CNNs) for analyzing medical images. For instance, NAS can optimize architectures for tasks like detecting tumors in MRI scans (similar to the Brain Tumor dataset) or segmenting organs in CT images, potentially leading to faster and more accurate diagnostic tools to aid clinicians. The application of AI in medical image analysis is a rapidly growing field, as highlighted by institutions like the National Institutes of Health (NIH). Managing such specialized models and datasets can be streamlined using platforms like Ultralytics HUB. You can even use YOLO11 for tumor detection.

Read all