Neural Architecture Search (NAS) is hindered by the trade-off between exploring the search space and achieving efficiency, particularly for complex tasks. Recent LLM-based NAS approaches are promising but typically rely on fixed LLM capacity across the entire search and on ambiguous architecture representations, resulting in wasted compute and brittle code generation. We present PhaseNAS, an LLM-driven NAS framework that (i) dynamically allocates LLM capacity across phases via score-triggered transitions, and (ii) employs a structured architecture template language that reliably maps prompts to executable code. We further introduce a zero-shot detection score that extends Zen-NAS to multi-scale feature perturbations, enabling fast screening of YOLO-style detectors without full training. On NAS-Bench-Macro, PhaseNAS discovers higher-ranked architectures. On CIFAR-10/100, it reduces search time by up to 86% while matching or improving accuracy. On COCO, it automatically yields YOLOv8 variants with higher mAP and lower FLOPs/parameters. These results show that PhaseNAS provides an efficient, adaptive, and general NAS solution across classification and detection.