Scholarlink

In the field of artificial intelligence and deep learning, optimizers are the core tools for model training, directly influencing the convergence speed and performance of models. This paper proposes a novel optimization algorithm - comprehensive gradient-free optimization with kernel density estimation-based cyclical learning rate (CGFO-KCD), aimed at solving highly non-convex, discontinuous, and non-differentiable objective functions. CGFO-KCD combines gradient-free optimization with cyclical learning rate (CLR) scheduling, utilizing Kernel Density Estimation (KDE) to capture parameter distributions and adaptive CLR to dynamically adjust the learning rate, achieving efficient exploration and exploitation of the solution space. Experimental results on customized complex objective functions, which are non-convex, discontinuous, and non-differentiable, demonstrate that CGFO-KCD outperforms traditional optimizers in terms of convergence speed and solution quality.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Comprehensive Gradient-Free Optimization with Adaptive Kernel-Based Cyclical Learning Rate

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Hu Yu,
  • Rupeng Zhu,
  • Weiping Yan

摘要

In the field of artificial intelligence and deep learning, optimizers are the core tools for model training, directly influencing the convergence speed and performance of models. This paper proposes a novel optimization algorithm - comprehensive gradient-free optimization with kernel density estimation-based cyclical learning rate (CGFO-KCD), aimed at solving highly non-convex, discontinuous, and non-differentiable objective functions. CGFO-KCD combines gradient-free optimization with cyclical learning rate (CLR) scheduling, utilizing Kernel Density Estimation (KDE) to capture parameter distributions and adaptive CLR to dynamically adjust the learning rate, achieving efficient exploration and exploitation of the solution space. Experimental results on customized complex objective functions, which are non-convex, discontinuous, and non-differentiable, demonstrate that CGFO-KCD outperforms traditional optimizers in terms of convergence speed and solution quality.