Normalization techniques have been extensively utilized in deep learning due to their ability to enable higher learning rates and reduce the sensitivity to initialization. However, the effectiveness of commonly used normalization methods is often constrained to specific domains. In contrast to the standard Batch Normalization (BN) and Layer Normalization (LN), where BN computes the mean and variance across the (N, H, W) dimensions and LN computes them across the (C, H, W) dimensions (where N, C, H, and W represent the batch, channel, spatial height, and width dimensions, respectively), this paper introduces a novel normalization technique called Batch Channel Normalization (BCN). BCN is designed to leverage both channel and batch dependencies, thereby combining the advantages of BN and LN in an adaptive manner, depending on the dataset or task at hand. BCN normalizes inputs independently along the (N, H, W) and (C, H, W) axes, subsequently combining the normalized outputs based on adaptive parameters. As a fundamental building block, BCN can be seamlessly integrated into existing models for a wide range of applications in computer vision. Empirical results demonstrate that the proposed technique can be effectively applied to various versions of Convolutional Neural Networks (CNNs) or Vision Transformer architectures. The code is publicly available at https://github.com/AfifaKhaled/Batch-Channel-Normalization .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

BCN: Batch Channel Normalization for Image Classification

  • Afifa Khaled

摘要

Normalization techniques have been extensively utilized in deep learning due to their ability to enable higher learning rates and reduce the sensitivity to initialization. However, the effectiveness of commonly used normalization methods is often constrained to specific domains. In contrast to the standard Batch Normalization (BN) and Layer Normalization (LN), where BN computes the mean and variance across the (N, H, W) dimensions and LN computes them across the (C, H, W) dimensions (where N, C, H, and W represent the batch, channel, spatial height, and width dimensions, respectively), this paper introduces a novel normalization technique called Batch Channel Normalization (BCN). BCN is designed to leverage both channel and batch dependencies, thereby combining the advantages of BN and LN in an adaptive manner, depending on the dataset or task at hand. BCN normalizes inputs independently along the (N, H, W) and (C, H, W) axes, subsequently combining the normalized outputs based on adaptive parameters. As a fundamental building block, BCN can be seamlessly integrated into existing models for a wide range of applications in computer vision. Empirical results demonstrate that the proposed technique can be effectively applied to various versions of Convolutional Neural Networks (CNNs) or Vision Transformer architectures. The code is publicly available at https://github.com/AfifaKhaled/Batch-Channel-Normalization .