Fast rough mode decision algorithm and hardware architecture design for AV1 encoder

To enhance compression efficiency, the AV1 video coding standard has introduced several new intra-prediction modes, such as smooth and finer directional prediction modes. However, this addition increases computational complexity and hinders parallelized hardware implementation. In this paper, a hardware-friendly rough mode decision (RMD) algorithm and its fully pipelined hardware architecture design are proposed to address these challenges. For algorithm optimization, firstly, a novel directional mode pruning algorithm is proposed. Then, the sum of absolute transform differences (SATD) cost accumulated approximation method is adopted during the tree search. Finally, in the reconstruction stage, a reconstruction approximation model based on the DC transform is proposed to solve the low-parallelism problem. For hardware architecture design, the proposed fully pipelined hardware architecture is implemented with 28 pipeline stages. This design can process multiple prediction modes in parallel. Experimental results show that the proposed fast algorithm achieves 46.8% time savings by 1.96% Bjøntegaard delta rate (BD-Rate) increase on average under all-intra (AI) configuration. When synthesized under the 28nm UMC technology, the proposed hardware can operate at a frequency of 316.2 MHz with 1113.14 K gate count.