<p>Sparse matrix-vector multiplication (SpMV) is one of the key kernels extensively employed in both industrial and scientific applications, with its computation and random access incurring a lot of overhead. To capitalize on higher compute rates and data movement efficiency, there have been efforts to utilize mixed precision SpMV. However, most existing techniques focus on single-grained precision selection for all matrices. In this work, we concentrate on hierarchical precision selection strategies tailored for irregular matrices, driven by the need to achieve optimal load balancing among thread groups executing on GPUs. Based on the concept of strong connection, we firstly introduce a novel adaptive row-grained precision selection strategy that surpasses existing strategy within multi-precision Jacobi methods. Secondly, our experiments have uncovered a range within which converting double-precision floating-point numbers to single-precision floating-point numbers incurs a loss smaller than the machine precision FLT_EPSILON. This range is used for element-grained precision selection. Subsequently, we propose a hierarchical precision selection compressed sparse row format(CSR) storage method and enhance the CSR-Vector kernel, achieving higher relative speedups and load balancing on a benchmark suite composed of 41 matrices compared to existing methods. Finally, we integrate the mixed precision SpMV into the generalized minimal residual method(GMRES) algorithm, achieving faster execution speeds while maintaining similar convergence accuracy as double-precision GMRES.</p>