Accurate cancer survival prediction is crucial for personalized treatment. Current methods often rely on single-magnification patch features from Whole Slide Images (WSIs), overlooking multi-level image information and facing limitations in patch labeling and feature aggregation. To overcome these challenges, we introduce HiMulti, a novel multi-instance model that integrates pathological images and pathway features for improved survival prediction. HiMulti first processes WSIs into patches and creates a multi-level pyramid through downsampling. It then utilizes an improved mamba-inspired linear attention model and a linear attention transformer for intra- and inter-level feature fusion. Simultaneously, pathway features are constructed from RNA-Seq data. A dual-branch attention mechanism selects key image regions by generating multimodal and single-modal attention weights. Finally, patient-level features are aggregated from these key images for cancer survival prediction. Compared with the existing weakly supervised methods, the average C-Index of HiMulti on the TCGA-LUAD, BRCA, and BLCA datasets increases by 1.39% compared with the sub-optimal value, and the visualization results confirm its superior performance. Implementation is available at: https://github.com/weiba/HiMulti .