The high accuracy and reliability of the navigation system is the “multiplier” of the unmanned-aerial-vehicle performance. Conventional integrated navigation systems are mostly based on Kalman filtering and adopt customized system solutions, which do not have the ability of “plug-and-play” processing of asynchronous and heterogeneous navigation information. This can have serious consequences. Factor graphs are capable of meeting the needs of all-source autonomous navigation in complex flight environments due to their “plug-and-play” characteristics, and have a wide range of application prospects. This paper firstly describes the necessity, importance and significance of factor map fusion theory in ensuring the stable, reliable and high-precision operation of aircraft navigation system, discusses the existing architectural solutions of factor map fusion, and analyzes the key technologies such as inertial pre-integration, sensor model construction and probability density function calculation in factor graph information fusion, and finally points out the challenges of factor graph for integrated navigation.