<p>The brushless dual-port (BDP) machine is a good candidate for tidal power generation systems because of its compact structure, minimal mechanical components, and rotor decoupling. However, existing BDP machines have a low magnetic flux density and poor torque performance. To address these problems, this paper proposes a compactly structured, novel modulation-enhanced dual magnetic flux brushless dual-port (MDMF-BDP) machine. When compared with conventional BPD machines, the MDMF-BDP machine adds a modulator outside the stator slots, enabling the machine to have two magnetic flux paths, producing a dual magnetic flux modulation effect. The outer rotor of the MDMF-BDP machine adopts a composite rotor structure. This design enhances the magnetic flux modulation effect and increases the effective working harmonics of the air gap magnetic field. This paper introduces the topology and the operating principle of the novel MDMF-BDP machine, establishes a finite element model of the MDMF-BDP machine, and optimizes it using a layered sequential optimization method based on parameter sensitivity. Finally, the finite element method is used to compare the performance of the proposed MDMF-BDP machine with conventional BDP machines. Results show that the inner rotor torque of the proposed MDMF-BDP machine is increased by 33.3%, the outer rotor torque is increased by 303.4%, and the torque ripple is significantly reduced, leading to smoother operation. This study provides a new approach for the development of the tidal power generation field.</p>