The paper concerns the experimental testing and simulations of the impact and fragmentation of metal matrix composite—AlSi12/(SiC or Al2O3). The microstructure of the composite is complex and consists of a metallic phase (85%), ceramic skeleton, porosity, and a system of imperfect interfaces. The presented data-driven micromechanical numerical model is based on micro-CT scanning of composite material to get information about the internal structure and assessment of local mechanical properties done by microindetation. The description of the dynamic response of interpenetrated composites impacted by spherical balls is investigated in a few scenarios. The exemplary scenario is realised by imposing the initial conditions on the sample. The second one corresponds to split Hopkinson pressure bar experiments. The last one is the hitting of an elastic impactor (ball) against the plate sample. The influence of the impact velocities and material parameters of the phases on the failure modes is observed. The proposed finite element model of the AlSi12/SiC composite behavior describing gradual degradation under impact loading was tested for different impact scenarios. In all cases, the damage growth in the composite is very realistic. These results conclude that the proposed finite element model is very effective.