The proposed AI Act lays down requirements for so-called high-risk AI-systems that need to be met by the high-risk AI-system and its provider. These requirements contain core elements for trustworthy AI such as transparency, technical robustness, and human oversight. Automated vehicles are considered to be high-risk AI-systems, but nevertheless these requirements do not (fully and directly) apply to these vehicles. Automated vehicles are excluded from the scope of the proposed AI Act, as the functioning, safety and security of cars driving on EU public roads are governed by existing acts specifically applicable to vehicles. This causes a gap between the findings of two ethics expert groups (EC High-Level Expert Group on AI and the Horizon 2020 Commission Expert Group to advice on specific ethical issues raised by driverless mobility) and the EU legislation for AI-systems of automated vehicles. This contribution will therefore explore two questions: what current legislation is in place that contributes to achieving the core elements or principles of trustworthy AI in automated vehicles and how can a (future) legislative framework for automated vehicles realize these core elements for trustworthy AI?