In order to harness the benefits AI offers, tomorrow’s workforces and societies alike need to be competent in how to use, shape, and develop AI applications. As part of the KNIGHT research project of Stuttgart University of Applied Sciences, this contribution develops a framework for competence-based AI education at higher education facilities, based on a review of selected literature. The bulk of the competencies focuses on technical aspects of AI, however, since AI applications are oftentimes faced with potential ethical challenges, special care is taken to integrate dedicated ethical competencies. The competencies are then structured according to the German Qualifications Framework for Higher Education to yield a comprehensive AI competence matrix. As a proof of concept, the matrix is then applied to structure an AI certificate, available to students and staff of Stuttgart University of Applied Sciences.