In the context of knowledge graphs expressed in Description Logics, we address the problem of learning simple classifiers as probabilistic graphical models from incomplete data. Specifically, we start with a binary encoding of individuals and target Naive Bayes classifiers based on multivariate Bernoullis. Then, we extend these classifiers to two-tier networks that connect classification models to a lower layer consisting of a mixture of Bernoullis. We demonstrate how (probabilistic) axioms or rules can be extracted from these models, thereby improving interpretability. In addition, these models can be initialized leveraging expert knowledge. We present and discuss the results of an empirical evaluation testing the effectiveness of the models on random classification problems across several ontologies.