Artificial intelligence is the discipline that pursues the understanding, artificial replication and possible enhancement of human intelligence. Among others, AI-based recommender systems, persuasive systems as well as decision support systems are used for making decisions that have direct impact on people’s lives. In AI-supported decision making, the initiative may lie on the user’s side (such as recommender systems), or be largely automated (such as navigation systems or automotive driving). Many organizations also employ AI systems to make decisions that concern their users, customers, or citizens. But how do these systems learn about the people involved, how are these people represented, and—most importantly—how complete, realistic, reliable and fair is this representation? In this chapter, we discuss various considerations for user modeling in human-centered AI.