This chapter introduces the modeling and recognition of activities. Activities are generally larger in scale, longer in duration than actions. Activities often represent complex actions performed by multiple persons in a sequence (and possibly interacting), and need the cognitive computation of visual information. In order to better identify activities, it is usually necessary to establish corresponding models and conduct comprehensive analysis. According to the model established for the activity, different methods and means can be used to identify the activity. They consider the mutual relationship between actors (action initiators) and actions, as well as the application of various deep learning techniques. This chapter will first introduce some basic methods and classifications of action modeling and activity modeling. Then, several practical techniques, including single-label and multi-label actor-action recognition as well as actor-action semantic segmentation, for joint modeling of actors and actions will be presented. This chapter will provide a number of typical human skeleton representation, and an overview of various methods for constructing neural networks using skeleton data for activity modeling and recognition.