<p>How do we understand actions performed by others? Recent studies suggest that actions can be represented as points in a multidimensional space, where the perceived similarity of two actions is thought to be related to their proximity within this space. Here we present a data-driven approach to reveal key dimensions underlying this space using a carefully selected stimulus database of 768 one-second video clips spanning 256 action categories. We gathered similarity ratings for these videos from 6,036 participants and used a computational modeling procedure to identify key dimensions underlying these ratings. This approach revealed 28 meaningful dimensions (e.g. interaction, sport and craft) which capture information concerning human actions as well as a broad range of related domains (e.g. living and non-living things). Explicit ratings of actions along these dimensions gathered in a separate group of participants revealed a high correlation between ratings and weights along these dimensions, demonstrating that these dimensions are interpretable and can be used by participants. The multidimensional action space established in the current study enables the quantification of the similarity between different actions, which will be useful for the generation of hypotheses and future experimental manipulations. Together, our results provide a window into the nature of the representations underlying the ability to interpret other people’s actions and pave the way for future lines of research.</p>