Understanding how social interactions shape collective behavior in group-living species is a challenging problem in movement ecology. These interactions determine how individuals adapt their movements in response to both conspecifics and the environment. Recent advances in animal tracking software provide high-resolution spatiotemporal data on animal groups in various settings. Here we introduce a method for modeling and reconstructing social interactions within animal groups, using extensive experimental data on animal trajectories. We describe a procedure for finding analytical expressions of interactions between individuals and their environment, capturing the complexity of individual behaviors that lead to collective patterns. This method avoids restrictive assumptions about the form or intensity of interactions. We identify two fundamental types of social interactions, attraction and alignment, which can mathematically describe other pairwise interactions such as repulsion. Additionally, we apply this method to characterize social interactions in Hemigrammus rhodostomus, a species of schooling fish with an intermittent burst-and-coast swimming mode. More broadly, this method can be used to predict and understand group dynamics in various environmental conditions.