The advent of integral field data has revolutionised the study of galaxy evolution. A key component of this is dynamical modelling methods which have allowed for crucial insights to be made from kinematic data. Despite this importance, most dynamical models make a number of key assumptions which do not hold for real galaxies. These include assumptions about the geometry (axisymmetry or triaxiality), the shape of the velocity ellipsoid, and the shape of the underlying stellar distribution. At the same time, machine learning methods are becoming increasingly powerful, with many applications appearing in astronomy. As a first step towards building new dynamical modelling methods with machine learning, it is important to understand the types of machine learning architectures that are best fit for dynamical modelling. To investigate this, we construct a training set of dynamical models of early-type galaxies using Jeans Anisotropic Modelling (JAM). We then train a neural network on this data using the parameters of JAM and mock photometry as the input. We are able to accurately model JAM galaxies with relatively simple machine learning architectures, leading to a significant speed increase over traditional JAM modelling.