Algorithmic decision-making (ADM) through automation has benefits but must be implemented responsibly. Several mathematical definitions of fair outcomes exist, but it remains unclear how these align with human perceptions of fairness. We conducted a survey experiment (N = 258) examining common machine-learning definitions of fairness (demographic parity, equal opportunity, and equalized odds) in the context of algorithmic job interview invitations. We find that humans perceive the simple fairness definition of demographic parity as less fair than a more complex one that considers whether the invitees were eligible.