The primary outcome for evaluating people is whether the evaluation is fair. These evaluations might include typical organizational assessments like how well employees are performing or who to hire, promote, or fire. In a non-organizational context, machines may evaluate people for loans, admission, or to determine their needs. A machine penalty in these evaluations leads us to see machines as inherently less fair in evaluating people, regardless of the actual fairness of the evaluation. If the evaluation is personal or involves any loss of control, this penalty is stronger. AI as evaluators are only preferred in non-evaluative monitoring or distribution of tasks, or in cases where we expect humans to be unfair. Interestingly, when biased evaluations occur and are known, we tend to find them more unfair when they are perpetrated by a human instead of an AI. This means that a diminishing fairness from AI can lead to unchecked bias where we may reject fairer AI evaluations in favor of less fair human evaluations but also fail to hold machines as responsible when they are biased.