Increasingly, peoples’ emotions are datafied through emotional AI. In recent years, the development and deployment of emotional AI has evolved rapidly. One category of data processed by emotional AI is biometric data: hard biometric data, relating to identified or identifiable individuals, and soft biometric data, that cannot identify or single out individuals. Both the processing of hard biometric data and soft biometric data allow to manipulate individuals, and as such may constitute a violation of fundamental rights: in particular, the right to human dignity; the right to a private and family life; and the right to the protection of personal data. Besides those human rights, at an EU level, hard biometric data fall within the scope of the General Data Protection Regulation (GDPR). By definition, as soft biometric data do not enable the identification of an individual, they remain outside the remit of the GDPR. In this chapter we argue that the European Union (EU) regulatory regime governing the usage of emotional AI technologies that process soft biometric data insufficiently safeguards the full exercise of the aforementioned fundamental rights. We analyse the AI Act, and the various inconsistencies it exhibits, prompting questions on its expected effectiveness in regulating the processing of soft biometric data by emotional AI.