The advent of artificial intelligence (AI) has revolutionised how businesses leverage customer data to enhance service delivery, tailor marketing efforts, and optimise operations. However, the use of AI raises significant ethical and privacy concerns, particularly for vulnerable populations such as the elderly, economically disadvantaged individuals, and those with limited digital literacy. This research aims to explore the implications of AI deployment on customer data, focusing on the privacy risks and ethical considerations associated with handling data from vulnerable customers. The study will examine current practices, regulatory frameworks, and technological safeguards to ensure that AI applications do not exacerbate inequalities or compromise the privacy of these sensitive groups. By analysing case studies, and reviewing literature, this research will propose best practices and policy recommendations to balance innovation with ethical responsibility and privacy protection.