While the advancements in Artificial Intelligence and Machine Learning expand and become more prevalent, privacy and security professionals in all related fields are likely to face a dilemma in deciding where and when to control gathering and retaining personal data and being transparent about its use. With the advancements and dependability on AI that we see today in the twenty-first century, it is frightening to imagine a world without it. But the privacy and security is still an unresolved major concern—because the range of risks and threats that this can pose to the humans is unimaginable. This case study examines the privacy and security concerns prevailing right now and that which may arise in the future. This study has also analysed the various threats associated with artificial intelligence systems like unauthorized data access, misusing personal information and other potential security threats which can result in defamation of person or organization. It also aims to draw probable solutions for these existing concerns by shedding light on what can be toggled or modified in the existing framework in order to reduce the severity of the issue. This study also throws light on how stakeholders can navigate the technological changes and embrace it without compromising on the privacy and security.