The technological world is changing under the influences of the innovations of Artificial Intelligence (AI) and its integration with cloud computing. The change, however, brings along its own challenges. In response, new privacy policies are being established in the entire world. In this chapter, we will discuss how data privacy and governance have evolved in the age of AI. We will discuss in detail about the privacy risks associated when using AI in the cloud. Various countries have built laws and regulations such as GDPR, CCPA, DPDPA, and PIPL that companies need to follow to preserve privacy of user data. We will learn about those in detail, as well as we will see how companies can build governance frameworks to design systems that are secure and privacy-preserving. Throughout the next few years, as AI continues to advance and integrate into daily activities, the way we manage and protect data will become even more important. This chapter will help you understand the challenges and learn how to build AI-cloud systems that are both powerful and responsible and how companies can be proactive with what’s coming up in the field of AI and cloud.