As applied artificial intelligence (AI) continues to advance in education, addressing the ethical and privacy concerns related to machine learning (ML) systems becomes increasingly important. These concerns often necessitate compromises among project collaborators involved in designing and deploying these systems, which can affect teachers, students, and other stakeholders. Our work emphasizes the integration of value-sensitive design (VSD) principles into the technical design and performance of ML systems within educational environments. To achieve this, we propose utilizing distributed and federated learning (FL) frameworks as decentralized approaches to training ML models, effectively tackling key issues such as data privacy and security. These frameworks enable us to maintain privacy and ethical standards while leveraging the collective expertise of educational and ML partners. We conclude by identifying potential directions for future research aimed at enhancing the capabilities of privacy-preserving ML in education. By combining VSD principles with technical innovation, we promote responsible and ethical ML practices in education while ensuring practical application.