Challenges, Novel Approaches and Next Generation Computing Architecture for Hyper-Distributed Platforms Towards Real Computing Continuum

Computing platforms are evolving from centralized cloud datacenters, hosting huge computing and storage capacity to process data coming from external sources, to a more distributed and data-oriented architectures. As the result of this paradigm shift, we assisted to the raise of Edge computing as a more efficient way of (pre-)processing data on-site. Further, modern distributed computing architectures envisage the integration of a plethora of heterogeneous systems. This complexity comes with new challenges: the effective management of the vast heterogeneous resources and the dynamic nature of these platforms, with nodes joining and leaving frequently, necessitates adaptive and resilient management strategies, resource discovery, scheduling, and dynamic decentralized orchestration, to ensure and maintain system stability and performance. This paper provides an in-depth analysis of these challenges and draws a reference architecture exposing the necessary features to overcome the complexity of managing such complex distributed heterogeneous systems. By leveraging AI scheduling algorithms and federated learning techniques, proposed architecture aims to provide a robust and efficient solution matching with the demands for processing always growing masses of data and complex workloads.