The Serial Data Processing Unit: High Performance with Parallelism in Network Processing

With the emergence of compute and network-intensive applications in AI/ML, data center infrastructure is being pushed to the limits to meet stringent performance and scaling requirements. Additionally, it is basically a given that data center infrastructure must be completely malleable and adaptive. One area that has been troublesome for meeting the requirements is server network processing. Getting high performance in-network processing and being flexible at the same is essentially an oxymoron. The options are either to use fixed function hardware that offers high performance with no flexibility or run software on general purpose CPUs that provides immense flexibility but is very underperforming compared to hardware. In this paper, we propose a novel alternative solution called the Serial Data Processing Unit or just SDPU. The central idea is that the critical network processing path is manifested by a software program that runs in Domain Specific CPUs. Domain Specific CPUs follow the principles of Domain Specific Architecture and are tailored for high performance in network processing. As CPUs they are inherently programmable, and we’ve defined a complementary programming model which makes it easy for a developer to program their networking data path. Achieving high performance comes from their domain specific nature – we start with a base CPU architecture and add features to accelerate workloads and processing in the domain of networking. Our baseline CPU is RISC-V and we’ve enhanced the CPU architecture in three areas: specialized CPU instructions, integrated accelerators, and fine-grained parallelism. The latter is perhaps the most impactful innovation and thus is a focus in this paper. Taking all these together, we project that the SDPU will deliver high performance and yet be fully flexible and cost effective.