Parallel bi-state deep reinforcement learning approach for SFC placements and deployments

The SFC (service function chain) consists of virtual network function chains (VNFs) that generate provisioned network services on demand. To properly structure a network service, SFC traverses a series of VNFs in a predefined order. Thus, the VNF placement problem has become surrounded by other constraints that enshrine the order of VNFs according to their chain type. This restriction requires a method of extracting the best and shortest traffic from a chain of VNFs while considering duplication of the same kind of VNF. In this context, we propose to study and solve the problem of VNF placement and chaining in order to reduce the resources cost and minimize the end-to-end delay (chain of VNFs) while respecting the resources constraints (CPU, memory, and storage) in a multi-instance VNF environment taking into account the bandwidth (traffic congestion). To address these problems, we propose an ILP modeling and a novel model of classification named parallel bistate module deep reinforcement learning (PBDRL) algorithm. This algorithm is based on two modules: The first is MDP (Markov decision process) which is used to capture service chain state transition, and the second is LSTM (long short-term memory) which enables to detection of the long-term history of the service chain and its transitions. Our proposal aims to extract the characteristics of the VNF environment through MDP and LSTM simultaneously taking into account the dynamicity and history of the NFV environment. Simulation results show that our approach achieved \(47\%\) $47\%$ more reward than VNF deep approach and \(52\%\) $52\%$ than first-fit algorithm.