The bivariate discrete-valued auto-regressive time series (BINAR) process has gained lot of popularity in the recent years due to the number of applications involving bivariate discrete data. Some examples include the intra-day transactions of two competing stocks, the time series of day and night accidents, the weekly number of COVID-19 infected and death observations, the series of male and female domestic violence cases, and among others. These data series are exposed to several challenges such as mutual over-dispersion, high ordered lags, significant sample cross-correlation, non-stationarity and especially are mutually high-ordered. Thus, there is the need to develop high-ordered bivariate integer-valued auto-regressive models with different innovation distributions under time-dependent moments to account for the non-stationarity. This chapter therefore focuses on the development of the different BINAR models of general order p. The estimation of parameters is conducted using the Conditional Maximum likelihood (CML) approach. The proposed model is applied to the COVID-19 series in Mauritius.