Recent years have witnessed a notable surge in deep learning models for the Long-term multivariate Time Series Forecasting (LTSF) problem. These models are mostly built on the Transformer architecture due to its strong capability to unravel the inherent complexities of LTSF. However, a critical study has indicated that simple linear models can perform as well as or even better than Transformer-based models while requiring much fewer computational resources. Motivated by this, we propose a new approach that not only embraces the efficiency of linear models but also effectively integrates multivariate information for LTSF. Our approach focuses on capturing the intra- and inter-variable (a.k.a. channel) relationships jointly in multivariate forecasting. This requires us to design a MultiVariate Information Network (MVINet) model to extract dependencies among multiple channels over time intervals for improved prediction performance. Extensive experimental results demonstrate that our proposed MVINet model outperforms several baseline methods for LTSF on five benchmark datasets. In particular, it achieves prediction accuracy close to or better than that of Transformer-based methods while having much higher efficiency. Meanwhile, its accuracy is significantly better than that of existing linear models.