Multimodal sentiment analysis in videos is a key task in many real-world applications, which usually requires the integration of multimodal streams including visual, verbal, and acoustic behaviors. To improve the robustness of multimodal fusion, some existing methods allow different modalities to communicate with each other and model the crossmodal interaction via transformers. However, these methods only use single-scale representations during the interaction and do not exploit multi-scale representations that contain different levels of semantic information. As a result, the representations learned by transformers could be biased, especially for unaligned multimodal data. In this paper, we propose a multi-scale cooperative multimodal transformer (MCMulT) architecture for multimodal sentiment analysis. Overall, the “multi-scale” mechanism is capable of exploiting the different levels of semantic information of each modality, which are used for fine-grained crossmodal interactions. Meanwhile, each modality learns its feature hierarchies by integrating the crossmodal interactions from multiple level features of its source modality. In this way, each pair of modalities progressively builds feature hierarchies respectively in a cooperative manner. The empirical results illustrate that our MCMulT model not only outperforms existing approaches on unaligned multimodal sequences but also has strong performance on aligned multimodal sequences.