Text classification tasks require a reduction from a sequence of tokens down to a single token. State-of-the-art Transformer models usually employ a class (CLS)-token to represent the entire sequence or pool all output token representations into a single token. In both schemes, the sequences are reduced in a single step. Although these approaches yield commendably high performance, we propose an integration of pooling operations into the scaled dot product mechanism that would facilitate a more nuanced reduction of sequence length. This article introduces a competitive and innovative pooling attention mechanism that is comparatively efficient and utilizes 2-dimensional pooling operations on the attention map generated in the scaled dot-product calculation. We further devise a sequence classification model that uses pooling attention to gradually reduce the input sequence to a single token within 8 or 4 attention steps. Our model performs comparably to well-established non-attention classification models while offering a promising new approach to nuanced sequence reduction in smaller pure attention models. A more nuanced approach to sequence reduction can aid explainability in attention models by examining sequences at different granularities across attention layers and extending the reduction process to multiple steps. Additionally, the computational cost is saved by processing shorter sequences in subsequent attention layers, improving overall efficiency. Our results show a significant speed-up in training time when comparing pooling attention with standard attention approaches. This work establishes a foundation for future exploration into sequence-reducing attention models.