Handwritten text generation (HTG) aims to imitate specific writers to synthesize handwritten text images, which has a wide range of applications and thus has significant research value. However, current studies on HTG face a central bottleneck. Dominant models, built utilizing generative adversarial networks (GANs), constantly struggle with error conditioning from the style classifier due to long-tailed distributions. The issue results in samples generated by mainstream HTG methods lacking diversity. In this work, we propose a plug-and-play method, namely Dynamic Category Compression (DCC), to enhance the recognition ability of the style classifier and provide potent constraints for the generator. Specifically, DCC improves the distinction of tail class samples by adaptively increasing the density of backbone features. We apply DCC to state-of-the-art methods and previous prominent works, then evaluate them on widely used handwriting datasets. Extensive experiments fully demonstrate the effectiveness and generality of our method. Our code is publicly available at:  https://github.com/Fyzjym/DCC .