RGB-Thermal (RGB-T) crowd counting addresses the challenge of estimating the number of people using both RGB and thermal images. To capture global information, an increasing number of methods leverage transformer-based approaches. However, these methods directly predict either the number of people or the density map, without estimating the precise position of each person. Furthermore, the effective fusion of complementary information from RGB and thermal images in transformer-based RGB-T crowd counting methods remains largely unexplored. To address the first challenge, we introduce a point-query transformer, where each point query is tasked with predicting a location and determining whether the location corresponds to a person. This approach enables the provision of fine-grained information rather than solely the crowd count. To tackle the second challenge, we introduce two methods to integrate complementary features of both RGB and thermal modalities. One method is the Cascade Coarse-to-Fine network (CCF), where separate RGB and thermal decoders predict the crowd count based solely on their respective modalities firstly. Then an RGB-T decoder is introduced to refine results using features from both RGB and thermal decoders. Specifically, for the RGB-T decoder, queries are initialized using output features from both decoders, enhancing its representation learning capacity significantly. The other method, i.e., Feature Enhancement Module (FEM), enriches cross-modal contextual features. FEM achieves this by integrating self-attention and cross-attention mechanisms within the features from transformer encoder. In summary, we present Cascade Coarse-to-Fine point-query Transformer (CCFT) for RGB-T crowd counting. Experimental results demonstrate that our CCFT consistently outperforms state-of-the-art methods on RGBT-CC dataset.