Progress in Chinese Named Entity Recognition (CNER) has highlighted lexicon-based methods that use word information to boost performance. However, these methods neglect two crucial aspects: the regularity of word boundary and the characteristics of the NER task. To address these shortcomings, we introduce BADA-LAT, a Transformer architecture that incorporates word boundary information and introduce local attention computation. It can enhance entity boundary recognition and concentrate the attention weight of the target token on adjacent characters and matched words. Furthermore, to mitigate the issue of class imbalance, we augment the original training data using large language model (LLM). Our method outperforms other lexicon-based ones, as shown in experiments on four Chinese datasets.