Byte pair encoding (BPE) tokenization is a popular technique for subdividing text into relevant subwords and is frequently used in large language model systems. Since the tokenization procedure is deterministic and produces regular languages, it is beneficial to characterize it in terms of finite automata to allow for further automata-aided processing, such as pattern matching. In this paper, we demonstrate how to build such automata efficiently in practice, applying a series of optimization techniques to represent them compactly.