In Chinese speech engineering, many projects use a conventional, syllable-based prosodic hierarchy as an underlying framework to process natural or synthesized speech. However, Chinese as a tone language has its own way of expressing prosody, that is, through tonal interaction, especially tone sandhi. By utilizing the capacity of tone as a dual unit of pitch and timing, the present study proposes a tone-based, three-layer-four-level structure for Chinese prosody. The three layers are tone, tone prosody, and intonation, respectively composed of one level of pitch units, two levels of tone prosody units (basic and derived), and one level of intonation units. These four levels of units are used to replace syllable, prosodic word, phonological phrase, and intonational phrase in a conventional prosodic hierarchy. Tone prosody units are established based on sizes or types of tone sandhi domains, so when applied to the same clause uttered in Mandarin and Shanghai Wu Chinese, they are timed differently and branched toward different directions at different levels, hence capable of capturing rhythmic and melodic patterns of the two distinctive types of Chinese. Overall, given its theory-friendly design, the proposed structure may be used as a unifying framework in Chinese speech engineering.