The construction industry has the largest number of fatal accidents compared with other industries. Many factors affect the site safety management outcome and can be attributed to labor factors, environmental and equipment factors, engineering factors, management factors, and time factors. In addition, safety management program is promoted inside construction contractual organization; i.e., owner, contractor, and subcontractors. Their interrelated influence on the safety promotion is rarely explored in the past. This study employs Structural Equation Modeling (SEM) utilized occupational accident reports from 2014 to 2020 in Taiwan to investigate the direct, indirect, and total effects among construction contractual organization on occupational accidents. Furthermore, the explicit form was explored utilizing multiple logistic regression (MLR) for practical assessment. In conclusion, the SEM output indicates that the effect of safety management could be enhanced thru the organization of the construction safety management authorities to improve the effectiveness of overall construction safety management.