This paper presents a teleoperation framework for operating robots in unstructured environments or in a dynamic manner. Teleoperation systems frequently encounter challenges such as collision risks and abrupt deviations in robot movements. To address these issues, we introduce a framework that incorporates a constrained nonlinear disturbance observer (C-NDOB). The C-NDOB combines a nonlinear disturbance observer (NDOB), an extension of the disturbance observer (DOB), with quadratic programming (QP). This approach allows us to impose constraints on the nominal dynamics, resulting in contact-responsive motion during collisions and stable motion when performing dynamic tasks. Importantly, this method simplifies the constraint design process for various teleoperation scenarios, allowing for broader applications. The proposed framework is validated through real-world experiments.