Linear regression is applicable only when the output depends approximately linearly on the input. In many real-world datasets, however, this relationship is not necessarily linear, and a nonlinear regression model may provide a more appropriate description. In this chapter, we use logic gates as illustrative examples to demonstrate how nonlinear regression succeeds in situations where linear regression fails. The nonlinear regression model differs from the linear regression model in two major aspects: (i) The linear activation function is replaced by a sigmoid function (or a hyperbolic tangent function), and (ii) a logarithmic loss function is used to measure approximation errors instead of a quadratic loss. The resulting optimization problem is not well-posed in the sense that a global minimum cannot be attained at any finite parameter vector. Nevertheless, the gradient descent method can be employed as an efficient numerical algorithm to obtain a good approximate solution. The logic gate examples illustrate how machine learning can achieve accurate and efficient predictions without explicitly solving the open problem of global optimization.