Some Comparisons of Linear and Deep ReLU Network Approximation
摘要
Influence of depth of ReLU networks on growth of their nonlinearity is studied. Lower bounds on worst-case errors in linear approximation are derived for sets of highly-oscillatory functions that can be exactly represented by ReLU networks. Dependence of these errors on network depth is analyzed.