Max pooling layers are the basic building blocks of convolutional neural networks. The theoretical characterization of their computational power is therefore a question of central interest. This paper deals with the representability of the max pooling layer by neural networks (NNs) employing the ReLU activation function. We provide two upper bounds on the size (number of ReLU neurons) and depth (number of layers) of the NNs that implement the maximum \(\text{ MAX}_n\) of n nonnegative numbers. We show that the \(\text{ MAX}_n\) function can be computed either by a NN of size n and logarithmic depth, or by a NN of quadratic size and constant depth for bounded input numbers of limited precision, where the constant depth depends on the magnitude of the weights. As a lower bound, we prove that no NN of depth 2 can compute the maximum of more than two nonnegative numbers. This confirms that the max pooling layer cannot be replaced by just two convolutional layers.