Federated Learning as a Service (FLaaS) enables 3rd-party applications to implement federated learning (FL) without needing to build and manage the complex infrastructure. However, the deployment of FL tasks faces the problem of heterogeneity in real-world devices, which may lead to significantly unbalanced per-iteration training speed and gradient quality among different devices, decreasing its Quality of Service (QoS). A promising solution is to customize the model for different devices, but existing approaches bring numerous computational overhead to resource-limited devices. To address this issue, we propose HierCust, a three-layer hierarchical model customization framework for FLaaS on heterogeneous edge devices with a SuperNet-InitNet-SubNet architecture. We conduct extensive experiments over two widely adopted public datasets, i.e. CIFAR-10 and ImageNet, and use large-scale trace data from 136k smartphones to faithfully reflect heterogeneity in real-world settings. The results demonstrate the superiority of HierCust over state-of-the-art FL model customization approaches in terms of accuracy, computational overhead, and communication overhead.