We introduce a unifying framework showing how modular architectures emerge naturally when connection patterns are regularized by an optimal-transport (OT) cost. We represent any candidate architecture as a probability distribution over possible links and consider a dynamic in which links are created or removed as part of the same learning process as modifying link weights. We then add to the usual task loss a penalty that charges more for creating or strengthening distant connections than for local ones. Under mild formal conditions indicating that there is an underlying modular structure in the problem the network is learning to solve – even if heavily obscured by other phenomena – we then show that the optimal architecture places almost all its mass on a small number of modules (with only a bounded “leakage” outside), and that any gradient-based update of the combined loss is likely to stay trapped within those modules until convergence. We illustrate this in three settings. In predictive-coding neural networks with columnar structure, an OT penalty on inter-column links drives the system to form edge, stroke and loop detectors before ever wiring far-flung columns, yielding a hierarchical, stroke-based feature scaffold. In probabilistic logic networks (PLN), chaining and pruning of belief links under a reasoning-distance cost produces clusters of related concepts with sparse bridges between them. Finally, we give speculative arguments that these same dynamics may occur in biological brains – i.e. synapse growth and pruning under metabolic and wiring-length pressures may instantiate a generalized OT flow that gives rise to orientation columns, place fields and motor primitives. Across these domains, minimizing task error plus an OT transport penalty provides a general principle for the self-organization of functional modules.