<p>As large language models (LLMs) continue to scale, emerging evidence suggests diminishing performance gains from further pretraining on high-quality, high-resource datasets, particularly those dominated by English-language corpora. This Perspective interprets this trend not only as a limitation of data quantity, but as a manifestation of semantic redundancy and epistemic narrowing within prevailing training distributions. We argue that contemporary LLMs increasingly function as proto world-models, implicitly encoding assumptions about meaning, causality, and knowledge that are disproportionately shaped by dominant linguistic and cultural sources. In response, this paper proposes a conceptual reframing of data strategy in LLM development. Rather than persistently extracting value from saturated corpora, we advocate for the systematic inclusion of culturally grounded and underrepresented knowledge systems, including low-resource and Indigenous languages, oral traditions (such as folklore, proverbs, and oral histories), and religious, philosophical, and historical texts. These sources introduce not only novel syntactic and semantic structures, but distinct epistemic frameworks that remain largely absent from conventional datasets. We suggest that such cultural corpora should be understood not merely as instruments of representational diversity, but as carriers of alternative world-models that can expand the epistemic capacity of language models. Their integration offers a potential pathway for mitigating redundancy, reducing reliance on synthetic data feedback loops, and enhancing generalization across diverse semantic domains.</p>