The Berlin Conference of 1884–1885 which seemingly settled the scramble for Africa by European powers created opportunities for arbitrary creation of artificial states in Africa. The artificial demarcation of territories resulted to separation of people who were originally meant to live as a nation, and unification of peoples who originally had no form of common history, culture, language and values. This study investigated how colonial rule in Africa impinged on the formation of a collective national identity within African countries. The ethnic sentiments and rivalries orchestrated by colonialism have proved to be the greatest Achilles Heel of many African countries, which have triggered post-independence civil wars as was the case in Sudan, Somalia, Central African Republic, Mali, Nigeria, Rwanda and the Democratic Republic of Congo and so forth. Although the artificiality of the African states is found to be a fundamental factor but the recurrent waves of inter-ethnic conflicts are traced to a deliberate deployment of ethnic sentiments by the political elite—demagogues who weaponise poverty and ignorance to either win elections or to pursue parochial and primordial interests. Without instituting justice systems that criminalise the use of ethnicity for political leverage, achieving nation-building in Africa will be a mirage.