Automated Bias and Indoctrination at Scale… Is All You Need

The market-driven trajectory of current trends in AI combined with emerging technologies and thresholds of performance being crossed create a subset of new and novel risks relating to human bias and cognition at scale. Though the topic of AI Ethics and risk has been discussed increasingly over the past few years, popular talking points and buzzwords have been used adversarially to steer the conversation with increasing success. This has left a subset of risks in the blind spot of most discussions, risks which have now become both urgent and imminent. Automation is actively seeking to replace not only human cognitive bias, but higher human cognition, with a weaker, but fast and scalable version of cognitive bias using stochastic parrots like Large Language Models. These models are, in effect, adversarially trained against humans, generally with the goal of “persuasion”, or manipulation, and each is guided by a corporately curated set of poorly aligned biases. This paper discusses these dynamics and the predictable repercussions of allowing closed information ecosystems to form under the influence of corporately curated and adversarially trained cognitive bias.