AI and epistemic injustices: Garbage data in, garbage out

Artificial intelligence is taking over every sector and aspect of life. Numerous companies and ‘experts’ are promoting generative AI as a solution for everything. Our lives will be better if we use AI, we are told: we will save time and effort to solve complex problems; we will be smarter; we won’t have to waste time to study things and learn the hard way; we can achieve what was until now unachievable, and so on.

Sure, there is potential for generative AI to aid humans in many tasks, sectors and industries that require processing of large and complex datasets. But we need to pause and critically examine the challenges that this technology and its corporate creators and proponents may bring to humanity, as well as to all levels of education.

Education: ‘A deeply human act’

A recent UNESCO paper on AI and education highlights that “education is – and should remain – a deeply human act, rooted in social interaction”. On the other hand, some educational organisations are already piloting the use of generative AI to teach and tutor students. For some, human teachers, educators and lecturers are not as effective as computer tools and thus need to be replaced by machines and chatbots.

We need to pause and critically engage with the purpose and processes involved in learning and teaching. While there is nothing wrong with innovation and technology, and while some of it can help us improve our work and practices, we cannot give up on traditional learning, human interaction and deep engagements with texts and materials.

We can forget about human intelligence and critical thinking if we rely on AI to give us summaries of texts and materials. The purpose of education cannot be learning to write prompts and then reading summaries given to us by chatbots.

Others have written about the potential of AI in higher education, and about students using AI to plagiarise essays. My focus here is on how AI tools and platforms propagate biases, bigotry and epistemic injustices that have plagued higher education worldwide since the European colonial conquest and propagation of white supremacy and racism that followed, and that continues to this day.

AI biases

The United Nations’ human rights experts have warned that without strict rules and oversight, generative AI platforms can enable the spread of disinformation that “promotes and amplifies incitement to hatred, discrimination or violence on the basis of race, sex, gender and other characteristics”.

This is already happening. The deep rooted, systemic and structural injustices and inequalities that have existed in the real world for centuries and have spread over the past few decades online are now being reproduced through AI, which relies on the data available online.

Just like knowledge and scholarship, “technology is never ideologically neutral. It exhibits and privileges certain worldviews and reflects particular ways of thinking and knowing.”

Researchers at the University of Copenhagen have found that ChatGPT, currently one of the most popular generative AI platforms, “is heavily biased when it comes to cultural values”. It promotes American norms and values and often presents them as ‘universal’ when asked to provide responses about concepts and other countries and cultures. This way, the researchers argue, ChatGPT acts as a “cultural imperialist tool that the United States, through its companies, uses to promote its values”.

An attempt by an Asian American student in the United States to use an AI tool to create a professional photo for her LinkedIn profile produced a photo in which she appeared white and blue eyed. For AI tools, this is what a ‘professional’ is supposed to look like. Not Asian or black, but white and blue-eyed.

These biases and racism by AI are nothing new. Researchers and journalists have been writing about this for years (see, for example, here, here and here).

Garbage data equals garbage results

Most importantly, AI tools do not become biased on their own. They learn to be biased and bigoted from humans.

AI platforms are trained using the data found on the internet. Anna Bacciarelli from Human Rights Watch highlights that using data available online to train AI tools “risks perpetuating the worst content-related problems we already see online – presenting opinions as facts, creating believable false images or videos, reinforcing structural biases, and generating harmful, discriminatory content”.

For many AI platforms, English language materials and the dominant Euro-American hegemonic, racist, sexist, discriminatory and bigoted worldviews and perspectives are the primary sources of data.

No wonder that the responses that come out of these tools are spewing the bigoted and hegemonic dogmas and presenting them as ‘universal truth’.

As the saying goes in computer science: when garbage data goes in, garbage results come out.

Decolonising knowledge

A lot of attention has been given to the use of AI to cheat in higher education, from students using it for essays to academics using it for their research projects and then presenting the results as their own. All this is relevant, but we must broaden the discussions and our focus.

Higher education and knowledge production globally continue to be highly unequal, with the Euro-American epistemic hegemony and domination, rooted in colonialism and white supremacy, being presented as the real and ‘universal’ knowledge. At the same time, ‘other’ worldviews, largely from the Global South, continue to be sidelined and presented as something of lesser value and relevance.

While the online ‘garbage’ which feeds AI platforms is the problem, the solution is not to feed the platforms the data from university libraries and scholarly research, as most of the materials there are white and propagate Euro-American-centric perspectives.

Structural change

As the journalist and New York University academic Professor Meredith Broussard points out, ‘better data’ is not the solution as long as the world remains riddled with systemic racism, sexism, ableism, patriarchy and other bigotries and social ills. Without systemic and structural societal change in the real world, cleaning up the data that is fed into the AI platforms won’t solve most problems.

Before we can talk about AI’s potential for learning, teaching, education and research, we first need what Professor Grieve Chelwa from The New School calls “wholesale decolonisation. Wholesale breaking down and building anew”.

We have to first decolonise knowledge and then use the pluralistic and non-hegemonic knowledges and perspectives to teach AI platforms. Anything else will only further deepen global inequalities and epistemic injustices.

Dr Savo Heleta is a researcher and internationalisation specialist at Durban University of Technology in South Africa.