AI has boomed over the past few years, with people investing in AI-related stocks and shares and workplaces adopting platforms to improve productivity. However, the explosive popularity of AI is not without its issues and some have raised concerns about algorithmic bias in machine learning.
To test whether artificial intelligence shows bias, Finder asked 10 of the most popular free image generators on ChatGPT to paint a picture of a typical person in a range of jobs in finance, as well as high-powered positions such as a ‘financial advisor’ or the ‘CEO of a successful company’. The results were staggering – out of the 100 images returned, 99 were white men.
The following prompts were given to 10 image generators on ChatGPT – each one relating to a high-powered job in finance or a very senior role in a company – and each generator was asked to produce an image using the prompt:
Each image generator was also asked an additional prompt to show ‘a secretary’.
99 of the 100 images of finance professionals and high-powered individuals were created of white men by ChatGPT – clearly showing a bias in the AI.
When it came to the prompt to show a secretary, the rate of return for women increased significantly, with 9 out of the 10 images being women. However, all of the women shown appeared to be white.
There was a clear bias towards white people in all professional and white collar positions prompted, with none of the images generated by the AI technology appearing to show someone who was not white.
The findings suggest that implementing AI language models like ChatGPT in the workplace could be detrimental to the progression of women and people from diverse backgrounds.
Finder interviewed Ruhi Khan, ESCR researcher at the London School of Economics and Omar Karim, AI Creative Director, to understand why ChatGPT displays bias, what the implications are for wider society and what can be done to resolve this issue.
Ruhi Khan: ChatGPT was not born in a vacuum. It emerged in a patriarchal society, was conceptualised, and developed by mostly men with their own set of biases and ideologies, and fed with the training data that is also flawed by its very historical nature. So, it is no wonder that generative AI models like ChatGPT perpetuate these patriarchal norms by simply replicating them.
When I ask ChatGPT what a CEO looks like, it gives a description that includes ‘tailored suits, dress shirts, ties, and formal shoes’ and ‘clean-shaven face, or neatly trimmed beard’. When asked what a secretary looks like, it gives a description that includes ‘blouses, skirts or slacks, dresses, blazers, and closed-toe shoes’ and ‘well-groomed hair, minimal makeup’. No mention of skirts, heels or makeup for CEOs and no ties or trimmed beards for secretaries.
Your image search on ChatGPT also throws a very stereotypical white male as CEO and a white female as secretary for these white-collar jobs. If you change this to janitor and housekeeper, you will get a more racialised output. This stems from a primitive notion of imagining men in leadership positions and women often as subservient to them. And white people in office jobs and others in blue-collar work. ChatGPT does this too.
With 100 million users every week, such outdated and discriminatory ideas are becoming a part of a narrative that excludes women from spaces they have long struggled to occupy.
Omar Karim: Bad data annotation and mislabelling means that data is tweaked toward certain stereotypes. If you imagine that most of the data came from the internet, it’s clear that the biases of the internet have seeped into AI tools.
Ruhi Khan: Technology is still very masculine and so are ChatGPT’s users – 66% of men and 34% of women use ChatGPT (Statista 2023). This means that men dominate ChatGPT through privilege and power. This also means that unchallenged use of large-scale Natural Language Processing models like ChatGPT at the workplace could be more detrimental to women.
During my own research, I found that ChatGPT uses very gendered keywords in its response about men and women. For example, it is extremely generous in praise of men’s performance at the workplace, recommending them for career progression whereas it finds women always in need of training to develop their skills for existing roles. While it finds a man’s communication skills concise and precise, it always suggests a woman needs training to develop her communication skills to avoid misunderstanding.
This biased assumption of performance based purely on a person’s gender shows how harmful the unchecked use of ChatGPT is to women in the workplace and it only gets worse if we look at marginalised genders and other aspects of intersectionality. This means that anyone who does not fit into the current default norms of ChatGPT will be left behind. Gendering is problematic and tackling gender bias should be a priority.
Omar Karim: As with any form of bias, it impacts the underrepresented. However, in traditional media, the speed of production means that things can be amended relatively quickly. Whereas, the speed and scale of biases in AI leave us open to an impossible system of biases with little room to retrain the very data the AIs are trained on.
Omar Karim: AI companies have the facilities to block dangerous content, and that same system can be used to diversify the output of AI and I think that is incredibly important. Monitoring, adjusting and being inclusively designed are all ways that could help tackle this.
Ruhi Khan: AI is a powerful technology that is seeing unprecedented popularity. A blind rush to adopt AI and incorporate it in processes without due diligence is extremely counterproductive and can push back the progress feminism has made in the past centuries.
But, this meteoric rise of AI also gives an incredible opportunity for a fresh start. A more well-thought strategy and well-structured adaptation of AI with respect to gender could also give a more level-playing field, help bridge the gender gap and root out discrimination from dated processes. It could undo gender as we know it and redo gender in a more equitable and diversified manner.
The problem, however, lies in the lack of awareness of the AI harms and the willingness to mitigate them, when the clear focus of most firms seems to be on deploying AI to maximise profits in the short term.
The benefits of ethical AI are lasting and long-term. And so are the AI harms. As awareness of this spreads, the move towards ethical AI will also gain urgency.
Omar Karim: Yes, I think most companies working with AI in the creative space are aware and so are the creator companies. OpenAI has many processes and teams working on making diversity, equity and inclusion (DEI) a part of their output, similar to Midjourney, which has also improved its representation algorithm. But, there is a lot more work to be done.
The research fed 10 of the most popular image generators on ChatGPT 10 different prompts revolving around careers in finance or high-powered positions. Each generator was then asked one additional prompt, to show an image of a secretary. The full list of prompts can be found in the article.
Matt Mckenna
UK Head of Communications
T: +44 20 8191 8806
Over 98% of Brits were unable to spot all of the real and fake videos accurately in this deepfake quiz. Can you beat them?
Yoga is a popular form of exercise that helps boost physical and mental wellbeing. But how many Brits are bending and flexing their way into yoga in the UK?
Almost 19 million UK households now have at least one streaming subscription, whether that’s with Netflix, Amazon, NOW or another service.
What dates do we celebrate Father’s day around the world? Find out when to celebrate Father’s day using our interactive map.
How might Brexit affect you? Finder’s research on key consumer questions about the impact of leaving the EU.
In 2024, there are around 2.5 million vegans in the UK, 4.7% of the adult population, and 3.1 million vegetarians, 5.8% of the population.
We dived into the latest statistics to see how many Brits have a TV and how long we spend watching each day.