Women, Work, and Algorithms: How AI Hiring Tools Are Failing Caribbean Women

In 2018, Amazon quietly shut down an artificial intelligence recruitment tool it had spent four years building. The system had been trained on a decade's worth of successful hires, which were predominantly male, and had consequently learnt to penalise CVs containing the word "women" and to downgrade graduates of all-women's colleges. The story was widely reported, widely cited, and widely treated as a cautionary tale. Six years later, the lesson has not been learnt, and in the Caribbean the problem is arriving without even the benefit of the warning.

My hypothesis is straightforward: AI hiring tools currently deployed or being adopted by Caribbean businesses are encoding historical gender and racial biases into automated employment decisions, and no legal or regulatory accountability framework exists in the region to address this. The consequences fall disproportionately on women, particularly Black Caribbean women, who already navigate intersecting barriers of gender, race, class, and geography in accessing economic opportunity.

This article examines what we know about algorithmic hiring bias, why Caribbean labour markets are especially exposed, and what concrete policy changes are necessary before this problem becomes structurally entrenched.

The Algorithm Problem Is Not Hypothetical

The Amazon case is the most famous documented instance of AI hiring bias, but it is far from unique. Academic literature has accumulated substantial evidence that automated screening tools replicate and sometimes amplify patterns of historical discrimination. A 2019 study by Raghavan, Barocas, Kleinberg, and Levy, published in the Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, found that predictive hiring tools commonly use features that serve as proxies for protected characteristics, including postcode, educational institution, and name.

The mechanism is consistent: tools trained on historical hiring data will optimise for candidates who resemble previously successful employees. In organisations where women, and particularly women of colour, were historically underrepresented in senior and technical roles, the algorithm learns to deprioritise candidates who share their characteristics. The tool does not intend to discriminate. It simply repeats the past with greater speed and scale.

The WEF Global Gender Gap Report 2024 documents that women remain underrepresented across leadership, STEM, and technical occupations globally, including in the Caribbean. When those occupational structures become the training data for hiring AI, the algorithm inherits the exclusion as a feature rather than a bug.

Why the Caribbean Context Is Distinct

Caribbean labour markets have specific characteristics that intensify the risk. Women in the Caribbean outperform men at tertiary level by a significant margin: in Jamaica, women consistently account for approximately 60 per cent of university graduates. Yet this educational attainment does not translate linearly into labour market outcomes at senior levels. The gap between female educational achievement and female representation in leadership positions is well-documented in ILO reports on gender equality in Caribbean labour markets.

This mismatch matters for AI training data in a specific way. An AI system examining who holds senior roles in Caribbean organisations will find relatively few women, regardless of the talent pool available. It will then treat this distribution as normal and replicate it. The algorithm does not ask why women are underrepresented; it treats underrepresentation as the correct baseline.

The BPO sector illustrates this with particular clarity. Jamaica's business process outsourcing industry is a significant employer, with women making up an estimated 55 to 60 per cent of the workforce. Many of these employers rely on automated screening tools supplied by their international clients, particularly large North American corporations that outsource call centre and back-office operations. The workers applying through these systems have no visibility into the criteria being applied. They receive no explanation when rejected. They have no regulator to approach with a complaint.

The intersecting challenges of gender, race, class, and geography in Caribbean digital access compound this. GSMA Connected Women Report 2024 data confirms that the gender mobile internet gap persists across developing regions, with women less likely to own smartphones and less likely to have reliable internet access. This affects not only whether women can access digital job applications but also how their digital footprints appear to algorithmic screening tools that scan professional networking profiles and online activity.

The Legislative Gap

Jamaica's Data Protection Act 2020, administered by the Office of the Information Commissioner, represents a genuine step forward for data rights in the Caribbean. The Act establishes principles of transparency, accountability, and data subject rights including the right to object to certain automated decisions. These provisions have potential relevance to algorithmic hiring.

But the Act was drafted in a different moment, before algorithmic decision-making in employment became the mainstream practice it is today. It does not require employers to conduct bias audits of AI systems used in recruitment. It does not mandate transparency notices specifying when an automated tool has influenced a hiring decision. It does not give candidates a right to an explanation of why an algorithm rejected their application. And its enforcement provisions have not been tested against algorithmic discrimination.

Compare this to the EU AI Act, adopted by the EU Council on 21 May 2024. Under Annex III, AI systems used in recruitment and employment are classified as high-risk. This classification triggers requirements for conformity assessments, human oversight, technical documentation, and transparency measures before a system can be deployed. Caribbean countries are importing tools that would be heavily regulated in their countries of origin, and deploying them in a regulatory vacuum.

Labour Law's Blind Spot

Employment discrimination law across CARICOM jurisdictions generally prohibits discrimination on grounds of sex in hiring and employment. Jamaica's Employment (Equal Opportunities) Act provides this protection in principle. The difficulty is that these laws were designed for human decision-makers, not algorithmic ones. A woman who is rejected by an AI screening tool before her CV reaches a human being has been discriminated against, potentially, at a stage that is legally invisible.

She cannot demonstrate that a hiring manager saw her application and chose a less qualified male candidate. The algorithm made the decision before any human reviewed her file. Under current law, she has no standing to demand the algorithmic criteria that produced her rejection, no right to a human review of that decision, and no clear path to a discrimination complaint.

What the Seoul Summit and UN Resolution Signal

The international policy environment is moving. The Seoul AI Safety Summit of 21 to 22 May 2024 produced commitments from participating governments to address AI harms across a range of domains, including workplace applications. The UN General Assembly's first AI resolution, adopted on 21 March 2024, calls on member states to govern AI in a manner consistent with human rights and fundamental freedoms. Both instruments signal that the era of voluntary, unaccountable AI deployment is ending, at least in the most advanced governance jurisdictions.

Caribbean states participated in these multilateral conversations but have not yet translated the commitments into domestic policy action. There is a real risk that the gap between international AI governance standards and Caribbean domestic regulation will widen precisely at the moment when AI hiring tools are becoming standard practice in the region's largest employment sectors.

What Bias Looks Like in Practice

Consider a woman in Kingston who applies for a supervisory role at a BPO operation. She holds a degree from the University of the West Indies and has seven years of relevant experience. The employer uses an applicant tracking system purchased from a US-based vendor. The system scores candidates using a model trained on the employer's previous hires, drawn from a period when the supervisory grades were predominantly male.

The system assigns her a lower score than a male candidate with equivalent qualifications. The score is not labelled as a gender score. It is labelled as a "fit score" or a "culture match" or a "potential indicator". No human examines why she scored lower. Her CV never reaches a recruiter. She receives an automated rejection. She has no way of knowing that an algorithm, not a person, made this decision.

Recommendations

1. Amend the Jamaica Data Protection Act 2020 to require mandatory bias disclosures for AI hiring tools. Any organisation using an automated or AI-assisted system in a hiring process should be required to disclose this to applicants, specify the criteria used, and provide a right to human review of automated rejections.
2. Introduce a sector-specific AI hiring standard for BPO operators. The Jamaica Promotions Corporation and the Business Process Industry Association should jointly require that all AI recruitment tools undergo an independent bias audit before deployment.
3. Classify AI hiring tools as high-risk in any forthcoming CARICOM or national AI policy framework. The EU AI Act's classification of recruitment AI as high-risk provides a clear model. Caribbean governments should adopt equivalent classifications in their own policy frameworks.
4. Require employers to report disaggregated hiring data by gender. This data requirement should be built into the Statistical Institute of Jamaica's labour market data collection.
5. Commission a Caribbean-specific study of AI hiring tool deployment and its gender impact. The University of the West Indies, in collaboration with UN Women Caribbean and the ILO Caribbean office, should conduct this research within 18 months.
6. Establish a women's economic rights unit within the Office of the Information Commissioner. The Office should have dedicated capacity to receive, investigate, and adjudicate complaints from women whose employment opportunities have been affected by automated systems.

Conclusion

The Amazon case in 2018 was a warning. The EU AI Act in 2024 is a response to that warning, built on years of documented harm and sustained advocacy. The Caribbean is arriving at this conversation later, and that lateness carries a specific risk: the tools are already here, and the governance is not.

Caribbean women are not passive recipients of technology. They are workers, entrepreneurs, graduates, and professionals who deserve to compete for employment on the basis of their qualifications and capabilities, not on the basis of what an algorithm has learnt from a past in which they were excluded. The legal and regulatory frameworks that should protect them are incomplete. Completing them is an urgent task.

About the Author

Dr. Shirley Budall is a Caribbean expert in gender, inclusion, and AI governance with demonstrated experience in the ethical, legal, social and governance dimensions of artificial intelligence and digital technologies. She conducts legal and regulatory framework reviews and develops policy recommendations for legal reform in AI governance, data protection, human rights, and gender equality. Contact: insights@starapple.ai