This week I attended a UN multi-stakeholder consultation on AI governance as an observer. As a wee little fly on the wall—positioned somewhere at the intersection of digital learning systems, quality assurance, and accessibility across multi-country environments—I listened to contributions from across the globe. There were structured, thoughtful, and often highly articulated perspectives on frameworks, safeguards, and the future of responsible AI.
Yet as the conversation unfolded, I became aware of two parallel experiences. On the surface, there was coherence and confidence in how these systems could be governed. Beneath that, however, were a number of assumptions that felt less examined—particularly around neutrality, control, and who ultimately shapes the direction of these systems. It occurred to me, quietly but persistently, that there is still so much we do not fully understand—and while that uncertainty is human, I am not entirely convinced we should be comfortable handing it over to systems we cannot fully see into.
Because opacity is not just a technical limitation; it is a condition in which gaps in understanding remain unchallenged. And systems trained to optimize, predict, and generalize will inevitably operate within—and potentially exploit—those gaps in ways we are not always equipped to anticipate or interrogate.
Much of the discussion around artificial intelligence still treats bias as a technical issue that can be identified and corrected. Yet machine learning systems, especially large language models, are trained on vast corpora of human-generated data that are themselves shaped by history. That history includes uneven distributions of power, linguistic dominance, economic disparity, and cultural prioritization. These systems are not learning from an objective or balanced record of humanity; they are learning from an archive that reflects existing hierarchies. In that sense, bias is not a defect introduced during development but an inheritance embedded within the data itself. When such systems scale, they do not leave those inheritances behind—they replicate and amplify them.
This ties back to my issue with opacity. The increasing reliance on complex models whose internal decision-making processes are not fully interpretable introduces a significant governance challenge. Even within development teams, there are limits to how precisely outputs can be traced back to specific inputs or reasoning pathways. Despite this, these systems are already being operationalized in contexts that affect real people—filtering job applicants, shaping access to information, influencing decisions in education, finance, and beyond. The combination of inherited bias and limited interpretability creates a situation in which outcomes may carry real consequences without a clear mechanism for explanation or redress. When systems cannot be meaningfully interrogated, accountability becomes difficult to establish, and the normalization of subtle, cumulative harm becomes more likely.
Another dimension that warrants closer scrutiny is the geopolitical structure within which AI governance is being developed. While the language used in these discussions emphasizes global collaboration, the reality is that much of the infrastructure, funding, and standard-setting authority remains concentrated in institutions based in or aligned with the Global North. These actors are not only building and deploying AI systems but are also heavily involved in defining the frameworks intended to govern them.
This raises important questions about whether current approaches to AI governance are genuinely global in orientation or whether they risk extending existing power asymmetries into the digital domain. When those who design the systems also shape the rules that govern them, there is a natural tendency toward top-down influence, even when inclusivity is formally acknowledged.
This dynamic becomes more complex when considering how different regions participate in and are represented within these systems. Populations from the Global South are increasingly present in datasets, whether through language use, behavioral patterns, or digital activity. However, representation within data does not necessarily translate into proportional influence over system design or governance. When participation is broad but decision-making authority remains concentrated, the relationship begins to resemble a familiar pattern—one in which value is extracted and incorporated, while control is retained elsewhere. In this sense, discussions of AI governance cannot be separated from longer histories of knowledge production, resource control, and whose perspectives are prioritized in shaping global systems.
These concerns are not only theoretical. Since the beginning of this year, I have also been navigating markets that are increasingly mediated by AI-driven processes. Application systems, candidate vetting platforms, and automated assessments now form a significant part of how professional opportunities are accessed and filtered. What stands out in these interactions is not only their efficiency but also their opacity. Roles appear and disappear with little explanation, evaluation processes unfold without transparency, and feedback is often absent. The experience can feel less like professional engagement and more like passing through a series of automated filters whose criteria are not fully visible.
Over time, this raises a more unsettling question about the nature of participation in such systems. Increasingly, interactions within these platforms do not simply evaluate candidates; they also generate data that can be used to refine and train the systems themselves. As someone whose work involves breaking down complex bodies of knowledge into structured, modular components for learning, I recognize parallels between my professional practice and the types of input these systems require at scale. The processes of categorizing, tagging, sequencing, and simplifying information are central both to instructional design and to machine learning workflows. This overlap invites reflection on where the boundary lies between contributing professional expertise and, inadvertently, contributing to the ongoing training of systems that operate beyond one’s visibility or control.
Taken together, these experiences—listening to governance discussions on one hand and engaging with AI-mediated systems on the other—highlight a gap between how AI is framed and how it is lived.
Governance conversations often operate at a level of abstraction that does not fully capture the complexities of implementation across diverse contexts. In practice, systems must function across languages, infrastructure constraints, cultural differences, and varying levels of digital access.
From my work across multi-country learning platforms, I have seen how quickly systems can fail when these contextual factors are treated as secondary considerations. At scale, such oversights do not simply result in inefficiency; they can reinforce and extend existing inequities.
For AI governance to be meaningful, it must grapple more directly with these realities. This includes acknowledging that bias is structurally embedded rather than incidental, that opacity limits accountability, and that global participation must extend beyond consultation to include genuine influence over design and decision-making. It also requires recognizing that human interaction with AI systems is not neutral—that the data generated through everyday use may itself become part of the system’s ongoing development.
At a more personal level, this conversation is not abstract. As a woman of colour from a small island, part of a region often overlooked in global decision-making, it is difficult to ignore how uneven the playing field already is. This is not only about geography, but about structure—about the inherited patterns of patriarchy and hierarchy that continue to shape who is heard, who is trusted, and who is positioned to lead. These dynamics are not incidental; they are deeply embedded, coursing through institutions and everyday interactions alike.
Within that context, the structures being built feel, at times, as though they are designed to position people like me primarily as consumers of technology or, at best, as anomalies ( at worst, phantoms) within it. There is limited space to exist as a contributor with influence, despite the depth of expertise that exists outside of traditional or historically dominant centers of power.
And yet, that is precisely why this matters. Even within systems that are not designed with us in mind, there are those of us who continue to engage, question, and push against the edges of what is assumed—not from a place of disruption for its own sake, but from a recognition that the future being built will affect us regardless of whether we are meaningfully included in shaping it. I remain attentive to where these conversations are going, and to where they fall short, because if governance is to be global in any meaningful sense, then it must account not only for those with clout, but also for those who have long learned to navigate systems that were never designed with them in mind. If governance remains opaque, centralized, and detached from lived realities, we are not correcting imbalance—we are standardizing it.