A troubling pattern has emerged in corporate adoption of artificial intelligence: as companies increasingly integrate AI agents into their organisational structures and formal employee rosters, the humans managing them are paradoxically paying less attention to the quality of their output. This counterintuitive finding, documented by Boston University researcher Emma Wiles and her collaborators at Boston Consulting Group, challenges the assumption that treating AI like a teammate simply translates to smoother workflows and better productivity gains.
Wiles first encountered the phenomenon at a human resources conference in October when executives described integrating AI agents into their teams as a cutting-edge strategy for boosting efficiency. The appeal is obvious: AI doesn't demand salaries, healthcare, or time off, yet can process information and execute tasks at inhuman speed. Yet when her team conducted experiments across dozens of organisations, they uncovered a fundamental flaw in this anthropomorphic approach. Managers tasked with reviewing documents deliberately told they were produced by AI employees demonstrated noticeably weaker scrutiny compared to those reviewing identical work attributed to human colleagues or even generic "AI tools." The oversight gap was particularly pronounced at companies that had formally listed AI agents on their organisational charts, treating them as peers rather than software.
This oversight deficit reflects a psychological phenomenon rooted in how managers conceptualise responsibility. Traditional management practice, refined over centuries of industrial and corporate evolution, embeds accountability deeply into human relationships. A manager reviewing work from a direct report instinctively accepts personal responsibility for catching errors; if a subordinate submits flawed work, that failure reflects on the manager's supervision. This ingrained sense of duty makes managers scrutinise human-produced materials with particular care. However, when the same manager encounters work from an entity labelled an "AI employee," a different mental framework appears to activate. The presence of AI on the organisational chart seems to trigger a diffusion of responsibility, with managers unconsciously reasoning that technical teams or executive leadership bear ultimate accountability for AI performance. If something goes wrong, they reason, it belongs to someone else's domain.
The research also revealed that labelling matters more than substance. When managers knew they were reviewing work from an "AI tool" without formal employee status, they maintained relatively normal oversight standards. But the specific framing of AI as a "teammate" or formalised employee—particularly when accompanied by internal naming conventions like "Scout"—appears to fundamentally alter how managers engage with that work. Some organisations have even extended this logic to include AI agents in succession planning and team structures, deepening the sense that these systems occupy legitimate positions within the hierarchy. This linguistic and organisational sleight of hand may ultimately be more consequential than the technical capabilities of the AI itself.
The implications extend far beyond missed typos or minor errors. As Malaysian and Southeast Asian companies increasingly race to adopt AI to enhance competitiveness in global markets, they may be inadvertently exposing themselves to substantial operational and reputational risks. In sectors ranging from financial services to manufacturing to healthcare, the consequences of reduced oversight could be significant. An overlooked algorithmic error in pricing decisions could trigger costly price wars; a missed inaccuracy in hiring recommendations could create legal exposure related to discrimination; a failure to catch confidentiality breaches could compromise client data. The promised productivity gains and cost savings that justify AI adoption could easily evaporate under the weight of compounded errors that less-vigilant managers allow to slip through.
Wiles's broader research programme has identified numerous other pitfalls in workplace AI deployment that corporations appear to underestimate. She describes these as "unknown unknowns"—problems that companies haven't even begun to conceptualise because they've moved too rapidly into implementation. One emerging concern involves algorithmic bias that favours AI-generated content. Research from Ohio State University found that AI evaluation systems tend to prefer resumes written with AI assistance over those composed entirely by humans, potentially creating feedback loops that favour AI-augmented candidates regardless of actual qualifications. When recruiting firms learned of this bias, some inquired about remediation, but such awareness remains far from universal in the corporate world.
Another critical gap lies in how AI models approach strategic decision-making. Large language models tasked with determinations about pricing, market expansion, or competitive positioning tend to adopt a game-theoretic framework that prioritises narrow optimisation over the collaborative, cooperative instincts that typically govern human business relationships. An AI model might recommend aggressively undercutting competitors to maximise immediate market share, a mathematically rational approach that can spiral into destructive price wars harming all parties. As researcher Jiannan Xu from the University of Maryland observed, most large language models systematically overestimate human rationality and fail to account for the cooperative impulses that prevent optimal game-theoretic solutions from becoming real-world disasters. Companies deploying AI for such consequential decisions without recognising this divergence between AI logic and human business norms risk systematic strategic errors.
Further complicating the landscape are well-documented but underappreciated vulnerabilities in AI systems themselves. Models can produce confident-sounding answers that are factually incorrect; they can inadvertently expose proprietary or confidential information; they can encode biases against minority groups and underrepresented populations. Many companies remain vaguely aware these problems exist but lack systematic approaches to detection and mitigation. The rapid pace of AI adoption—driven by competitive pressure and fear of being left behind—has left insufficient time for most organisations to develop robust governance frameworks or implement the kind of intensive human oversight that might catch systemic issues before they propagate through operations.
Operations researchers emphasise that these problems aren't inevitable properties of AI technology itself, but rather emerge from the gap between technical capability and organisational readiness. Jane Yi Jiang from Ohio State stressed that the core issue is corporate haste: companies are "moving so fast to use LLMs without thinking too much about the implications, biases." The technology is advancing faster than organisational learning can accommodate, creating a window of substantial vulnerability. Companies that treat AI integration as a technical implementation challenge rather than a comprehensive organisational and governance challenge will likely face unnecessary losses and missed opportunities.
For Malaysian businesses operating in competitive regional and global markets, the lesson is sobering. The enthusiasm for AI adoption is understandable—the technology genuinely offers capabilities that can enhance productivity, reduce costs, and enable new business models. But uncritical deployment that mirrors Western corporate patterns without adapting them to local circumstances, or that treats formal employee status as a solution rather than recognising it as a source of new problems, risks squandering the potential benefits. Companies would be better served by viewing AI as a capability requiring intensive human oversight, not as an autonomous agent deserving of trust simply by virtue of its prominent position in the organisational chart. Only through deliberate, thoughtful integration supported by clear accountability structures and systematic error-checking protocols can organisations realise the genuine benefits AI promises.
