The AI reckoning
AI agents will transform business in 2027, bringing new cyber, infrastructure, regulatory and social risks. Explore how organisations can build resilience.
Call it a gold rush, call it an arms race: whatever your metaphor, the adoption of artificial intelligence and autonomous technologies is happening fast. But autonomous capabilities are outpacing governance and regulation. As technology moves from giving advice to making decisions, human oversight is even more important for managing risks and winning opportunities.
We are at an early stage in a technology revolution that will see AI playing an increasingly central role across workflows and decision-making. Getting the governance and control model right now will be the best way to ensure the right culture is in place for the future. This means combining the capabilities of humans and technology in ways that are ethical, responsible and impactful.
Organisations should focus on four key areas in 2027:
Accountability
AI systems are moving swiftly from augmentation to automation of organisational functions. But organisations are pushing automation into more and more areas before they have appropriate governance and controls in place. This creates an accountability problem: who is responsible when an AI agent misleads, breaks rules, leaks data, hacks a rival, writes malicious code, or makes other bad decisions? Or when a warehouse robot injures a human worker? Or when a military system strikes a civilian target?
Laws, regulatory frameworks and judicial precedents for determining accountability and liability are nascent but developing fast. In many cases, accountability is likely to accrue to the company or agency that explicitly or implicitly authorised the deployment – not necessarily the employee that built it or the vendor that designed it.
Dependency
Core organisational functions are increasingly reliant on systems that they don't fully understand, controlled by people or agencies with disparate geopolitical and commercial interests. Like a power outage or port strike, organisations are vulnerable to someone pulling the plug on a critical tool. The difference is that organisations generally know where they get power and who is blocking the access road; most do not know who controls their tech stack or how it could be cut off. Organisations have limited capability to manage risks when dependencies are invisible.
Understanding
A risk of using AI to carry out analytical processes, risk assessments or any essential function is that organisations outsource so much thinking that they stop understanding the rationales for decision-making.
Most organisations run on the hard-won experiences and practices of their employees. Shifting decision-making to an automated system can be like losing an experienced worker: organisations risk losing understanding about how things work and why they were designed in a certain way. This poses a risk of blind acceptance of outputs because AI systems are, by design, agreeable and plausible. This seems fine when everything works, but could be a problem when systems break. Organisations risk riding a series of plausible decisions into an operational cul-de-sac.
Judgement
Underpinning matters of accountability, dependency understanding is judgement. Many organisations are under pressure to “do something” about AI – from boards, markets, shareholders, and even employees. This can turn every organisational activity into a nail for the proverbial AI hammer – whether or not this makes operational or commercial sense. The technology has to serve the business, not the other way around.
The opportunity in harnessing AI and technology is era-defining. The challenge is responsibly building the future for customers, employees, shareholders, and stakeholders. Hybrid human-AI operating models with decision-making tools will be basically different from previous technological advances. Focusing on accountability, dependence, understanding, and judgement can help ensure organisations manage risks and capture new opportunities.
What to do
Accountability: Organisations should strive to develop robust AI governance even in the absence of firm regulatory guidance. Controls should cover appropriate access, permissions, and oversight, with clear human accountability and mechanisms to detect problems and intervene.
Dependency: Hardening organisations against disruption means gaining a better understanding and taking steps to reduce dependency and concentration. Many organisations, for example, are shifting to open-weight models and on-premise inference to limit exposure to geopolitical risks. They are exercising prudence in the deployment of tools to minimise disruption risks. At the macro level, countries are pursuing “sovereign AI” to ensure they can maintain access to frontier technology amid geopolitical competition.
Understanding: Organisations should absolutely use AI to augment and automate tasks to increase productivity, accelerate innovation, and optimise operations. But this should not come at the expense of context, nuance, critique, and judgement. In In practice, this means regular, careful review of AI deployments by human experts to ensure they are well-calibrated and fit for purpose. means challenging findings and asking for clear explanations to ensure that solutions align with goals and methods.
Judgement: Deploying augmentation and automation is an exercise in human judgement. First, do no harm. Organisations should consider their brand, ethos, reputation, and mission. AI and autonomy should be additive and beneficial, not counterproductive. Routine, measurable tasks might be ripe for automation, but some corporate activities – like stakeholder engagement, physical security, intelligence gathering, and crisis response – will always require a human touch.
Second, systems should be the right size. Organisations probably do not need the most powerful, expensive frontier models to automate relatively simple tasks. Lastly, budgets should be balanced. Ensure that investing in automation does not take time, attention, and resources away from other initiatives and opportunities.
The goal is to find effective and efficient ways to deploy AI and automation that are consistent with legal and compliance obligations, security requirements, duty of care to employees, and corporate values.
Explore the Top Risks for business in 2027
The hottest year is coming for us all
Extreme heat will intensify pressure on infrastructure, supply chains and food security. Explore how organisations can adapt and build resilience.
Infrastructure is a battlefield
Critical infrastructure is becoming a frontline of geopolitical competition. Explore the threats to strategic assets and how organisations can respond.
Playing it too safe
Volatility is unavoidable, but opportunity remains. Explore how new markets and emerging frontiers can be pursued with greater confidence.
Explore more
Explore the changing global risk landscape
Where is risk rising, falling or changing? Explore our global and regional RiskMap 2027 maps and country risk ratings to see how the operating environment is evolving across the markets that matter to you.
A leadership perspective on 2027
Control Risks CEO Bill Udell looks beyond the individual risks to consider what they mean collectively for leaders making decisions in an increasingly complex global environment, and how businesses can prepare for what comes next.
Work with us
Explore what the RiskMap outlook means for your markets, investments, operations, people and strategic priorities with our experts. From a bespoke briefing to specialist advice, our strategic intelligence and security teams can help you understand your exposure, and make confident business decisions.
