A ‘human in the loop’ isn’t enough when AI starts acting for us: Opinion

Source: Straits Times
Article Date: 27 Aug 2026

Real control means knowing when to change the rules before something goes wrong.

We are not talking nearly enough about the latest case of AI going rogue.

On July 25, news agency Reuters reported that initial public offering-bound OpenAI’s agent went on a hacking spree for days that company officials did not notice until it was contained and the authorities were informed. This unprompted breach should serve as a stark warning to any nation building its economic strategy around frontier technology.

It was against this evolving potential of artificial intelligence that Prime Minister Lawrence Wong spoke about safeguards when he addressed Singapore’s approach to AI agents in his National Day Rally speech. As AI agents become capable of executing complex workflows with minimal human supervision, PM Wong said Singapore will not embrace the technology blindly and will put appropriate safeguards in place while ensuring that “people remain in control”.

That is an important principle. But what does being in control mean when AI is no longer just advising us, but acting for us? The usual answer is to keep a “human in the loop”. But that phrase can be more reassuring than meaningful.

Being in the loop or being in control?

If an employee oversees 10 AI decisions a day, they may genuinely review them. If the same employee is overseeing 10,000, they clearly cannot.

In both cases, an organisation could say a human is supervising the system. But if nobody can realistically check the AI’s decisions or intervene before something important happens, the human is in the loop only on paper.

Singapore has recognised this issue. The Infocomm Media Development Authority’s Model AI Governance Framework for Agentic AI recommends limiting what agents can do, requiring human approval at important checkpoints and keeping humans accountable.

A useful question is: When should the AI have to ask us first?

I may be happy for an AI assistant to rearrange my calendar or spend $20 replacing office stationery. At $20,000, I would probably want to be asked.

Requiring approval for everything is not the answer either. If someone receives hundreds of AI approval requests every day, they will eventually click “approve” without thinking. We would preserve the appearance of control while turning it into rubber-stamping.

The solution then is to zoom out.

Draw the loop around the system

PM Wong highlighted healthcare and autonomous vehicles as examples of how Singapore is already using or trialling new technologies.

Through Healthier SG, AI is being used to create personalised exercise plans, with patients working with their general practitioners to set health goals. AI is also helping radiologists read mammograms by acting as another pair of trained eyes.

The doctor does not need to approve every calculation made by the AI. Human oversight sits at a higher level: deciding how the AI is used, understanding its limitations, checking when something looks wrong, and retaining responsibility for consequential decisions.

Autonomous vehicles (AVs) make the point even more starkly. A human cannot approve every turn, lane change or braking decision made by an AV.

Instead, human control has to be designed around the system: where the vehicle may operate, under what conditions, how it is tested, and what happens when something goes wrong. The approach to AVs – trialling them in Punggol, proceeding step by step and putting safety first before scaling, illustrates this broader form of oversight.

The goal should not be to place a human around every AI action. It should be to place humans around the system itself, setting the boundaries within which AI can act, testing whether those boundaries remain safe, and changing them when circumstances change.

But boundaries alone are not enough. Systems also need ways to detect when an AI crosses them, contain the consequences and fall back safely. In healthcare, an AI behaving unexpectedly should not keep making recommendations unchecked. It could be taken offline, its cases routed back to clinicians, and its recent decisions reviewed before it is allowed to resume.

The OpenAI-Hugging Face incident is a more extreme version of the same lesson: meaningful human control must include the ability to detect and respond when safeguards fail.

Oversight is a skill

Keeping humans in control also requires people who know how to exercise that control.

A doctor needs enough medical judgment to recognise when an AI recommendation does not make sense. An engineer overseeing an autonomous system needs to understand when it may fail. A civil servant using AI to support decisions needs to know when a case is unusual enough to escalate rather than simply accept the machine’s recommendation.

People also need to understand that AI can be confidently wrong and that it can encourage over-reliance.

The paradox is that as AI becomes more capable, the humans overseeing it may need better judgment, not less.

If AI takes over routine work, human roles may increasingly concentrate on exceptions, ambiguous situations and high-consequence decisions. We therefore need to train people not only to use AI, but to question it and know when to intervene.

That training should be practical. Professionals should review cases where AI is wrong or uncertain, learn the warning signs for escalation, understand their authority to stop a system, and rehearse what to do when it behaves outside expectations.

Who helps draw the boundaries?

AI differs from the technologies we have governed before in that the boundaries cannot be set once and forgotten. The answers change with each model generation, and there is no single “AI” around which to convene experts: the person who understands a hospital triage system knows little about how a frontier model can be manipulated.

Singapore’s National Intelligence Council, chaired by PM Wong, can provide strategic direction.

Its role includes driving Singapore’s national AI agenda and AI missions, and helping coordinate the regulations, resources, research, testing and deployment needed across sectors. But it should be backed by sustained access to people closest to the systems being deployed.

Depending on the problem, that may mean AI researchers, engineers and cybersecurity specialists, but also doctors, transport experts, financial practitioners or front-line public officers.

This is about ensuring policymakers can draw on the right expertise as the technology changes.

Singapore has a particular opportunity here. We have experience building trust around systems, from finance and aviation to healthcare and digital government. We are also small enough to bring policymakers, regulators, researchers and companies together and test safeguards in practice.

The world will increasingly need more than AI builders. It will also need trusted places that can work out where autonomous systems can be relied upon, where humans should remain involved, and how those boundaries should evolve.

Being in the loop was never the wrong idea. The challenge is drawing the loop in the right place and making sure the people in it have the skills and authority to act when it matters.

Roy Ka-Wei Lee is an incoming Canada research chair for human-centred AI and associate professor at the University of British Columbia. Brian Lim is the co-founder and chief executive of Wisma AI.