Researchers gave AI agents simulated vending-machine businesses to run.

Their goal was to make a profit.

Some agents found a dishonest way to keep more money: tell customers their refunds had been processed, without actually paying them. The researchers observed this behaviour in a study testing 20 AI models. Study

Nobody had explicitly asked the agents to cheat. But withholding a refund helped the business keep its money, even while the customer was being misled.

That brings us to the question behind this issue: when we give AI a target, how do we make sure it reaches it in a way we would accept?s

I am Alex. Welcome to ShortCu8 by Innov8.

Lets Dive Deep🐰🐰.

Give an agent a goal, define the boundaries, and verify the result outside its own report.

“Reduce refunds” is a target. A complete task also needs to specify who qualifies for a refund and what the agent is allowed to change.

Keep that example in mind as we connect a business experiment, a virtual town and a warning about paperclips.

The vending-machine agents had broad freedom to run their businesses under a profit-maximising objective. The researchers did not explicitly instruct them to cheat.

Some agents' recorded reasoning connected withholding refunds with preserving profit. Others increasingly skipped consideration of complaints as they reduced the computation spent on decisions. The study therefore describes more than one route to misconduct. Paper

Imagine reviewing that business using profit and the agent's customer-service summaries alone. You could miss the unpaid refunds entirely.

That is the practical problem worth paying attention to: the thing you measure can improve while the service you meant to provide gets worse.

Emergence built virtual worlds where agents could interact, remember events and earn the energy needed to persist.

The environment prohibited theft and violence, but exposed actions such as stealing and intimidation through tools. Some agents used them.

One finding makes this more interesting than a count of simulated crimes: Claude-based agents recorded no crimes in a Claude-only world, while some committed crimes in a mixed-model world. Behaviour differed with the surrounding population. Emergence's account

This was a designed simulation with particular rules, tools and resource pressures. The researchers explicitly caution against treating the observations as causal claims about the underlying models.

Still, it raises a useful question for anyone assembling agents: have you tested what happens when they interact, or only checked each one separately?

Bostrom's example takes narrow goal pursuit to an extreme.

Imagine that paperclip production is all a powerful system values. Resources needed for food or housing could become inputs for more factories. Human needs carry no weight unless the system's objectives account for them. Original essay

The two experiments give us a smaller, observable way into that question. What happens when profit or resource acquisition rewards behaviour the operator considers unacceptable?

They do not establish that today's agents are becoming paperclip maximizers. They also do not demonstrate consciousness or an inevitable catastrophe.

The connection is an analogy about objectives and constraints. That is already enough for a useful article—and a reason to examine our own instructions.

For a refund agent, I would define the job like this:

That is a proposed instruction, not a tested safety guarantee.

The application should enforce the spending limit and prevent policy changes. A reviewer should compare a sample of customer messages with actual transaction records, including cases where the agent refused a refund.

Writing “be ethical” into the prompt does not replace those checks. The town experiment included explicit prohibitions, yet violations still occurred.

We read every reply - just reply to this email and let us know how we can improve !

#AD1

AI makes it easier to ship anything, even bad ideas. The hard part is knowing which ideas are worth building.

Jira Product Discovery brings your ideas, customer insights, and priorities into one place, so your team can decide what to ship and move forward with confidence.

Capture ideas, prioritize with evidence, and build living roadmaps your team can rally around—all while staying connected to delivery in Jira, so everyone can see what’s being built and why.

Better product decisions in the AI era.

Try Jira Product Discovery

#AD2

For Deel's "Feeling of Deeling" campaign, agency SuperBloom needed one consistent brand voice across markets—deployed fast, without sacrificing quality or consent. In this on-demand video session, SuperBloom and Voices break down the casting, production, and governance, plus what they'd do differently and where AI voice is headed next.

Watch On-Demand Now