Topics and themes have an uncanny way of surfacing when you least expect them. I spent ~4 years researching, building and working with digital twins in different settings. Between 2017-18, it was in the context of dynamic RF environments for ultra-reliable machine-to-machine wireless communications during my time at Huawei’s Munich Research Center. Between 2021-22, it was in the context of driver-in-the-loop simulators for teleoperated driving at Imperium Drive. Fast forward five years and I find myself building AI for high-stakes finance and the digital twin is back, albeit in a slightly different setting (as is usually the case).

The problem

Reinforcement learning works when an agent can act, observe consequences and try again thousands of times. Markets deny it all three. You cannot replay a trade against real liquidity, the environment reacts to your own actions, and the rules shift as regimes change.

The usual workaround is a simulator, but a market simulator is only as good as its model of other participants. Get that wrong and the agent learns to exploit the simulator’s flaws rather than the market’s. This is why RL successes in games have not translated cleanly into portfolio construction, execution or risk.

The idea

Instead of modelling the whole market, build a digital twin scoped to one decision: this rebalance, this hedge, this execution. The twin does not need to be true everywhere, only faithful where the decision is sensitive.

Self-play then supplies the missing environment. The decision agent is paired with adversarial and counterparty agents trained on the same historical and live data, each trying to make the decision fail. As the decision agent improves, its opponents get sharper, and the twin becomes more sophisticated with every cycle. New data, including the outcomes of real decisions, is folded back in so the loop keeps evolving rather than converging on a stale picture.

Why it could work in finance

  • Markets are already adversarial. Every trade has a counterparty with opposing incentives, so self-play mirrors the real structure rather than imposing an artificial one.
  • Narrow scope is tractable. A twin of one decision needs far less fidelity than a twin of the market, which sidesteps the modelling problem that stalls classic RL.
  • Stress is generated, not just sampled. Opponents search for the scenarios that break the decision, surfacing tail risks that history alone under-represents.
  • Data compounds. Each decision taken creates new evidence, so the system improves with use rather than decaying between model refreshes.

Open questions and risks

  • Collusion and drift. Self-play agents can co-adapt into strategies that beat each other but not the real market. What anchors them to reality?
  • Validation. How do you prove the twin is faithful enough for a high-stakes decision, and to whom: the PM, risk, the regulator?
  • Reflexivity. If many firms run similar loops, the twins start modelling each other, and the market itself changes.
  • Explainability. A decision shaped by an evolving adversarial process must still be defensible to an investment committee.

Themes to develop next

  1. Decision-targeted versus world-modelling twins: where the line sits and why it matters.
  2. The analogy to AlphaGo and poker solvers, and where markets break it (non-stationarity, partial observability, no fixed rules).
  3. Human expertise as an opponent: using PMs and traders to seed or challenge the adversarial agents.
  4. A worked example, such as a fixed income rebalance or a block trade execution.
  5. What this implies for how institutions should build and govern AI for decisions, not just for analysis.