Evidence, Not Echoes

Real-enough worlds on demand from one YAML profile

The Gap

"Tests pass" isn't proof the software actually works

AI-generated software is verified in the very environment and context it was built in — so agents can claim success without ever confronting deployment reality.

The Digital Twin Universe README names this gap itself. So what does it take to close it?

"This leaves a gap between ‘tests pass’ and ‘this actually meets all the requirements and works in a real-world scenario.’" — DTU README, lines 3–6 (microsoft/amplifier-bundle-digital-twin-universe)

The Answer Is Real

The Digital Twin Universe ships today, and it's heavily tested

This isn't a sketch. The engine is ~4,397 lines of Python across 7 modules, guarded by hundreds of tests spanning unit, integration, e2e and bugfix suites.

A mature answer exists — but why was the previous answer not enough?

65

commits, 24 merged PRs

~4,397

lines of Python across 7 modules

262

test functions

33

test files

The Incumbent

The prior answer was an Amplifier-only, command-only overlay

The Amplifier shadow environment described itself as "Amplifier ecosystem-specific." It was driven only through shadow.create(local_sources=...) and shadow.exec — a local-source overlay.

You could run commands. But you could not see the world the code would live in.

Amplifier-specific"Amplifier ecosystem-specific shadow environment support"
CLI / exec onlyshadow.create + shadow.exec, command execution
A local-source overlayno browser access, no mock services, no DNS rerouting

Echoes, Not Evidence

An overlay that only runs commands can't mirror the real world

If all you can do is execute commands over a local-source overlay, you still can't watch the software behave the way a real user would experience it. Green tests stay echoes — not evidence.

So the Digital Twin Universe rebuilt the environment from the ground up.

Real Isolation

One YAML profile stands up a genuinely isolated world

A DTU is declared entirely in a single declarative profile. It runs on Incus containers plus Docker sidecars — not a thin overlay — with incus referenced 76 times in the engine and a dedicated 869-line wrapper.

Isolation is real. But is the world inside it believable?

1

declarative YAML profile per environment

7

example profiles ship with the repo

76

incus references in engine.py

869

lines in the Incus wrapper (incus.py)

Real Enough

Real APIs pass through; mitmproxy reroutes the rest transparently

Stateless external APIs like LLM endpoints are proxied through with forwarded credentials rather than mocked. For everything else, mitmproxy reroutes the real domains — WebSocket upgrades included — so code inside needs no changes.

The world is believable. Now can a human actually use it?

Passthrough for real APIspassthrough.allow_external forwards credentials (e.g. ANTHROPIC_API_KEY)
Transparent reroutingmitmproxy routes declared domains, WebSocket upgrades included (31 references)
No code changes insidethe code sees the real hostnames it expects

Not Headless

A human opens a browser and uses the app for real

Profiles declare access.ports to forward a container port to localhost via Incus proxy devices — so you witness the app working, not just a passing test. The amplifier-chat UI opens at http://localhost:8410/chat/.

Tests-green becomes demonstrable evidence. The repo then makes that binding.

launchstand up the isolated DTU from the profile
access.portsforward container 8410 → localhost 8410 (works on WSL2)
open a browseruse the Chat UI at http://localhost:8410/chat/ as a real user

The Payoff

Enforced discipline

"Evidence, not echoes" is a binding review rule, not a slogan

AGENTS.md tells reviewers to expect "evidence in the PR body, not just tests pass," backed by a verification gradient that requires launching a live throwaway DTU for any engine or incus change.

Validate against the world the code will actually live in. Ship evidence, not echoes.

"The reviewer expects evidence in the PR body, not just ‘tests pass.’" — AGENTS.md, line 36, added via #21 (2026-05-18); paired with a verification gradient requiring a live throwaway DTU launch for engine/incus changes.

Sources

Research Methodology

Repository: microsoft/amplifier-bundle-digital-twin-universe (origin resolves to microsoft/; robotdad/ fork also exists). HEAD 2087575, 2026-06-18.

Feature status: SHIPPED — bundle.md 0.3.1; pyproject.toml 0.3.0; last release v0.3.0 (2026-05-15). Timeline: first import 2026-03-25 → latest commit 2026-06-18.

Commands run:

git log --oneline | wc -l → 65 commits; grep -c '(#' → 24 merged PRs; git branch -a → single main branch
wc -l src/amplifier_bundle_digital_twin_universe/*.py → 4,397 total (engine.py 1,784; profile.py 968; incus.py 869)
find tests -name 'test_*.py' | wc -l → 33; grep -rh 'def test_' tests | wc -l → 262
grep -ni 'incus' src/.../engine.py | wc -l → 76; grep -rni 'mitmproxy' src profiles docs | wc -l → 31
find profiles -name '*.yaml' | wc -l → 7; sed -n reads of README.md, AGENTS.md, and amplifier-chat.yaml / amplifier-user-sim.yaml
Shadow-env contrast: cat amplifier-foundation/behaviors/shadow-amplifier.yaml + head amplifier-foundation/context/amplifier-shadow-tests.md
git shortlog -sne --all for contributor attribution

Gaps: PR authorship/state (#14–#26) derived from squash-merge commit subjects, not the GitHub PR API. The "Mock service catalog" of pre-built images is marked TBD in the README. The shadow-env contrast is grounded in amplifier-foundation, not the DTU repo. macOS/colima localhost access is a known issue with a documented workaround.

Primary contributors: David Koleczek (DavidKoleczek) — 47 commits (project lead / original author); Brian Krabach — 13 commits (recursive file ops, incus timeout config, AGENTS.md verification discipline).