Reality-Check Validates Itself

Using the Digital Twin Universe to test the Digital Twin Universe — and everything that rides on it.

Author David Koleczek

Part 1

The Irony.

I built the thing made for testing — and couldn't use it to test my own stuff.

Container-in-container poses challenges.

As I've been working on all of the Digital Twin Universe, Universe bundles — I realized that I'm literally not able to use it for the thing it's made for, which is testing.

DTUs can have their own DTUs.

Now we have examples where the outer DTU hosts an inner one — so the testing infrastructure can test itself, and anything else riding on it.

Certain commands would hang.

Root cause: subprocess.run with capture_output=True was waiting for grandchildren (like lxc monitor spawned by a nested incus launch) to close inherited file descriptors — even after incus exec itself had exited.

The fix — tempfiles instead of pipes.

# incus.py :: exec_command()

- result = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
- return result.returncode, result.stdout, result.stderr

+ with tempfile.TemporaryFile("w+") as out_f, tempfile.TemporaryFile("w+") as err_f:
+     result = subprocess.run(
+         cmd,
+         stdin=subprocess.DEVNULL,
+         stdout=out_f,
+         stderr=err_f,
+         text=True,
+         timeout=timeout,
+     )
+     out_f.seek(0); err_f.seek(0)
+     return result.returncode, out_f.read(), err_f.read()

subprocess.run now returns as soon as incus exec exits, regardless of what grandchildren are still holding inherited fds. stdin=DEVNULL keeps the child from blocking on inherited input.

Part 2

Self-Validating reality-check.

A concrete example — profile → run → evidence → summary.

Start with a profile.

reality-check-in-incus.yaml stands up an outer DTU ready to host an inner one.

Installs

Incus, Docker, and other prereqs. Configures Amplifier and bundles.

Sources

Uses the reality-check bundle from Gitea if provided — local changes flow in without hitting GitHub.

Verifies

Confirms it works with commands like amplifier bundle list.

Run Amplifier inside the outer DTU.

Drive reality-check against an inner app, with stdout/stderr streamed to the terminal and tee'd to a log.

# inside the outer DTU
amplifier run "Can you run the reality-check-pipeline recipe against
              the app at /software with spec and conversation at /user"
  | tee $OUT/run.log

Pull out the evidence. Summarize.

When the run completes (or fails), pull the artifacts out of the DTU. Then let Amplifier read the logs and tell you what happened.

# pulled out of the DTU
/root/home                -> $OUT/home/
/root/.amplifier/projects -> $OUT/sessions/

Then: Amplifier generates a summary of what happened based on the logs.

Part 3

Why This Matters.

Self-validation generalizes — to bug reports, feature requests, anything.

Scripts that automatically validate anything.

You can easily set up scripts that automatically validate anything.

For example: if you get a bug report or feature request — you can start with a DTU profile and have Amplifier write a script that uses the DTU to send commands for automation.

DTUs inside DTUs.
Profile, run, evidence, summary.
Automate the loop.

The thing made for testing now tests itself — and anything else you point it at.

amplifier-bundle-reality-check amplifier-bundle-digital-twin-universe

Author David Koleczek