Amplifier Foundation Agent

Smoke Test Agent

Objective validation for shadow environments.
Pass means your changes work. Fail means stop and investigate.

Active microsoft/amplifier-foundation

February 2026

The Problem

Can I trust my shadow?

🐳

Isolated Containers

Shadow environments run your local Amplifier changes in Docker containers. But are your local source snapshots actually being used?

🔀

Git URL Rewriting

Shadows redirect git fetches to local snapshots via URL rewriting. If the rewriting fails silently, you're testing the wrong code.

🏗️

Multi-Repo Complexity

Changes span amplifier-core, amplifier-foundation, modules. Each repo needs its snapshot verified independently.

Shadow-operator creates the environment. But who verifies it actually works? You need an independent second opinion.

The Solution

An Independent
Validation Agent

A single 290-line agent in amplifier-foundation that runs separately from shadow-operator to provide unbiased verification.

290

Lines of Markdown

100

Point Rubric

15

Individual Checks

Architecture

The 100-Point Rubric

Five categories, each weighted by criticality. Score ≥ 75 to pass.

Category	Points	Checks	What It Validates
Source Verification	25	4	Snapshot commits exist, git URL rewriting configured, installed package uses snapshot
Installation Health	20	3	Package installs clean, imports work, CLI responds
Code Execution	30	3	Changed modules load, basic functionality works, integration test passes
Isolation Integrity	15	3	Container hostname differs, host home not accessible, env vars correct
No Regressions	10	2	Standard imports work, basic operations complete

Critical failures override the score. If Source Verification, Code Execution, or Installation Health scores zero, the verdict is FAIL regardless of total points.

Key Design Decision

Halt, Don't Work Around

The agent's most important design principle: if it can't do its job, it stops immediately and tells you why.

🛑

Mandatory Halt Conditions

No shadow_id provided. Shadow not found. Container not running. Shadow tool unavailable. Any of these = immediate halt with structured error.

🚫

What It Must Never Do

Never test on the host as a "workaround." Never skip shadow-dependent checks. Never give PASS if it couldn't actually run tests.

🔍

Independent Verification

Runs separately from shadow-operator. Doesn't trust claims about the environment. Verifies everything with direct evidence.

📋

Structured Verdicts

Every check produces evidence. The final verdict includes scores per category plus specific pass/fail evidence for each of 15 checks.

Amplifier-Specific Tests

What It Actually Validates

Tailored tests for each Amplifier repo type.

# Testing amplifier-core changes
shadow exec <id> "python -c 'from amplifier_core import Session, Coordinator; print("OK")'"

# Testing amplifier-foundation changes
shadow exec <id> "python -c 'from amplifier_foundation import load_bundle; print("OK")'"

# Testing CLI changes
shadow exec <id> "amplifier --version"
shadow exec <id> "amplifier provider install anthropic -q"

# Testing module changes
shadow exec <id> "python -c 'from amplifier_module_tool_filesystem import mount; print("OK")'"

Results

What a Verdict Looks Like

+================================================================+ | AMPLIFIER SHADOW SMOKE TEST | | Shadow ID: shadow_abc123 | | Local Sources: amplifier-core, amplifier-foundation | | Tested: 2026-02-20T11:00:00Z | +================================================================+ ## Source Verification (25/25) - [✓] Snapshot commits exist - [✓] Git URL rewriting configured - [✓] Installed package uses snapshot - [✓] Unregistered repos NOT redirected ## Installation Health (20/20) - [✓] Package installs without errors - [✓] Package imports successfully - [✓] CLI tools respond Total Score: 100/100 | Pass Threshold: 75 VERDICT: PASS

Ecosystem

Part of the Shadow Stack

Three components work together to provide safe, isolated testing of multi-repo Amplifier changes.

🏗️

shadow-operator

Creates and manages shadow environments. Snapshots local repos, builds Docker containers, sets up git URL rewriting. Lives in the shadow bundle.

🔬

amplifier-smoke-test

Independent validation agent. Runs the 100-point rubric against the shadow. Lives in amplifier-foundation. This deck.

🧪

shadow-smoke-test

Generic shadow validation (466 lines). Tests container health without Amplifier-specific knowledge. Lives in the shadow bundle.

Activated via behavior: The shadow-amplifier.yaml behavior in amplifier-foundation includes the shadow bundle and adds this Amplifier-specific smoke test agent.

Development Velocity

6

Commits

1

Repository

19

Days (Jan 16 – Feb 4)

1

Contributor

Primary contributor: Brian Krabach (6 of 6 commits, 100%)

Repository: microsoft/amplifier-foundation

Sources

Research Methodology

Data as of: February 20, 2026

Feature status: Active (in main branch of amplifier-foundation)

Research performed:

File discovery: find ~/dev/ANext/amplifier-foundation -name "*smoke*" (1 agent file found)
Line count: wc -l amplifier-smoke-test.md (290 lines)
Shadow bundle cache: find ~/.amplifier/cache -path "*shadow*" -name "*smoke*" (3 related files in shadow bundle)
Git log: git log --all --oneline --grep="smoke" (6 commits found)
Contributors: git log --all --format="%an" --grep="smoke" | sort | uniq -c (1 contributor: Brian Krabach, 6 commits)
Commit date range: Jan 16, 2026 to Feb 4, 2026 (19 calendar days)
Structure check: cat agents/amplifier-smoke-test.md confirmed agent markdown, not YAML recipe
Behavior file: cat behaviors/shadow-amplifier.yaml confirmed inclusion via foundation:amplifier-smoke-test

Gaps and corrections from prior deck:

Prior deck called this a "bundle" with "700 lines of YAML" — it is a 290-line agent markdown file
Prior deck claimed "12 test phases" — actual: 5 rubric sections, 15 individual checks, 100-point scale
Prior deck's mock output listed 11 tests but summary said "12 passed" (off-by-one)
Prior deck referenced samueljklee/amplifier-bundle-smoke-test — not the actual location
Prior deck described general Amplifier health checking — actual purpose is shadow environment validation

Primary contributor: Brian Krabach (100% of smoke-related commits)

Get Started

Try It

Available as part of amplifier-foundation's shadow-amplifier behavior.

# Enable the shadow-amplifier behavior in your config
# This includes the shadow bundle + Amplifier smoke test agent

# Then after creating a shadow environment:
"Validate that the shadow environment is using my local amplifier-core"

# The agent runs the 100-point rubric and returns PASS/FAIL

github.com/microsoft/amplifier-foundation

Open Source Part of Foundation