You're Not the Bottleneck

Automatic context compaction in Amplifier

The problem

As history grows, someone has to prune it — that someone shouldn't be you

As an agent works, its conversation history keeps growing. Left unmanaged, a human ends up pruning context by hand, or the session crashes on a cryptic token-overflow API error. Either way the person becomes the bottleneck.

Amplifier's answer: make managing the budget the agent's job, not yours.

History grows every turn — unbounded by default
Manual pruning puts the human in the loop
Overflow crashes the session with a cryptic API error

Already the default

This isn't a proposal — it's the reference context manager, shipped

SimpleContextManager is Amplifier's reference and default context manager (1390 lines in one module). And its compaction was designed to happen before overflow, not after — proactively, so sessions don't crash.

So how does it keep the budget under control on its own?

1390

lines in amplifier_module_context_simple/__init__.py

Default

reference & default context manager (per README)

2eb7729

"proactive compaction and overflow protection" — Oct 24 2025

How it works

Cross 92% of the budget, and it compacts down to ~50% automatically

It watches the token budget — provider context_window, minus a 50% output reserve, minus a 4096-token safety margin. When usage crosses the threshold, it compacts to restore headroom so work continues.

But automatic compaction is only safe if nothing is actually lost.

92%

compact_threshold default — trigger compaction

50%

target_usage default — compact down to this

4096

token safety_margin (raised from 1000 in PR #7)

Non-destructive by design

The model sees a compacted view — the full history is never lost

Compaction is ephemeral: get_messages_for_request() returns a compacted VIEW without modifying stored messages, while get_messages() always returns the full, uncompacted history for transcripts and replay.

Trustworthy automation — which is exactly why silent failure is so dangerous.

1

Ephemeral view — the LLM request gets a compacted list; stored history is untouched

2

8 progressive levels — prefer truncation (preserves structure) over removal (loses context); stop at target

3

Tool pairs stay atomic — no orphaned tool_result that would break the Anthropic API

Then it quietly broke

On 1M-token models the configured max_tokens was ignored — compaction never fired

Whenever a provider advertised a context window, context-simple ignored the configured max_tokens. On large-context (e.g. 1M-token) models, compaction effectively never triggered — and context grew unbounded again.

The automation stopped, and the human quietly became the bottleneck again.

# On a 1M-token model, before the fix:
provider_budget = very large
max_tokens      = configured  # ...but ignored

usage >= 92%?  # budget never
                # gets close
-> compaction never triggers
-> context grows unbounded

The fix (HEAD, Jun 28 2026)

Honor max_tokens as a budget ceiling: min(provider_budget, max_tokens)

Commit 020bd8c makes an explicitly-configured max_tokens cap the provider-derived budget. The effective budget becomes min(provider_budget, max_tokens) — so the budget is honest even on huge context windows.

Cap the budget, and compaction starts triggering again.

# 020bd8c "honor explicitly-configured
# max_tokens as a budget ceiling"

budget = min(provider_budget, max_tokens)

# now usage climbs to the ceiling,
usage >= 92%  -> compaction fires

Shipped & default — merged to HEAD

Compaction fires again on big-context models — the agent, not you, owns the token budget.

The pattern

Give the agent an honest budget and a non-destructive view

Make context management the agent's job: watch an honest budget, compact automatically before overflow, and keep the full history intact behind an ephemeral view. Do that, and the human stops being the bottleneck.

Automation you can trust only works when the budget it enforces is real.

Honest budget — honor the configured ceiling, not just the provider's window
Compact before overflow — proactively at 92%, down to ~50%
Non-destructive — ephemeral view; full history preserved

Sources

Research Methodology

Data as of: HEAD commit 020bd8c, June 28 2026

Feature status: Shipped & default (reference context manager)

Repos: microsoft/amplifier-module-context-simple (HEAD 020bd8c); ramparte/amplifier-core

Research performed:

wc -l amplifier_module_context_simple/__init__.py → 1390 lines
grep -n 'compact_threshold' __init__.py → default 0.92 (line 55/75)
grep -n 'target_usage' __init__.py → default 0.50 (line 56/76)
grep -n 'safety_margin' __init__.py → 4096 (line 1338); git show e702dcc (1000→4096, PR #7)
git show 1ac0631 → ephemeral non-destructive compaction; grep -n interfaces.py lines 170/192
git show 020bd8c (HEAD) → honor max_tokens as a budget ceiling; git show 2eb7729 → proactive compaction
sed -n '104,125p' __init__.py → 8 progressive levels; uv run pytest tests/ -q → 44 passed / 1 failed

Gaps: No benchmark numbers (latency, tokens-saved, session-length) exist in code or commits; every metric shown is a config default, constant, or commit fact — not a measured performance result. Test suite currently reports 44 passed / 1 failed (test_tool_result_truncation_phase1); CI status on the canonical remote was not queried.

Primary contributor: Brian Krabach (50 commits); HEAD fix 020bd8c by Sam Schillace.