Amplifier · Cost Efficiency

Skills index, now riding the cache

Prefix placement for skills visibility — full, always-current skill discovery at a fraction of the cost.

Merged · Now the Default · amplifier-bundle-skills PR #48 + #49
July 2026
The Discovery

Every request was carrying a passenger

A systematic audit of what Amplifier injects into every LLM request found the skills-visibility hook re-broadcasting the full skills index on every single provider request — whether the turn needed it or not.

The payload

~3.5k tokens, every request

The complete skills index, re-attached to each provider call so the agent always knows what skills it can load.

The placement

Tail of the message stream

It rides at the end of the messages — after content that changes every turn — so it can never land on a cache hit.

The consequence

A wasted cache write, every turn

The same bytes, paid for fresh on every request, with no chance of ever being served from the prompt cache.

The Measurement
32–39%

of root-session cost on Anthropic models was the skills-index rebroadcast — measured by controlled ablation, with zero quality gain on skill-independent tasks.

$9.69 → $6.59
One task's root-session cost, with vs. without the rebroadcast
0
Quality regression when it was removed on skill-independent tasks
n=3 / cell
Ablation trials per cell, each with wire-level delivery verification
The Design Question

Just remove it? It’s not that simple.

The index exists for a reason: it is how the agent discovers the skills it can load. Every obvious alternative fails a different way.

Option 1

Remove it

Cheapest by far — and the agent goes blind to its skills. Discovery breaks entirely.

Option 2

Inject once

Decays into history and gets evicted by context management. Worse: skills change mid-session — modes add and remove them — so a stale copy is context poison.

Option 3 · the old default

Rebroadcast every turn

Always current, always visible — at 32–39% of root-session cost on Anthropic models.

The Solution

Put it where the cache lives

Move the index into the stable system-prompt prefix — the one part of every request the provider prompt cache is built for.

Rides the prompt cache

The index sits in the cached prefix instead of the uncacheable tail — sent with every request, but no longer paid for fresh each turn.

Always current

Full skills visibility on every request. No decay into history, no eviction, no reliance on a copy from ten turns ago.

Staleness solved structurally

Content re-renders only when the skill catalog actually changes — a mode adds or removes skills, the prefix updates. Exactly one current copy ever exists.

# tool-skills config — now the default (PR #49) visibility: placement: "prefix"
The Proof

Measured three ways

Same tasks, three placements, wire-level verification that each arm ran what it claimed. Root-session cost per task:

Request mode
the old default
$1.30
Task A
$9.69
Task B
Prefix
now the default — full visibility
$0.90
Task A
$8.24
Task B
Disabled
floor — no visibility
$0.79
Task A
$6.59
Task B
Quality identical to the old behavior. Prefix recovers roughly half to three-quarters of the rebroadcast premium — while keeping everything the index provides.
Skill recall, now measured: planted skills with non-guessable tokens — the agent must notice the index entry and load_skill to succeed, graded programmatically. Request arm 6/6 recalled+correct; prefix arm 5/5 (one trial excluded as an infrastructure failure with zero requests).
Sources & Methodology

How these numbers were made

Now the Default

Nothing to configure

Prefix placement is now the default — it auto-applies on your next bundle refresh. Cheaper long sessions, same skills discovery.

# tool-skills config — only if you want the old behavior back visibility: placement: "request" # byte-identical old behavior
github.com/microsoft/amplifier-bundle-skills/pull/48 · pull/49
More Amplifier Stories