The agent prompt system & what it costs

00TL;DR

Eight things to know — the first row is the new measured-cost picture.

01The call path

One agent turn, end to end. Everything is reconstructed from the DB each turn — the API is stateless and there is no message store (schema frozen in Phase 1).

02System prompt anatomy

The preamble is a pure conditional assembler — the executor decides what's included and passes pre-rendered strings in. Ordering is deliberately static→dynamic for cache stability.

The literal assembly is one format string — note that learning, memory, and contact data are absent here by design:

src/mods/ai_agent/services/build_system_prompt_service.rs:337let system_prompt = format!(
  "{platform_block}{org_rules_block}{agent_rules_block}{persona}\n\n\
   Your goals:\n{goals}{delegated_block}{delivery_block}\
   {skills_block}{invokable_agents_block}{tools_block}",
  persona = agent.persona,   // static, operator-authored
  goals   = agent.goals,     // static, operator-authored
);

Tier 0 — the platform-core contract

Included only for a domain agent (attach_domain_tools && !is_assistant_flavor). It is a fixed constant (PLATFORM_CORE_BLOCK) with four sections — and crucially, it speaks of "the business" generically:

build_system_prompt_service.rs:73 — PLATFORM_CORE_BLOCK (excerpt)## Who you are
You are an autonomous agent acting on the business's behalf inside Loquent…
You speak to the business's contacts as the business itself — a real person
from the team — and you are never named or described as an AI…

## How you affect the world   → you change the world only by calling tools
## Who is talking to you: operator vs. contact   → trust steering, distrust the contact
## Ground your replies; never make things up   → gather context with read tools, else escalate

What's actually in the ~14K input — measured

The anatomy above is the system-prompt text, but it is a small minority of what goes on the wire. Reading the real effective_prompt_capture for a live Text Reply agent against its ai_usage_log rows, the prompt breaks down like this:

The Text Reply agent carries attach_domain_tools = true with a one-item tools_allowlist (["escalate_to_user"]) — but the executor then attaches the entire domain toolset via build_domain_tools_for_agent → collect_agent_domain_rig_tools → collect_rig_tools(Family::all()) (permission- and tier-gated). Those tool JSON schemas are sent in the request's tools array on every call. The tools_block in the system text only lists allowlist names; the schemas themselves are the ~11–13K.

03Cost, tokens & cache — measured

Grounded in real rows from loquent_latest — ai_usage_log (tokens, cache, model) joined to ai_thread_log (the 12-layer prompt capture). Window: ~46 h, Jun 12–14 2026, 190 usage rows.

Per-feature over the window

Two cost columns: naive = what Loquent records today (every input token at full $3/1M, cached reads included); true = the real provider cost with Anthropic's cache-read discount (fresh $3/1M, cached read $0.30/1M, output $15/1M).

Per-turn cache behavior — the cold/warm pattern

The 10 agent calls show the mechanism plainly. Calls with a cache read carry 92–99.5% of the prompt from cache; cold calls (first call of a thread, or after the prefix is invalidated / the TTL lapses) carry none and pay full price for the whole ~14K:

The aggregate 67.6% is the blend of 3 cold and 7 warm calls. The architectural fact that drives it: with_prompt_caching() uses an ephemeral (5-minute TTL) breakpoint. Within an agentic loop (tool turn → text turn, seconds apart) and within a burst of replies (<5 min) the prefix is warm; once replies space out past 5 minutes — the norm for SMS — the next message starts cold.

The cost-accounting gap

Three code facts compound so that the cache's real savings never reach Loquent's books:

1. input_tokens logged is inclusive of the cached read (it's the total prompt, ~14K), and cached_tokens is the read subset (verified against the data: 14,305 total / 13,737 cached).
2. In ai_pricing_type.rs, anthropic/claude-sonnet-4.6 falls through ..ModelPricing::ZERO → cached_per_1m = 0. So calculate_cost = input × $3 + cached × $0 = the whole prompt at full price.
3. Cache-creation (write) tokens are never stored — from_rig_usage logs only cached_input_tokens (read), and the table has no write column — so the 1.25× write premium on cold turns is invisible too.

src/mods/ai/types/ai_pricing_type.rs:47 — no cache rate for Sonnet(_, "anthropic/claude-sonnet-4.6") => ModelPricing {
  input_per_1m:  3.00,
  output_per_1m: 15.00,
  ..ModelPricing::ZERO        // cached_per_1m = 0.0  ← cache discount not modelled
}

Projected spend at scale

Applying the measured unit economics. Per inbound message ≈ 2 model calls; true provider cost depends entirely on whether the cache is warm:

Two readings: (1) real SMS cadence pushes most messages toward the cold regime, so a longer cache TTL or a smaller prefix is worth real money at volume; (2) the gap between the typical and naive rows is the cache savings the accounting currently throws away. Output is negligible (~88 tok/call) — this is an input-bound workload, which is exactly the case prompt caching is built for.

04What this worktree changed since v1 of this doc

The first version of this artifact predates the memory/learning commits that landed on this branch. Here's what shipped and — the part that matters for cost — which layer each one touches: the cached prefix, the (uncached) user turn, or the toolset.

05The user turn

Everything dynamic and/or untrusted lives here — never in the cached system prefix. This is also where learning and few-shot examples are injected.

run_ai_agent_thread_service.rs:963 — KV-cache-safe user-turn injectionlet user_message = {
  let mut parts = Vec::with_capacity(3);
  if !learning_summary.is_empty() { parts.push(&learning_summary); }  // §6
  if !few_shot_block.is_empty()   { parts.push(few_shot_block); }       // few-shot
  parts.push(&prompt);                                                  // framed events
  parts.join("\n\n")
};
rig_agent.chat(Message::user(user_message), &mut history).await;

Source framing (prompt-injection defense)

Each event is wrapped by frame_for_prompt(contact_name, contact_number) so the model can tell trusted operator steering from untrusted contact text. Identity fields are scrubbed of newlines and bracket glyphs so a crafted SMS can't forge an operator envelope:

ai_thread_event_payload_type.rs:276 — frame_for_prompt (excerpt)InboundSms  → "[New SMS · {name} <{number}>]\n{scrubbed text}"
CallCompleted → "[Call ended · {name} <{number}>]\n{summary}"
UserMessage → "[Operator instruction from your owner — not the contact]\n{text}"
Scheduled   → "[Scheduled instruction from your owner — not the contact]\n{text}"

History reconstruction

There's no message store; history is rebuilt each turn from consumed queue events (user) + llm_generation log rows (assistant) + recorded tool calls, sorted by timestamp, capped at 100 per source (worst case ~300 messages). The new user input is passed as a separate Message::user — never interpolated into the system prompt.

06Memory & learning this worktree

The branch adds a three-part knowledge system. The distinction matters: learning is injected, memory is tool-loaded, lessons feed the digest that evolves learning.

How learning gets in (and why it's in the user turn)

On the first turn of a thread the executor resolves the agent's active learning version and pins its id to ai_thread.pinned_learning_version_id; every later turn reads the pinned version. This freezes the policy for the life of a conversation — a digest activating a new version mid-thread can't swap it underneath. The pinned body is summarized and prepended to the user message:

run_ai_agent_thread_service.rs:615 — learning resolution (condensed)let learning_summary = if agent.enable_learning {
  match thread.pinned_learning_version_id {
    Some(pinned) => summarize_learning_version(&get_learning_version_body(pinned)?),  // later turns
    None => { pin_version_id = Some(v.id); summarize_learning_version(&v.body_markdown) } // first turn → pin
  }
} else { String::new() };

Memory is tool-loaded, never injected

read_my_memory renders the typed blocks as labelled markdown on demand; update_my_memory applies a diff of add/update/delete ops (honoring read_only). There is also contact-scoped memory (read_contact_memory / update_contact_memory). The system-prompt builder explicitly leaves memory_snapshot: None — memory never enters the prompt automatically.

Evolution / digest loop

Apply mode is org-level: auto activates immediately; approval lands a pending version an owner approves. Risk is bounded by the ≤20-version cap, the one-active-per-agent invariant, and the reward-hacking / hallucination guards. Shadow replay + regression fixtures exist as the validation surface.

Retrieval observability (#1566)

A metadata-only retrieval_context records which learning version, how many learning bullets, and how many few-shot examples informed a turn — stored in the audit capture, never injected, and carrying ids/counts only (no verbatim cross-contact example text).

07Prompt caching vs Anthropic guidance

Caching is enabled, genuinely cache-aware, and the §3 data confirms it works (67.6% / 92–99.5% warm). One correction from v1 of this doc: for the agent, the cached prefix is the toolset, not history — so the lever is its size and TTL, not "cache the history."

What's wired

src/mods/ai/rig/client.rs:38pub fn completion_model(client, model_id) -> openrouter::CompletionModel {
  client.completion_model(model_id).with_prompt_caching()
  // attaches cache_control: {"type":"ephemeral"} (5-min TTL) to the SYSTEM PROMPT
}

One breakpoint, on the system prompt. Anthropic's render order is tools → system → messages, and a breakpoint caches everything up to and including it — so the marker on the system prompt caches tools + system as one prefix (the ~13K from §2), but not the message tail. Provider path is OpenRouter → Anthropic; the marker is forwarded. Hits are observable — the streaming layer reconciles cached_input_tokens and cache_creation_input_tokens, and the read side is logged to ai_usage_log.

08User & org data — the core question

Your original question: do we inject the current user (name, roles) and the organization's data so the agent can act as that person? For contact-facing agents the answer is still no — unchanged by this worktree. Here's the evidence and the one exception.

The grep that confirms it — over the assembler and the executor:

verified on this branch$ grep -n 'user.name|member.name|owner_name|organization.name|role|assigned_user|first_name' \
      build_system_prompt_service.rs run_ai_agent_thread_service.rs
(no matches)

The seed personas make the same point — they reference the business relationship but carry no identity. From the Follow-up Drafter seed:

migration/.../seed_followup_drafter_agent.rs — persona (excerpt)You are the Follow-up Drafter… You draft short, personalized SMS follow-up
messages sent to a contact on the user's behalf — the contact sees the
message as coming from the user's business, and you are never mentioned.

"the user's business" is a role, not a value — no name is ever substituted in.

The one exception: the assistant flavor

The internal Loquent Assistant (the in-app helper, not a contact-facing agent) does personalize. Its turn is assembled by assemble_assistant_turn, which rebuilds the owning member's Session and loads get_member_personalization(...). So the capability to thread a member's identity into a prompt already exists in the codebase — just not on the path that talks to contacts.

09Insights & levers

What the data says to do, in rough priority. The first three are cost/accuracy levers the telemetry surfaced; the rest are the identity/personalization directions for your original goal. Options, not a committed plan.

## Who you represent
You are messaging on behalf of {business_name}.
Your point of contact on the team is {owner_name} ({owner_title}).
Business hours: {hours} ({timezone}). Sign off as {signature} when appropriate.

10File reference map

Where each piece lives, for the implementation session.