Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents

Possible product use: a SaaS data assistant that repeatedly looks up the same customer tables, then makes mistakes after a schema change.

What you would change

After a task ends, a separate curator checks proposed memories with read-only tools before saving or revising them. The responding agent and its memory reader stay the same. The added work happens during memory maintenance.

For an existing memory product, start with the comparison against ordinary memory. The no-memory headline includes the benefit of adding memory at all. The extra three percentage points do not establish a reliable product gain, and the reported task-agent bill leaves maintenance out.

Source version reviewed 2026-09-14. No experiment reproduced by Paperraft.

Add a background memory checker to a sandbox copy of that assistant. Keep customer data and memory isolated. Give the checker only the read access needed to validate a record, and keep the current memory pipeline available as a fallback.

Compare against

Run three conditions: no memory, your current memory, and the same memory with checking. Keep the answering model, tools, retrieval, and task order fixed.

1. Prepare 30 representative database questions with verified answers. Change a table name or data convention halfway through the sandbox sequence. Define a spending limit and an acceptable failure rate before testing.
2. Run each condition from a fresh environment and empty memory. Let memory persist within a run, never between conditions. Hide future questions from the checker.
3. Repeat at least three paired runs with matching question orders. Record answering, distillation, checking, and retrieval usage separately. This pilot size is a Paperraft suggestion, not the paper's evaluation protocol.
4. Inspect wrong answers and the memory records that caused them. Save per-task results and check whether new records remain correct after the environment changes.

What to measure

Correct answers, unnecessary database calls, stale records, response latency, and total cost per successful task including failed attempts and all memory maintenance.

When to keep it

Keep checking only if it improves your chosen quality target at an acceptable total cost. If your goal is savings, the reduction in answering cost must exceed the added checking and maintenance cost. Use paired results to inspect variation; three pilot runs are not proof of a stable production gain.

recommendation

test

research area

memory and context

technical change

Environment-probing curation for agent memory

published

2026-09-10

The gains are large and deployment-compatible with API-based agents on constrained infrastructure, but the evidence is confined to one vendor harness and two task families, so the reader should validate on their own workload before production adoption.

1. claim 01source-linked

   On CLBench (40-question drift schedule, GPT-5.4), environment-probed memory raises pass rate and reward over the no-memory baseline while reducing queries and cost.

   result

   Pass 39% to 73%, reward 8.60 to 22.60, queries 8.8 to 4.7/question, task-agent cost $3.38 to $1.68

   baseline

   GHCP (No Memory)

   conditions

   GPT-5.4 task/distiller/curator, CLBench 40-question drift schedule, five paired seeded runs

   Pass rate rises from 39% to 61–73%, while total rewardrisesfrom8.60to20.00–22.60,or2.3–2.6×thebase- line.Thisisnotabrute-forceaccuracygain:queriesfallfrom 8.8 to 3.0–5.6 per question and task-agent cost falls from $3.38 to $1.68–$2.01.

   Open supporting passage on PDF page 4
2. claim 02source-linked

   Environment probing improves over trajectory-only indexed memory on the same CLBench drift schedule, indicating write-time evidence quality rather than task-time capability drives the difference.

   result

   Reward 20.00 to 22.60, queries 5.6 to 4.7, cost $1.99 to $1.68

   baseline

   GHCP + Mem (trajectory-only curation)

   conditions

   GPT-5.4, identical task-time model/tools/memory interface; only curator gains read-only environment tools

   Relative to trajectory-only memory, probing raises drift CLBench re- ward from 20.00 to 22.60, lowers queries from 5.6 to 4.7, and reduces task-agent cost from $1.99 to $1.68.

   Open supporting passage on PDF page 5
3. claim 03source-linked

   On the no-drift 30-question CLBench schedule, probing achieves the highest mean reward on both Sonnet 4.6 and Opus 4.7, exceeding both paired baselines and trajectory-only memory.

   result

   Sonnet 4.6: 0.748 vs 0.673 (Mem) and 0.327 baseline; Opus 4.7: 0.721 vs 0.696 and 0.458 baseline

   baseline

   Paired GHCP (No Memory) and GHCP + Mem per model

   conditions

   Fixed canonical 30-question no-drift order, five paired runs per model-memory comparison

   Sonnet 4.6 GHCP + Mem (w/ Env Probing)0.327±0.0550.748±0.030+0.421

   Open supporting passage on PDF page 5
4. claim 04source-linked

   Across six adapted APEX worlds, all 18 memory-versus-baseline reward comparisons are positive and probing gives the best reward gain per task-agent dollar in five of six worlds.

   result

   All 18 gains positive; probing best reward/$ in 5 of 6 worlds; tool calls fall e.g. world 941eba66 from 71.6 to 19.3

   baseline

   Three-run GHCP (No Memory) per world

   conditions

   GPT-5.4, 90 adapted APEX management-consulting tasks, five runs per stateful configuration

   GHCP + Mem (w/ Env Probing) achieves the best task-agent reward gain per dollar in five of six worlds (Table 2); GHCP + Mem is marginally betterintheremainingworld.

   Open supporting passage on PDF page 5
5. claim 05source-linked

   The probing advantage is heterogeneous: probing adds little or nothing where the trajectory already supports an actionable record, and the authors treat subgroup differences as mechanism interpretation rather than established effects.

   result

   World 941eba66 changes by -0.04 for probing vs trajectory-only memory; intervals overlap

   baseline

   GHCP + Mem

   conditions

   APEX per-world comparison and no-drift cross-model comparison

   while941eba66 changes by−0.04(Table 2). Its additional no-drift gain is alsolargeronSonnet(+0.075)thanonOpus(+0.025).

   Open supporting passage on PDF page 6

1. risk 01Paperraft inference

   All evaluation is on two benchmarks via a GitHub Copilot SDK harness; gains may not transfer to other agent stacks or task distributions.

2. risk 02Paperraft inference

   The improvement over plain memory is modest and confidence intervals overlap in places; in one of six APEX worlds probing was slightly worse, and the authors explicitly treat heterogeneity as unresolved.

3. risk 03Paperraft inference

   Requires a safe, least-privilege read-only environment surface for the curator; without one it degenerates to trajectory-only curation, and granting such access has security/auditing implications.

4. risk 04Paperraft inference

   Curator, distiller, and probe costs are excluded from reported task-agent cost; total system cost including curation is not the headline metric and could offset savings.

5. risk 05Paperraft inference

   Probing adds asynchronous compute after every task; latency and cost of the curation phase at production scale are not quantified in the main results.

6. risk 06Paperraft inference

   Results depend on specific frontier models (GPT-5.4, Sonnet 4.6, Opus 4.7) and benchmark graders; weaker models may not probe effectively.

1. Pick one of your own workloads with a stream of 20-40 related tasks sharing latent environment structure (e.g., recurring database or document corpus), including at least one unannounced drift event, and define a strict pass/fail grader.
2. Stand up two memory pipelines that are identical in task agent, retriever, memory schema, distiller, and CRUD policy; give only one curator a least-privilege read-only subset of the environment tools plus the two probe prompt blocks from Appendix D.3.
3. Run both conditions plus a no-memory baseline on the same seeded task order with a small number of runs (at least 3 paired runs), keeping task agent, model, and prompts fixed.
4. Measure pass rate, task-agent tool calls, and task-agent cost, with curator/probe usage tracked separately; check whether probe curation calls stay read-only and off the task budget.
5. Inspect the committed memory records in both arms: if probed records are not more executable (explicit joins, paths, current schema) than trajectory-only records, the mechanism is not operating as claimed.
6. The reported gain is disproven for your workload if probing does not improve reward or tool-call count over trajectory-only memory, or if probing cost/latency erases the task-agent savings.

• What is the total end-to-end cost including distiller and curator/probe sessions, which the paper says are tracked separately but does not headline?
• How many probe calls does the curator typically make per task, and what governance prevents read-only probes from leaking information or being abused?
• Does the gain persist on non-benchmark, messier enterprise environments where read-only surfaces are incomplete?
• How sensitive are results to the curator prompt and to the embedding retriever (intfloat/e5-base-v2)?
• Would probing help when memory errors come from retrieval failure rather than record quality, which this design does not address?