Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents

A support or data assistant repeats work across related customer requests.

What you would change

Try the paper's memory change in a replay of past tasks, with separate memory per customer.

Try the paper's memory change in a replay of past tasks, with separate memory per customer.

Compare against

Your current memory pipeline plus a no-memory control using the same agent.

1. Locate the paper's implementation and check its license, model support, data needs, and hardware requirements. Stop here if you cannot obtain the required artifacts.
2. Save a small set of representative product tasks and expected outcomes. Keep evaluation tasks out of any training or tuning, and choose quality and cost limits before running.
3. Run the baseline and the proposed change on identical inputs. Keep other settings fixed and repeat paired runs to expose variation.
4. Save per-task outputs, failures, timings, and all usage costs. Review regressions before expanding the test.

What to measure

Task success, stale memories, retrieval misses, response latency, and all memory maintenance costs.

When to keep it

Keep the change only if it meets your preselected quality and latency limits and improves the metric you care about after setup and operating costs. Otherwise retain the baseline. This is an exploration plan, not a validated recommendation.

recommendation

test

research area

memory and context

technical change

Read-time task-adaptive memory curation (JitMem)

published

2026-09-23

The method is implementable on constrained infrastructure using API models and modest storage, but its gains are demonstrated on agentic benchmarks rather than the reader's workload, so adoption should follow a small-scale validation against a simple write-time baseline.

1. claim 01Paperraft inference
   Read the brief interpretation

   Replaces write-time memory distillation, where each completed trajectory is compressed into a fixed reflection or skill artifact retrieved by similarity, with retention of raw trajectories and read-time synthesis of a compact task-adaptive payload conditioned on the current query. Costs include storing full raw trajectories (higher memory and retrieval footprint), an extra curator LLM call at read time adding latency and API spend, and optionally training the curator on task-success signals. Failure modes include curator errors when retrieved traces are noisy or irrelevant, degraded performance if the new task differs from anything in the trace store, and unbounded storage growth without a retention or pruning policy.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source
2. claim 02Paperraft inference

   Outperforms no-memory and write-time memory baselines on ALFWorld, WebShop, and tau2-bench, improving over the strongest baseline by 16.2, 16.3, and 3.9 absolute success-rate points; an untrained curator is already competitive.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source