AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories

Possible product use: an assistant that reads customer uploads or fetched web pages before calling your APIs.

What you would change

AgentDrift labels individual steps of synthetic tool-use histories. It supplies evaluation data, not a proven protective layer. Some examples contain attacks; others contain suspicious-looking but legitimate content.

Treat this as an evaluation resource. The lookup result exposes a shortcut, while recall measures attacks caught by one simple detector. Those are different questions. One weak detector also does not prove that every non-sequential approach must fail.

Source version reviewed 2026-09-14. No experiment reproduced by Paperraft.

Use the dataset to challenge your existing checks offline. Keep tools mocked so that a test cannot send messages, alter records, or expose real customer data.

Compare against

Your current detector against a small human-reviewed set. Separate real attack success, resisted attacks, and legitimate content that looks suspicious.

1. Fetch the official dataset linked by the paper, record its version and license, and inspect the label definitions. Start with a small sample of attacked and legitimate histories that resemble your workflow.
2. Split by world identity so the same world cannot appear in training and evaluation. Keep product test examples separate from any detector tuning. Check for repeated templates and names.
3. Replay the held-out cases through your existing checks with mocked tools. Record the exact step flagged, missed successful attacks, and legitimate actions blocked.
4. Only then compare a candidate detection change on the same cases. Manually review disagreements and repeat with held-out examples from your own agent before enabling enforcement.

What to measure

Attack recall, false-positive rate on legitimate tasks, delay until detection, review burden, and detector cost. Keep denominators and attack types visible instead of reporting one overall score.

When to keep it

Keep a change only if it catches more consequential attacks within your preselected false-alarm and latency limits. Synthetic benchmark performance alone is not evidence that a production agent is protected.

recommendation

test

research area

safety and guardrails

technical change

Step-labeled injection-hijack trajectory benchmark (AgentDrift)

published

2026-09-07

The corpus is a free, API-compatible resource for evaluating injection defenses on constrained infrastructure, but its documented synthetic-data limitations and lack of a trained detector mean it must be validated against the reader's own agent workloads before informing production guardrails.

equations from arXiv HTML · fetched 2026-09-27

1. claim 01source-linked

   The corpus contains 12,536 trajectories and 71,024 step labels across four categories and five domains, with all labels conforming to a stated regular grammar.

   result

   12,536 trajectories; 71,024 steps; 4,000 benign / 5,536 attacked / 1,500 failed-attack / 1,500 hard-negative; 100% positional grammar adherence

   baseline

   Prior resources (InjecAgent, AgentDojo, TraceSafe, StepShield, etc.) lack the combination of per-step injection labels, a resisted class, and hard negatives (Table I)

   conditions

   Synthetic data from a single generator model; labels are protocol-specified, not independent human gold annotations

   We present AgentDrift, a benchmark of 12,536 synthetic tool-call trajectories over five agent domains in which every one of the 71,024 steps carries one of four labels: benign, injection point, hijacked, or failed injection.

   Open supporting passage on PDF page 1
2. claim 02Paperraft inference

   A surface-feature logistic regression baseline recovers only 55.4% of attacks, establishing that the benchmark cannot be solved without sequence modeling.

   result

   Precision 0.778, recall 0.554, F1 0.647, AUROC 0.734; recall 88.0% full hijack, 8.2% partial hijack, 23.1% delayed execution; failed attacks flagged at 0.187 vs 0.105 benign

   baseline

   Six-dimensional hand-crafted feature vector with logistic regression (the paper's own baseline, not an external method)

   conditions

   Evaluated once on the released 1,881-trajectory test split; baseline has access to world-grounded external-recipient features

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source
3. claim 03source-linked

   World-identity leakage lets a trivial lookup predictor reach 86.1% binary accuracy on the default split, so uncontrolled results on this corpus are not evidence of injection detection.

   result

   86.1% binary accuracy overall (93.3%–98.2% in four domains) vs 55.8% majority-class rate; 47.7% in email

   baseline

   Majority-class baseline (55.8%)

   conditions

   Lookup of identical world object in training split; caused by concurrent SLURM jobs creating separate world pools per category in four domains

   The predictor reaches 86.1% binary accuracy on the test split against a 55.8% majority-class rate, and between 93.3% and 98.2% in banking, web, coding, and medical

   Open supporting passage on PDF page 13
4. claim 04source-linked

   An LLM judge is an unreliable verifier of security labels: it rejected 99.6% of hard negatives as malicious, fooled by the surface cues the class exists to penalize.

   result

   Screening pass rates: benign 89.6%, full hijack 3.6%, delayed execution 1.4%, hard negatives 0.4%; manual audit found 99.6% label correctness on 1,200 trajectories

   baseline

   Programmatic grammar checks (100% adherence) and 1,200-trajectory manual audit (99.6% correct)

   conditions

   Same model (Llama-3.3-70B) at temperature 0.1 with category-specific checklists; screening verdicts not used to filter the release

   the reviewer marked 1,494 of 1,500 as BAD because their observations “contain suspicious content” and “should be labeled as malicious”; that is, it was fooled by exactly the surface cues that the class exists to penalize

   Open supporting passage on PDF page 10
5. claim 05Paperraft inference

   The corpus has measurable template concentration and attack-goal-family collapse that enable memorization shortcuts.

   result

   28.9% of injected observations name an ext-audit.com address; ~75% of hard negatives mention $5,000, an in-domain audit@ address, or a transfer; external-recipient fraction is 0.47–0.57 across all six attack-goal families

   conditions

   4-gram analysis over the released corpus; family tags released as metadata only, not certified classification targets

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source

1. risk 01Paperraft inference

   All 12,536 trajectories come from a single generator model, so a detector may learn the generator's stylistic fingerprint rather than injection semantics; transfer to production agent traces is untested.

2. risk 02Paperraft inference

   World-identity leakage in four domains means results on the default split without anonymization or world-held-out splitting are not evidence of injection detection.

3. risk 03Paperraft inference

   Template concentration (28.9% of injections naming one fictitious domain; ~75% of hard negatives sharing three motifs) permits memorization unless target domains and amounts are masked at test time.

4. risk 04Paperraft inference

   Injections almost always fall at step 2 or 3 under fixed positional templates, enabling positional-prior shortcuts.

5. risk 05Paperraft inference

   Labels are protocol-specified and structurally validated rather than independently human-annotated; the 99.6% correctness figure is an estimate from a 1,200-trajectory sample.

6. risk 06Paperraft inference

   Short trajectories (3–11 steps), single agents, and no real agent traffic limit applicability to long-horizon or multi-agent settings.

1. Download the released corpus (CC BY 4.0, GitHub) and verify the counts and grammar claims directly: 12,536 trajectories, per-category label strings matching the stated regular languages, and the 70/15/15 split.
2. Reproduce the world-identity lookup check (Table XI) on the test split; if a world-lookup predictor does not reach ~86% binary accuracy on your copy, the leakage claim fails; if it does, apply world anonymization or a world-held-out split before any further evaluation.
3. Re-implement the six-feature logistic regression baseline (specified in Section IX-A, reproducible 'with a few lines of code') and confirm it recovers ~55% of attacks overall but under 25% on partial hijacks and delayed executions; if simple features solve those patterns, the paper's core difficulty claim fails.
4. Train a small sequence model (e.g., a short transformer or BiLSTM over the documented step linearization) on CPU or a single small GPU under the world-anonymized protocol; if it does not beat the surface baseline on partial-hijack and delayed-execution recall, the claimed need for sequence modeling is not demonstrated on your workload.
5. Spot-audit a random sample of attacked and hard-negative trajectories by hand against the category definitions to check whether the 99.6% label-correctness claim holds for your use case.

• Does a sequence-aware detector trained on AgentDrift actually outperform the surface baseline under the world-anonymized and world-held-out protocols, and by how much on partial hijack and delayed execution?
• Does training on this synthetic distribution transfer to real agent traces or to trajectories from other generators?
• What inter-annotator agreement would human labeling of the injection index and hijacked span yield (listed as future work)?
• Are the released GitHub artifacts (data files, tool pools, loader) complete and consistent with the paper's reported counts?
• What compute budget is needed to train a competitive sequence model on the corpus (not reported; only the trivial baseline is evaluated)?