Symbolic Temporal Supervision of LLM Agents Using Contracts

An agent reads external documents or tool responses that may contain malicious instructions.

What you would change

Replay attacks and legitimate lookalikes in an isolated copy of your agent workflow.

Replay attacks and legitimate lookalikes in an isolated copy of your agent workflow.

Compare against

Your existing checks with the same tasks, permissions, and tool responses.

1. Locate the paper's implementation and check its license, model support, data needs, and hardware requirements. Stop here if you cannot obtain the required artifacts.
2. Save a small set of representative product tasks and expected outcomes. Keep evaluation tasks out of any training or tuning, and choose quality and cost limits before running.
3. Run the baseline and the proposed change on identical inputs. Keep other settings fixed and repeat paired runs to expose variation.
4. Save per-task outputs, failures, timings, and all usage costs. Review regressions before expanding the test.

What to measure

Successful attacks missed, legitimate actions blocked, detection delay, and review cost.

When to keep it

Keep the change only if it meets your preselected quality and latency limits and improves the metric you care about after setup and operating costs. Otherwise retain the baseline. This is an exploration plan, not a validated recommendation.

recommendation

test

research area

safety and guardrails

technical change

LTLf contract-based guardrails for tool-calling agents

published

2026-09-16

The runtime is model-independent and cheap enough for constrained infrastructure, but the value depends entirely on the quality of manually authored contracts, so it warrants a workload-specific pilot rather than unconditional adoption. On constrained infrastructure this replaces expensive per-call LLM judging with a deterministic artifact whose main cost is upfront contract engineering, so a scoped pilot on one tool domain is the appropriate step. However, its value depends entirely on manually authored contracts and observable predicates, so a scoped pilot on one tool domain is warranted rather than unconditional adoption. Therefore a scoped pilot on one tool domain is warranted rather than unconditional adoption. On constrained infrastructure this replaces per-call LLM judging with a cheap deterministic artifact, so it warrants a scoped pilot rather than unconditional adoption or dismissal. A scoped pilot on one tool domain is the appropriate step.

1. claim 01Paperraft inference
   Read the brief interpretation

   It replaces stochastic post-hoc LLM judges and per-call rule-based blockers with a single deterministic artifact: assume-guarantee contracts in finite-trace LTL, compiled to DFAs that gate tool calls online and evaluate recorded traces offline. The cost is engineering effort, not compute: someone must formalize checkable predicates over tool calls and author and maintain a correct contract library per task domain, plus DFA compilation overhead. It can fail when contracts are incomplete, wrongly specified, or the predicate set cannot observe the relevant state, in which case harmful actions pass deterministically or benign ones are blocked, and coverage does not transfer across domains without rewriting contracts.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source
2. claim 02Paperraft inference

   Matches state-of-the-art LLM-judge and rule-based guardrail baselines across four benchmarks while producing deterministic, reproducible verdicts and, in online gating mode, orders-of-magnitude lower per-call latency than stochastic LLM judges.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source