How Much Does One Wrong Answer Cost? Error Propagation in Constraint Pruning with Noisy LLM Oracles

Tool errors or unavailable services cause your agent to abandon recoverable tasks.

What you would change

Inject repeatable tool failures and change only the recovery policy.

Inject repeatable tool failures and change only the recovery policy.

Compare against

Your current retry and fallback behavior under the same injected failures.

1. Locate the paper's implementation and check its license, model support, data needs, and hardware requirements. Stop here if you cannot obtain the required artifacts.
2. Save a small set of representative product tasks and expected outcomes. Keep evaluation tasks out of any training or tuning, and choose quality and cost limits before running.
3. Run the baseline and the proposed change on identical inputs. Keep other settings fixed and repeat paired runs to expose variation.
4. Save per-task outputs, failures, timings, and all usage costs. Review regressions before expanding the test.

What to measure

Recovered tasks, duplicate side effects, retries, time to recovery, and total cost.

When to keep it

Keep the change only if it meets your preselected quality and latency limits and improves the metric you care about after setup and operating costs. Otherwise retain the baseline. This is an exploration plan, not a validated recommendation.

recommendation

test

research area

failure recovery

technical change

Error-aware constraint pruning for noisy LLM oracles

published

2026-10-19

The reader already using LLM-based entity resolution or ranking with transitivity pruning can adopt pivot query ordering at zero infrastructure cost, but should validate error behavior on their own data since the analysis assumes small independent per-answer error.

1. claim 01Paperraft inference
   Read the brief interpretation

   The method does not replace constraint pruning itself; it replaces the assumption that a correct oracle makes query order irrelevant, prescribing query orders (notably pivot orders for entity resolution) that bound how far one wrong LLM answer propagates. It costs nothing in extra queries or infrastructure, only deliberate ordering of the same yes/no questions, with pivot ordering being simple to implement. It can fail when error independence or small-epsilon assumptions break down (correlated oracle errors, systematic biases toward certain record pairs), since the guarantees are first-order in epsilon and the common most-similar-first ordering is provably the worst in one dimension.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source
2. claim 02Paperraft inference

   For entity resolution, pivot orders amplify errors by less than a factor of 2 and are optimal, while random orders cost order m^(1/3) extra corrupted facts; random-order pruning under implication rules adds no first-order risk.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source