What Stops a Small Language Model From Driving a Database Agent

An assistant chooses the wrong API or sends arguments your product cannot use.

What you would change

Apply the proposed tool-selection or argument-generation change in a sandbox.

Apply the proposed tool-selection or argument-generation change in a sandbox.

Compare against

Your current agent with the same tool schemas and task budget.

1. Locate the paper's implementation and check its license, model support, data needs, and hardware requirements. Stop here if you cannot obtain the required artifacts.
2. Save a small set of representative product tasks and expected outcomes. Keep evaluation tasks out of any training or tuning, and choose quality and cost limits before running.
3. Run the baseline and the proposed change on identical inputs. Keep other settings fixed and repeat paired runs to expose variation.
4. Save per-task outputs, failures, timings, and all usage costs. Review regressions before expanding the test.

What to measure

Correct completed actions, invalid calls, retries, latency, and cost per completed task.

When to keep it

Keep the change only if it meets your preselected quality and latency limits and improves the metric you care about after setup and operating costs. Otherwise retain the baseline. This is an exploration plan, not a validated recommendation.

recommendation

test

research area

tool use

technical change

Tool-call argument logging and failure taxonomy for SQL agents

published

2026-09-18

For an engineer running small open-weight models as database agents on one GPU, the actionable lesson is cheap and directly applicable: log tool-call arguments and audit server behavior before concluding a model is too weak, since most observed failures here were transport and tooling defects rather than capability limits.

1. claim 01Paperraft inference
   Read the brief interpretation

   The work replaces the assumption that small-model agent failures reflect reasoning deficits with a failure-attribution methodology: capture model tool-call arguments (production ledgers record only refusal codes), classify losses into transport, capability, and related classes, and fix server-side defects before blaming the model. The cost is infrastructure rather than compute: logging tool arguments, building a scorer and failure taxonomy, and running clustered resampling by model, plus attention to a confound where an uncapped 262k-token context window let a 7.1 GB model hold 51 GB and produce runs indistinguishable from timeouts. After adoption, the main risks are that the transport-over-capability ordering is corpus-specific (holding in only 74.5% of model-clustered resamples) and that conclusions drawn without argument-level logging will misattribute server defects to model incapacity.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source
2. claim 02Paperraft inference

   No multiplicative improvement factor is claimed. The paper reports that 75.7% of model-attributed agent-mode losses came from runs that invoked at least one tool, that transport failures (no deliverable despite tool use) were the largest class at 36.2%, and that five server-side fixes with no model, prompt, or sampling changes moved six models by 6 to 21 cells out of 30.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source