Skip to content

archived edition · 2026-10-02

PaperraftFind the AI research
worth testing.

Understand what a paper changes, what it costs, and whether it fits your next build. Explore research on faster LLM inference, agent memory, and evaluation, with the original sources in reach.

Explore the researchHow papers are assessed

Start with a brief. Follow the evidence. Choose a test.

edition
2026-10-02
briefs
41
research areas
14
independent implementations
27

23 of 41 papers in the index (56%) have no independent implementations.

Research index

Search by topic, browse a research area, or sort by date. Read a brief, open the original paper, and inspect the available evidence before deciding what to test.

showing30 of 41
sort by
arXiv 2610.02206
First page of KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards
tool usetest

KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards

KaliBench replaces knowledge-based or end-to-end agentic cybersecurity evaluation with fine-grained natural-language-to-CLI pairs (8,504 queries, 1,642 tools) and runtime-free verifiable rewards usable for SFT and RLVR. Adopting it costs building or reusing the dataset, canonicalization and alias-aware scoring infrastructure, and fine-tuning runs, though an 8B model with parameter-efficient methods fits a single 24 GB GPU; teams outside the cybersecurity domain gain nothing directly. It can fail through reward hacking of exact-match signals, overfitting to the benchmark's 1,642-tool distribution while real environments use different tool versions and flags, and commands that score correct but are unsafe or non-executable in a live sandbox.

observed signal
2impl.
1stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.16098
First page of Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacks
safety and guardrailstest

Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacks

The method replaces undefended agent execution with a layered defense stack: Isolation Forest anomaly detection to filter attacker-injected tools, restoration of the original toolset before planning, plus Chain-of-Thought prompting, self-reflection, and task paraphrasing. It costs additional inference passes for reasoning and paraphrasing (raising latency and API spend), a white-box toolset snapshot requirement, and an anomaly detector that needs tuning against false positives on legitimate tools. It can fail against adaptive attacks crafted to evade the anomaly detector, against novel attack vectors outside the four studied categories, and through over-filtering that removes tools needed for task completion.

observed signal
2impl.
3stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01554
First page of QK-Wanda: Coupling Queries and Keys for Unstructured Pruning
sparsity and pruningtest

QK-Wanda: Coupling Queries and Keys for Unstructured Pruning

QK-Wanda replaces Wanda's per-projection independent weight scoring with closed-form scores that couple query and key projections under a shared pruning budget, using an unmasked pre-RoPE reconstruction objective. It costs only 1.3-3.1% more pruning time than Wanda, requires no gradients or weight updates, and preserves Wanda's one-shot calibration workflow. It can fail because lower local reconstruction error does not reliably predict downstream quality, as shown by Llama-3.1-70B's perplexity regression, and 80% unstructured sparsity yields limited practical speedup without sparse kernels.

observed signal
1impl.
4stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01139
First page of Do Multilingual Encoders Produce Language-Consistent Semantic IDs?
retrieval and RAGtest

Do Multilingual Encoders Produce Language-Consistent Semantic IDs?

The paper evaluates, rather than replaces, the standard practice of fitting residual quantizers on multilingual encoder embeddings to produce semantic IDs for generative retrieval. The cost is the same as baseline SID pipelines, but the evaluation reveals a quality risk: language-balanced quantizer fitting reduces cross-lingual first-code agreement (Spanish drops from 28.3% to 6.6%, versus 67.6% under an English-only fit). What can fail after adoption is cross-lingual retrieval consistency: translated items receive different SIDs than their source, degrading generative retrieval in multilingual catalogs, and the quantizer offers no selective robustness to translation-induced movement.

observed signal
3impl.
27stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.00894
First page of Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models
efficient architectureswatch

Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models

Clock Diffusion replaces prior continuous diffusion language model parameterizations, which lacked variable-length generation and KV-cache support, with a position-dependent noise schedule enabling semi-autoregressive block or sliding-window generation plus confidence-threshold and self-speculative samplers. The cost is training a custom diffusion language model from scratch, which exceeds the reader's no-foundation-model-training constraint, plus added sampler complexity relative to a standard autoregressive API or local model. After adoption, the approach can fail because reported gains are benchmark-level likelihood and GSM8K results on small models, so production quality at useful scale, tooling support, and serving stability are unproven.

observed signal
1impl.
4stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01509
First page of Sharpening Tax in Post-Training
efficient trainingwatch

Sharpening Tax in Post-Training

PTGS replaces fixed-temperature sampling during RL post-training of agentic LLMs with a Bayesian sampler that adapts temperature per prompt to estimated difficulty, aiming to preserve pass@K solution coverage that standard RL post-training erodes. It costs additional estimation overhead per prompt and requires the infrastructure to run RL post-training in agentic environments, which exceeds a single 24 GB GPU and is not available through third-party inference APIs. It can fail if the difficulty estimate is noisy on few rollouts, and its benefit is demonstrated only in two agentic environments, so transfer to other task distributions is unverified.

observed signal
2impl.
136stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01108
First page of AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines
speculative decodingtest

AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines

AgSpec replaces learned-draft speculative decoders (e.g., EAGLE-3) and generic retrieval drafters by retrieving token continuations from session-trajectory, workspace, and global corpora indexed in the agent's emission format, with per-agent draft lengths capped by offline profiling and adapted online from verification feedback. It costs the engineering of building and maintaining three corpora plus offline profiling of draft-length caps per agent, and adds retrieval and verification overhead per step, though it avoids training a draft model entirely. Gains depend on the agent actually repeating prior text; they shrink on workloads with little token reuse, and a stale or misformatted corpus or a miscalibrated draft-length cap can degrade throughput below the autoregressive baseline.

observed signal
3impl.
135stars
—citations
4.37xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01415
First page of Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States
memory and contexttest

Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States

PoS replaces passive history retention and context compression with an explicitly maintained belief state combining current world-state estimates and unresolved task requirements, plus consistency validation and trapping detection with tailored recovery. The cost is additional inference-time LLM calls for belief construction, validation, and progress monitoring, which increases token spend and latency per step and adds orchestration complexity. Failure modes include belief-state corruption from validation errors propagating through subsequent decisions, misclassified trapping patterns triggering inappropriate recovery, and reliance on backbone LLM quality for belief consistency that may not transfer to domains outside the tested benchmarks.

observed signal
3impl.
148stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.02150
First page of From Knowledge Access to Source Learning: Developing Source-Specific Competence
memory and contexttest

From Knowledge Access to Source Learning: Developing Source-Specific Competence

It replaces treating repeated queries to a persistent authoritative source as independent retrieval events (static RAG indices or generic agent memory) with a progressively refined, source-specific model built via self-directed gap identification and task-guided updates reconstructed from the source. It costs additional LLM calls for the self-directed learning loop and task-guided refinement, plus storage and maintenance of a persistent per-source model, adding latency and pipeline complexity relative to a single retrieval pass. It can fail when the source changes without triggering re-learning, when the workload does not repeatedly reuse the same source (amortizing the learning cost), or when reconstructed updates drift from the authoritative content and encode errors that persist across tasks.

observed signal
1impl.
121stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.02142
First page of Keyword Harnesses Fail Open: A Cheap Diagnostic Ladder for Tool-Use Claims in Small Language Models
agent evaluationadopt

Keyword Harnesses Fail Open: A Cheap Diagnostic Ladder for Tool-Use Claims in Small Language Models

It replaces keyword-matching tool-use benchmarks with verbatim-reproduction checks, a first-token prior probe, and embedding-drift checks. The diagnostic ladder costs minutes of CPU time; the optional repair is about 2,202 SFT steps and roughly 3.3 GPU-hours, feasible on one 24 GB GPU. After adoption, models can still over-trigger on negative prompts, suppression benefits remain seed-sensitive, and passing format diagnostics does not prove end-task correctness.

observed signal
1impl.
122stars
—citations
1.25xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.00983
First page of The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantization
quantizationadopt

The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantization

The technique replaces mean squared error with root mean squared error variants as the reconstruction loss when optimizing auxiliary quantization parameters in sequential learning-based PTQ. It costs nothing extra in memory or latency, since the square root is applied to the loss during the same gradient-based optimization already performed. It can fail if a given PTQ implementation already compensates for loss-scale imbalance through tuned per-stage learning rates, in which case the improvement may be smaller than reported, and the cross-stage loss-scale diagnosis should be verified on the target model before assuming benefit.

observed signal
1impl.
125stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01490
First page of The Persona Is Still There, but Who Is Speaking? Latent Identity Reversion in Persistent AI Agents
failure recoveryadopt

The Persona Is Still There, but Who Is Speaking? Latent Identity Reversion in Persistent AI Agents

It replaces one-time persona injection at conversation start with re-injecting the persona at the privileged system-prompt level on every turn, including resumed and automated turns. The cost is a few extra system-prompt tokens per request and minor prompt-management complexity. Failure mode: even with rich conversational history, an unanchored agent can appear to behave normally while silently reverting to the harness identity, so monitoring must check identity binding, not just output quality.

observed signal
1impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01618
First page of Agents Are Systems, Not Models: Rethinking Agentic Evaluation
agent evaluationadopt

Agents Are Systems, Not Models: Rethinking Agentic Evaluation

The paper replaces single success-rate evaluation of a fixed agent with evaluation of the agent as a configurable system, varying task information, reasoning, self-verification, time budget, and backbone model across repeated runs. Adoption costs multiple evaluation runs per configuration (the study used over 18,000 trajectories) plus engineering effort to expose configuration knobs and implement dedicated verification tools rather than relying on prompt instructions. Failures include mistaking run-to-run stochasticity for real improvement when run counts are low, and wasted compute from extending time budgets when the agent lacks sufficient task information or a capable model to use the extra time.

observed signal
1impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01766
First page of VideoEvolve: Evolving Agent Harnesses for Video Temporal Grounding
agent evaluationtest

VideoEvolve: Evolving Agent Harnesses for Video Temporal Grounding

VideoEvolve replaces manual, expert-driven refinement of agent workflows and prompt instructions around frozen video-language models with an automated evolutionary loop that edits harness code and instructions using execution feedback and validation. Its cost is the evolution compute: repeated candidate evaluation over video benchmarks, which means many frozen-model inference passes per iteration, plus engineering effort to define the Cloze-Structured Harness Representation and validation pipeline for a new task. It can fail if the search overfits to the validation split, if evolved code branches do not transfer across datasets or base models (the paper itself notes code-level benefits vary by evaluation setting), or if the evaluation budget is too small for the search to converge.

observed signal
1impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.02001
First page of Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks
failure recoverytest

Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks

Replaces cloud-oriented agent harnesses (e.g., goose, opencode) with a Windows/Ollama local-first harness whose ten mechanisms (net-zero prefill budget, finish gate re-reading the task, signature-level loop detection) target small-model failure forms for 2-9B models on a single 24 GB-class machine. Costs are integration effort with a Windows/Ollama-specific stack, additional guard logic per task step (latency and maintenance complexity), and reliance on a young third-party tool rather than an established framework. Can fail because the headline gains rest on a self-built benchmark with single-trial scoring, same-night replication variance equals the nominal ablation deltas, and the mechanism-level attribution is reported as directional only, so results may not transfer to other models, tasks, or operating environments.

observed signal
1impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.02038
First page of Mimir: Physics-Grounded LLM Agents for Long-Horizon Irrigation Control
tool usewatch

Mimir: Physics-Grounded LLM Agents for Long-Horizon Irrigation Control

The method replaces direct LLM-to-actuator control with an architecture where LLM proposals are checked and revised against a deterministic physics simulator, executed via bounded deterministic action selection, and improved slowly through consolidated contextual principles while the physical model stays immutable. The cost is substantial engineering: a domain-specific simulator, a structured physical interface, an evaluator, and per-site validation, plus ongoing LLM inference at every decision step. It can fail when the simulator misrepresents real dynamics, when persistent principles consolidate spurious patterns, and its evidence comes from retrospective evaluation on irrigation rather than live deployments or other domains.

observed signal
1impl.
131stars
—citations
2.04xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.02163
First page of AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents
memory and contextwatch

AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents

AutoCompact replaces heuristic or overflow-triggered context compaction with a compaction decision learned as part of the agent policy, trained via judge-corrected supervised fine-tuning followed by reinforcement learning with task-success rewards. Adoption requires the full training pipeline: rollouts of a base coding agent, a judge model to critique and correct compaction decisions, SFT infrastructure, and an RL stage with executable task environments. Failure modes include compaction policies that transfer poorly across models, repositories, or context budgets, judge errors propagating into corrected trajectories, and the added complexity of maintaining a training loop that a small team with one 24 GB GPU and no foundation-model training capacity cannot realistically reproduce.

observed signal
1impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.02002
First page of Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents
memory and contexttest

Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents

Mem++ replaces write-time distillation of documents into facts, notes, or graph edges with whole-document storage plus read-time retrieval that filters by the question's reference date and fuses lexical and semantic rankings, requiring no generative-model calls at write time. The cost is storage of every document version in full, per-query retrieval-and-fusion latency, and shifting all reasoning about which version held at a given time onto the answering model rather than the memory layer. It can fail when the answering model cannot reconcile conflicting document versions, when lexical-semantic fusion retrieves the wrong temporal slice for ambiguously dated questions, and because the headline gains are reported on the authors' own benchmark, so transfer to a different organizational corpus is unverified.

observed signal
1impl.
132stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.00899
First page of TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models
otherwatch

TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models

TOAST replaces FAST's single deterministic tokenization of each quantized robot action sequence with sampling among multiple valid token sequences that decode to the same motion during VLA policy training. It costs no additional demonstrations and only changes the training-time supervision targets, adding negligible compute but coupling the method to FAST-style tokenizers and autoregressive VLA architectures. Gains can fail to materialize when training data are abundant, when the action tokenizer lacks meaningful representational redundancy, or on tasks and robot embodiments unlike the LIBERO benchmark and the four real-robot tasks evaluated.

observed signal
0impl.
214stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.00910
First page of The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention
otherwatch

The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention

This is a mechanistic interpretability finding, not a deployable method: it shows that LLMs index in-context facts by order of mention via a low-rank 'ordinal vector' in late-middle layers, rather than replacing any production component. Applying it as activation steering costs an extraction pass over many fact lists to compute the averaged vector and a per-token residual modification at inference, adding implementation complexity with no quantified gain. It can fail because the demonstrated behavior is limited to simple structured fact-list tasks; steering generalization to arbitrary prompts, long contexts, or multi-hop reasoning is not established, and interventions at middle layers can degrade output quality.

observed signal
0impl.
121stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.00929
First page of Platonic Task Arithmetic
efficient trainingtest

Platonic Task Arithmetic

The method replaces per-model fine-tuning and single-architecture task arithmetic by extracting a model-agnostic task descriptor from one model and injecting it into a heterogeneous target via a single least-squares weight edit or a low-rank adapter, needing only unlabeled probe images and class-name prompts. The closed-form variant costs one least-squares solve with no optimization loop and composes edits by signed summation; the adapter variant requires one optimization run per edit, and both require access to target weights, excluding API-only models. It can fail because heterogeneous models share the task object only partially (the model-specific residual is comparable in norm to the shared component), so 20-26% of the in-model gain is lost in transfer, with likely degradation on tasks far from the classification settings evaluated.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.00964
First page of RPTune: Learned Context Curation for LLM Catalog Search
memory and contexttest

RPTune: Learned Context Curation for LLM Catalog Search

It replaces multi-stage retrieval and ranking pipelines for merchant catalogs that fit in the context window with learned ordering/pruning plus LLM post-training. It costs an encoder-reorganizer curator, catalog-grounded supervision generation, possible fine-tuning or API post-training, added curation latency, and per-merchant maintenance. It can fail when catalogs grow beyond effective context use, change frequently, have noisy generated supervision, or when curation prunes or misorders relevant products.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01088
First page of Polylogarithmic Sparsity of Randomly Reweighted NPMLEs for Gaussian Mixtures
othernot applicable

Polylogarithmic Sparsity of Randomly Reweighted NPMLEs for Gaussian Mixtures

The method replaces the classical Gaussian location-mixture NPMLE, whose support can grow linearly with sample size, with a Gamma-reweighted likelihood estimator that has provably polylogarithmic support while nearly maximizing the ordinary likelihood. It costs a small random perturbation of the likelihood and the same convex optimization machinery as the standard NPMLE, with Hellinger risk parametric up to logarithmic factors. It can fail in that the sparsity bound is asymptotic and probabilistic, the analysis is specific to Gaussian location mixtures, and the numerical evidence is illustrative rather than benchmarked on production workloads.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01170
First page of HeadEdit: Calibrating Language Model Behavior Through the Frozen Unembedding Matrix
safety and guardrailstest

HeadEdit: Calibrating Language Model Behavior Through the Frozen Unembedding Matrix

HeadEdit replaces manual activation steering, target-token heuristics, or additional preference-tuning runs for correcting residual misbehaviors such as over-refusal, unnecessary tool calls, and sycophancy, by applying a prompt-adaptive vocabulary-wide logit correction derived from a low-rank subspace of the frozen unembedding matrix. Its cost is a one-time extraction of paired completions to fit the behavioral subspace plus a small inference-time projection, with negligible latency overhead and no parameter updates or retraining. It can fail if the paired-completion data does not represent the production behavior distribution, if the linear decodability assumption breaks for a given model or behavior, or if the correction shifts logits on benign prompts in ways the paper's limited task suite did not expose.

observed signal
0impl.
121stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01275
First page of Know When to Hold 'em: Correct-Token Retention in Uniform-State Diffusion Language Models
efficient trainingwatch

Know When to Hold 'em: Correct-Token Retention in Uniform-State Diffusion Language Models

CTR-Reg replaces the standard USDM training objective (which penalizes errors only at corrupted positions) with an auxiliary loss rewarding retention of unperturbed clean tokens, requiring no sampler change. It costs an extra loss term during training or fine-tuning, which means it cannot be applied to third-party API models and presumes the capacity to train or fine-tune a diffusion language model. It can fail if the workload uses masked rather than uniform-state diffusion, and retention gains may not transfer to tasks or model scales beyond the six benchmarks evaluated.

observed signal
0impl.
121stars
—citations
2xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01326
First page of An ontology for cross-sectoral crisis management: core and public health modules
otherwatch

An ontology for cross-sectoral crisis management: core and public health modules

ECMO replaces ad hoc, inconsistent data schemas for crisis information with a formal modular OWL ontology (core plus public health module aligned with SNOMED CT and ICD-11) that constrains how unstructured epidemiological news is populated into a knowledge graph. The cost is ontology-engineering effort, OWL tooling and reasoning infrastructure, and the complexity of maintaining module alignments, none of which map to GPU or API budgets. It can fail through poor domain coverage for non-health crises, mismatch with existing data pipelines, and the demonstration remaining limited to a single JRC system with no quantified accuracy or interoperability benchmarks.

observed signal
0impl.
1stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01349
First page of PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents
safety and guardrailstest

PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents

PACE replaces admission-time vetting of tool metadata, retrieved pages, memory, and skills with per-call mediation that checks each tool call's provenance, schema-defined effects, and authority immediately before execution. The cost is a mediation layer on every tool call, compilation of authority from each authenticated request, and implementation of path confinement, effect verification, and a repair/restore mechanism—non-trivial engineering for a small team, though it is model-agnostic and adds no GPU or training burden. It can fail if the certified execution contract mis-specifies authority or effects, and residual risk remains in the attacks where it ties rather than wins and in adaptive threats beyond the reduced-scale 30-target evaluation.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01434
First page of MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs
sparsity and pruningtest

MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs

MWOP replaces dense attention and FFN execution in multimodal LLMs by independently pruning V2V, T2V, and T2T attention paths and selecting FFN channels separately per modality, while preserving the token sequence, using path-sparse Triton kernels for actual acceleration. It costs a one-time pruning pipeline with first-order Taylor importance estimation, FFN re-evaluation, LoRA recovery training, and dependence on custom Triton kernels that must be maintained per model. It can fail if the custom kernels are not ported to new architectures or serving stacks, if accuracy degrades on visual tasks underrepresented in the 12-benchmark average, or if the reader's workloads run on third-party APIs where weights cannot be modified.

observed signal
0impl.
0stars
—citations
1.6xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01508
First page of OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agents
safety and guardrailstest

OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agents

SelfAudit replaces unfiltered direct execution of model-proposed tool calls with a zero-shot inference-time pass that generates request-grounded justifications and drops unjustified calls before they run. It costs an extra generation step per candidate call, adding latency and API token spend proportional to the tool-call volume, with no additional infrastructure or training. It can fail when the justification step itself rationalizes unnecessary calls, when filtering removes legitimately needed calls and degrades task completion, and because the 43% figure comes from the authors' own benchmark and may not transfer to other tool schemas or domains.

observed signal
0impl.
10stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2610.01537
First page of FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization
efficient trainingwatch

FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization

FedFit replaces standard federated LoRA aggregation with a disjoint shared vector-bank parameterization, alternating single-bank and joint updates with Residual Spectral Aggregation, plus blockwise quantization with client-side error feedback, to cut FL communication overhead. The cost is substantial implementation complexity: a custom aggregation schedule, error-feedback state per client, and a non-standard adapter parameterization that must be maintained outside mature frameworks. It can fail in non-federated settings (the method provides no benefit without distributed clients), under high client heterogeneity where residual aggregation may degrade, and its convergence and perplexity parity are demonstrated only on Qwen2.5 within the paper's experimental conditions.

observed signal
0impl.
0stars
—citations
100xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗

Research signals

Three views of the same evidence: adoption against attention, publication cadence by research area, and reported gains when coverage is sufficient.

color = research area

01 · attention × adoption

The research frontier

Look to the upper left for independent implementation before mass attention.

compare by
Each point represents one paper. Higher means more independent implementations; farther right means more GitHub stars.Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacks: 3 GitHub stars; 2 independent implementationsClock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models: 4 GitHub stars; 1 independent implementationTOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models: 214 GitHub stars; 0 independent implementationsThe Geometry of Contextual Relations: Language Models Address Facts by Order of Mention: 121 GitHub stars; 0 independent implementationsPlatonic Task Arithmetic: 0 GitHub stars; 0 independent implementationsRPTune: Learned Context Curation for LLM Catalog Search: 0 GitHub stars; 0 independent implementationsThe Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantization: 125 GitHub stars; 1 independent implementationPolylogarithmic Sparsity of Randomly Reweighted NPMLEs for Gaussian Mixtures: 0 GitHub stars; 0 independent implementationsAgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines: 135 GitHub stars; 3 independent implementationsDo Multilingual Encoders Produce Language-Consistent Semantic IDs?: 27 GitHub stars; 3 independent implementationsHeadEdit: Calibrating Language Model Behavior Through the Frozen Unembedding Matrix: 121 GitHub stars; 0 independent implementationsKnow When to Hold 'em: Correct-Token Retention in Uniform-State Diffusion Language Models: 121 GitHub stars; 0 independent implementationsAn ontology for cross-sectoral crisis management: core and public health modules: 1 GitHub star; 0 independent implementationsPACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents: 0 GitHub stars; 0 independent implementationsBeyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States: 148 GitHub stars; 3 independent implementationsMWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs: 0 GitHub stars; 0 independent implementationsThe Persona Is Still There, but Who Is Speaking? Latent Identity Reversion in Persistent AI Agents: 131 GitHub stars; 1 independent implementationOverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agents: 10 GitHub stars; 0 independent implementationsSharpening Tax in Post-Training: 136 GitHub stars; 2 independent implementationsFedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization: 0 GitHub stars; 0 independent implementationsQK-Wanda: Coupling Queries and Keys for Unstructured Pruning: 4 GitHub stars; 1 independent implementationChaining Skills to Hijack LLM Agents: 0 GitHub stars; 0 independent implementationsExploiting the Interplay of Compute- and Memory-Bound kernels in MPI Applications: 0 GitHub stars; 0 independent implementationsAgents Are Systems, Not Models: Rethinking Agentic Evaluation: 131 GitHub stars; 1 independent implementationFedSAP: Federated Learning with Structured Adaptive Partitioning for Multi-Domain Heterogeneous Edge Devices: 0 GitHub stars; 0 independent implementationsMiLoop: Selective Memory Propagation for Neural Combinatorial Optimization: 0 GitHub stars; 0 independent implementationsTask-Oriented Rank Adaptation for Continual Learning in Text Classification: 121 GitHub stars; 0 independent implementationsVideoEvolve: Evolving Agent Harnesses for Video Temporal Grounding: 131 GitHub stars; 1 independent implementationCONTRA: Discovering and Qualifying Behavior-Changing Questions for Selective Clarification in LLM Code Generation: 0 GitHub stars; 0 independent implementationsContinuous Process-Level Evaluation for Evolving Enterprise AI Agent Skills: 10 GitHub stars; 0 independent implementationsSelection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents: 10 GitHub stars; 0 independent implementationsFastCI: Efficient GPU-Intensive CI for LLM Training Frameworks: 0 GitHub stars; 0 independent implementationsMingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks: 131 GitHub stars; 1 independent implementationMem++: Non-Destructive Memory for Long-Term Organizational LLM Agents: 132 GitHub stars; 1 independent implementationMimir: Physics-Grounded LLM Agents for Long-Horizon Irrigation Control: 131 GitHub stars; 1 independent implementationLearn the Directions, Normalize the Gains: Post-Training Normalization for LoRA: 0 GitHub stars; 0 independent implementationsHumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution: 9 GitHub stars; 0 independent implementationsKeyword Harnesses Fail Open: A Cheap Diagnostic Ladder for Tool-Use Claims in Small Language Models: 122 GitHub stars; 1 independent implementationFrom Knowledge Access to Source Learning: Developing Source-Specific Competence: 121 GitHub stars; 1 independent implementationAutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents: 131 GitHub stars; 1 independent implementationKaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards: 1 GitHub star; 2 independent implementations

Papers above 1000 stars or 200 citations are not scored. They have already broken through the attention threshold, while Paperraft is designed to identify earlier signals.

02 · cadence

Research areas over time

Compare monthly publication volume on a shared scale and calendar.

coding agentsmax 7Sep 26: 0 papersOct 26: 1 paperSep 26Oct 26efficient architecturesmax 7Sep 26: 0 papersOct 26: 1 paperSep 26Oct 26agent evaluationmax 7Sep 26: 0 papersOct 26: 4 papersSep 26Oct 26speculative decodingmax 7Sep 26: 0 papersOct 26: 1 paperSep 26Oct 26sparsity and pruningmax 7Sep 26: 0 papersOct 26: 3 papersSep 26Oct 26memory and contextmax 7Sep 26: 0 papersOct 26: 5 papersSep 26Oct 26othermax 7Sep 26: 0 papersOct 26: 7 papersSep 26Oct 26planning and decompositionmax 7Sep 26: 0 papersOct 26: 1 paperSep 26Oct 26quantizationmax 7Sep 26: 0 papersOct 26: 1 paperSep 26Oct 26failure recoverymax 7Sep 26: 0 papersOct 26: 2 papersSep 26Oct 26retrieval and RAGmax 7Sep 26: 0 papersOct 26: 1 paperSep 26Oct 26safety and guardrailsmax 7Sep 26: 1 paperOct 26: 4 papersSep 26Oct 26efficient trainingmax 7Sep 26: 0 papersOct 26: 6 papersSep 26Oct 26tool usemax 7Sep 26: 0 papersOct 26: 3 papersSep 26Oct 26

Every panel uses the same calendar and vertical scale. An empty column means no papers were recorded.

Method under scrutiny

Inspect the highest-ranked paper, every repository behind its signal, and the rule used to classify each one.

KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards

tool use · test · 2 of 2 implementations are independent

KaliBench replaces knowledge-based or end-to-end agentic cybersecurity evaluation with fine-grained natural-language-to-CLI pairs (8,504 queries, 1,642 tools) and runtime-free verifiable rewards usable for SFT and RLVR. Adopting it costs building or reusing the dataset, canonicalization and alias-aware scoring infrastructure, and fine-tuning runs, though an 8B model with parameter-efficient methods fits a single 24 GB GPU; teams outside the cybersecurity domain gain nothing directly. It can fail through reward hacking of exact-match signals, overfitting to the benchmark's 1,642-tool distribution while real environments use different tool versions and flags, and commands that score correct but are unsafe or non-executable in a live sandbox.

Exclusions and controls

Every exclusion is counted and published.

Exclusion counts were not recorded for this edition.

0 previously indexed papers were rechecked in this run.

Paperraft tracks one signal: how many independent implementations a paper attracts on GitHub after author-owned repositories are removed. This is a discovery signal to investigate, not a measure of implementation quality.

It does not claim that a method works. Paperraft runs no reproduction benchmarks. Reported gains remain author claims until independently tested. Papers above the attention threshold are excluded because this publication is designed to identify research before consensus forms.