Skip to content

archived edition · 2026-10-01

PaperraftFind the AI research
worth testing.

Understand what a paper changes, what it costs, and whether it fits your next build. Explore research on faster LLM inference, agent memory, and evaluation, with the original sources in reach.

Explore the researchHow papers are assessed

Start with a brief. Follow the evidence. Choose a test.

edition
2026-10-01
briefs
42
research areas
13
independent implementations
25

26 of 42 papers in the index (62%) have no independent implementations.

Research index

Search by topic, browse a research area, or sort by date. Read a brief, open the original paper, and inspect the available evidence before deciding what to test.

showing30 of 42
sort by
arXiv 2601.12164
First page of The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents
agent evaluationtest

The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents

Replaces single-language neutrality evaluation with semantically paired prompts in different languages. It needs paired prompts and extra API calls, with human interpretation because contested questions lack a definitive answer key. The study reports that ChatGPT 5.2 and Claude Opus 4.5 adopt the prompt language's dominant framing, but examines a single Ukrainian document and does not establish generalization to other language pairs or topics.

observed signal
2impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.40127
First page of Learning Functional Subspaces for Neural Network Compression
efficient architecturestest

Learning Functional Subspaces for Neural Network Compression

LSP replaces closed-form low-rank truncation criteria (activation energy, layer-wise reconstruction, quadratic loss approximations) with orthogonal projectors learned jointly against a global KL or training-loss objective, then merged into standard low-rank factors that also permit a shared narrow KV latent. The cost is an additional optimization pass over the projectors (weights stay frozen, so it fits one 24 GB GPU for models up to roughly 7B with careful handling), plus rank-allocation overhead and the engineering of tied-layer factor sharing, which is not yet an off-the-shelf library feature. It can fail when the KL-to-dense objective does not preserve task-specific behavior, when downstream kernels do not exploit the low-rank factors or shared latent (eliminating the speedup), and at extreme compression where even learned subspaces degrade.

observed signal
1impl.
1stars
—citations
1.6xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.16461
First page of Protocol-Preserving Context Trimming for Agentic Workflows: Benefits, Failure Regimes, and Budget Guardrails
memory and contexttest

Protocol-Preserving Context Trimming for Agentic Workflows: Benefits, Failure Regimes, and Budget Guardrails

The method replaces unconstrained context growth and naive trimming (recency, relevance, or summarization-based) with protocol-aware selection plus adaptive retained-context budgets that protect protocol-critical state in agentic workflows. It costs implementation complexity: the system must identify protocol-critical state (instructions, tool states, unresolved dependencies) and maintain budget-enforcement logic, and it retains roughly 44% more tokens than the most aggressive conventional trimming. Failure risk concentrates below 25% retained-context budgets, where failure odds rise 10.92-fold, and critical thresholds increase with workflow complexity, so a static configuration can silently degrade on harder tasks.

observed signal
2impl.
3stars
0citations
2.11xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39903
First page of OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software
agent evaluationtest

OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software

Replaces ad hoc or general-purpose agent evaluations (e.g., generic OSWorld tasks) with 146 expert-designed scientific software tasks featuring artifact-based evaluators that inspect molecular structures, segmentation masks, plots, and numerical outputs with partial credit. Adopting it costs setup effort for its harness and evaluation environments, plus compute/API spend to run agents, but requires no model training. Its findings indicate even state-of-the-art VLMs fail on many scientific workflows, so agents validated only on this benchmark may still underperform on the reader's specific software stack, and benchmark coverage across only a few domains may not transfer to other scientific tools.

observed signal
3impl.
135stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39564
agent evaluationtest

A2Z GameSpec-Bench: How Faithfully Can Coding Agents Generate Games from Game Design Specifications?

Replaces ad-hoc prompting and manual judgment of agent-built applications with a fixed dependency-aware contract (rules, constraints, prerequisite relations) plus code inspection and scenario-replay playtesting for measuring specification faithfulness. Costs include building per-specification contracts, running agent-generated test policies that consume API or GPU budget, and maintaining the evaluation harness across revision rounds. Contract construction can itself be incomplete or mis-specified, agent-generated test policies may miss failure modes or reward test-gaming, and results on game GDDs may not transfer to other application domains.

observed signal
3impl.
136stars
—citations
1.109xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39982
First page of Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents
coding agentstest

Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents

The method inserts a sample-and-verify step between the model and the execution harness, replacing the default practice of executing the first sampled action directly; it leaves the generator and harness unchanged. It costs additional inference per step (multiple sampled candidate actions plus verifier calls, which for the strongest reported result use a paid frontier-model API), adding per-action latency and token spend. It can fail when verification is weak: with a low-capability verifier, extra action sampling yields little benefit, so the gain depends on having a verifier stronger than the generator, and the reported results center on terminal benchmarks rather than general coding workloads.

observed signal
3impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.40111
First page of Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training
failure recoverytest

Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training

The method replaces discarding failed agent rollouts or training only on successful trajectories with a pipeline that converts failures into error-diagnosis and correction pairs for fine-tuning. Cost is pipeline complexity: it requires multi-model trace collection, LLM-generated diagnoses verified against recorded evidence, and replay infrastructure, plus separate fine-tuning runs that fit a 24 GB GPU only at the 8B scale demonstrated. It can fail when replay is unsupported in a given environment, when LLM-generated diagnoses are mislabeled against the recorded evidence, and when gains do not transfer beyond the specific harness families and benchmarks evaluated.

observed signal
2impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.40097
First page of AutoDataBench: A Data-centric Testbed for Accelerating Auto Research
agent evaluationwatch

AutoDataBench: A Data-centric Testbed for Accelerating Auto Research

AutoDataBench replaces general auto-research benchmarks by isolating data-related agent capabilities (diagnosis, organization, construction) while holding training framework, hyperparameters, and compute fixed; it is an evaluation testbed, not a production method. Adopting it costs the compute to run its three curated optimization tasks under task-specific resource budgets, plus engineering effort to integrate with an evaluation pipeline; the paper also suggests reusing its agent trajectories for mid-training, which costs fine-tuning compute. It can fail to transfer: performance on curated data tasks may not predict an agent's data-handling ability on a proprietary pipeline, and the mid-training benefit is reported only for downstream coding performance, not general workloads.

observed signal
1impl.
123stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39578
First page of Thinking Outside the Box: Can Language Models Rely on External Guidance Selectively?
failure recoverytest

Thinking Outside the Box: Can Language Models Rely on External Guidance Selectively?

The method replaces blind adherence to fixed agent workflows with counterfactual supervised fine-tuning and outcome-based reinforcement learning that teach a model to follow reliable guidance and override misleading guidance, evaluated on the Box^2-Bench benchmark. It costs fine-tuning compute on open-weight models (feasible at small scale with parameter-efficient methods), requires constructing counterfactual training data from bad workflows, and adds training and evaluation complexity without changing inference cost. It can fail if the training distribution of unreliable guidance does not match production failure modes, since frontier models remain vulnerable to misleading guidance and the learned selectivity may not transfer to unseen workflow or memory-corruption patterns.

observed signal
1impl.
121stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39551
First page of RankEvolve: A Reliable Multi-Agent Auto-Research Harness for Evolving Ranking Models
multi-agent orchestrationtest

RankEvolve: A Reliable Multi-Agent Auto-Research Harness for Evolving Ranking Models

RankEvolve replaces ad hoc single-agent automation of the ML experimental loop with a compiled state-machine protocol (EOP) that gates phases and composes multiple black-box coding-agent products (e.g., Claude Code, Codex) to review and repair each other's work, plus a knowledge layer carrying negative results across iterations. It costs subscriptions or API fees for several frontier coding-agent products, orchestration engineering to build the runtime and benchmarks, and added iteration latency from cross-agent review cycles. Even after adoption, roughly one in ten executions can carry a silent critical defect (data leakage, disconnected gradients, unwired train/eval flags), so expensive runs still require independent validation, and the absolute accuracy ceiling of 62.5% means the harness reduces but does not eliminate invalid experiments.

observed signal
1impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39765
First page of MemCodex: Self-Programming Hierarchical Memory for Language Agents
memory and contexttest

MemCodex: Self-Programming Hierarchical Memory for Language Agents

MemCodex replaces fixed, predefined memory workflows (and adaptive methods limited to a predefined design space) with a hierarchy of executable memory programs whose construction, indexing, retrieval, and routing are evolved automatically, traversed coarse-to-fine at query time. The cost is substantial system complexity: a program-evolution search loop, a unified runtime (MemArena) to host heterogeneous memory backends, and added infrastructure for maintaining and rewriting memory layers, none of which runs without ongoing LLM calls for evolution and routing. Failure modes include evolved retrieval programs degrading silently as workloads drift, early stopping at coarse layers missing fine-grained evidence, and the reported gains not transferring if the reader's tasks differ from the evaluation benchmark.

observed signal
1impl.
131stars
—citations
1.1xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39958
First page of Better Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking
agent evaluationtest

Better Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking

The harness replaces direct single-prompt generation with a pipeline of financial calculations, narrative templates, and validation checks, developed and scored under LLM judges. It costs additional engineering complexity, per-run compute for calculation and validation stages, and dependence on judge-in-the-loop development using a 27B model that fits on constrained hardware only in quantized form. Judge scores shift on repeated grading of unchanged decks, judges disagree more on final-deck rankings than pooled scores, and near-zero margins against direct frontier-model generation mean the measured gains may partly reflect grading behavior rather than document quality.

observed signal
1impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.40027
First page of Who Verifies the Graph? Misspecification Attacks on Causal Action Verification for Language Agents
safety and guardrailswatch

Who Verifies the Graph? Misspecification Attacks on Causal Action Verification for Language Agents

The method replaces trust in internally valid identification certificates from a causal action verifier with an attestation layer that tests certified executions against bounded randomized samples and refuses what fails or cannot be tested. It costs a randomized experiment budget in the live tool environment (127 experiments per 1,050 actions for safety alone, 614 more to recover beneficial executions), plus rejection-auditing cost that scales with rejections rather than executions. It can fail to deliver value even when safe: 97.1% of beneficial actions remain unexecuted at the published confounding strength, and the defense assumes the ability to run randomized interventions, which many production tool APIs do not permit.

observed signal
1impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.40118
First page of Persistent Context Graphs for Efficient Memory Compaction in LLM Agents
memory and contexttest

Persistent Context Graphs for Efficient Memory Compaction in LLM Agents

ReCAP replaces summarization-based and KV-cache-based history compaction with a persistent context graph of attention-derived importance scores and dependency links, so message selection for each new request uses stored signals plus lightweight relevance matching against the request rather than additional model calls. The cost is instrumentation to capture and store attention statistics and graph links per message, plus implementation complexity, and it presumes access to attention weights, which third-party APIs generally do not expose. It can fail when attention-based importance diverges from actual task relevance, when dependency links omit context a later request needs, or when the latency estimates do not transfer to the reader's real serving stack and workloads.

observed signal
1impl.
131stars
—citations
20xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.40169
First page of Learning from Research: Toward Lifelong Agent Harness Evolution
tool usewatch

Learning from Research: Toward Lifelong Agent Harness Evolution

Replaces manual, reactive prompt-and-harness tuning of tool use, memory, and execution scaffolding with a meta coding agent that proposes and evaluates harness modifications drawn from topic-modeled research literature. Costs include running a capable meta agent plus repeated evaluation of strategy combinations, which adds API spend, implementation complexity, and benchmark engineering without changing the base model. Can fail when retrieved strategies do not transfer to the production task distribution, when combinatorial evaluation overfits public benchmarks, or when literature-driven changes introduce regressions the evaluation harness does not detect.

observed signal
1impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.40303
First page of How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?
multi-agent orchestrationadopt

How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?

This approach replaces elaborate multi-agent orchestrators, retrieval subagents, and hand-crafted scaffolding for autonomous ML engineering with a single coding-agent session giving the LLM direct read, write, and bash access to the execution environment. It costs far less engineering effort and orchestration complexity, and reduces per-run token overhead from inter-agent communication, while relying on a frontier API backbone rather than local infrastructure. It can fail when tasks genuinely require parallel exploration or specialization beyond one session, when the available backbone is weaker than the frontier models used in the study, or when benchmark results do not transfer to the reader's production workloads.

observed signal
1impl.
131stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39449
First page of Distributionally robust linear regression through the lens of adversarial training
otherwatch

Distributionally robust linear regression through the lens of adversarial training

The method replaces standard least-squares or Lasso linear regression with a Wasserstein distributionally robust estimator that subsumes square-root Lasso and adversarial linear regression as special cases. The paper claims efficient solvability, so computational cost is comparable to classical regularized regression, but adoption requires reimplementing or integrating the estimator and validating the ambiguity-set radius on real data. It can fail if the assumed linear model is wrong for the workload, if the ambiguity radius is miscalibrated, or if the theoretical guarantees (O(n^{-1/2}) error bounds, pivotal property) do not translate into predictive gains on non-synthetic data; only numerical simulations are reported.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39489
First page of Towards Robust Time Series Learning via Capacity-Centric Modulation
efficient trainingtest

Towards Robust Time Series Learning via Capacity-Centric Modulation

SACM replaces uniform, fixed dropout regularization with sample-wise dropout probabilities computed from spectral sparsity of internal activations, integrated into existing backbones without architectural changes. It costs only training-time modifications and added implementation complexity; the inference pipeline is unchanged and deterministic. It can fail when the spectral-sparsity signal misestimates sample reliability on a given dataset or backbone, and gains may not transfer outside the reported benchmark suite, since improvements vary across tasks (17% F1 in anomaly detection versus low single digits elsewhere).

observed signal
0impl.
0stars
—citations
1.07xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39542
First page of Comparative study of adapting pre-trained models for driving behavior video captioning
efficient trainingtest

Comparative study of adapting pre-trained models for driving behavior video captioning

The method replaces task-specific model training from scratch with adaptation of a pre-trained video-language model (SpaceTimeGPT, VideoLLaVA) via full fine-tuning, LoRA, or prompt engineering for driving-behavior captioning. Full fine-tuning costs the most memory and is likely beyond a single 24 GB GPU without aggressive optimization, while LoRA reduces trainable parameters and fits constrained hardware at some risk of lower ceiling; prompting costs nothing but showed limitations in the study. Failures include overfitting to the narrow BDD-X domain, weak generalization to other driving footage, and metric gains that may not reflect caption correctness or safety relevance.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39549
First page of Speculative Safety Honeypot: Toward Proactive Defense Against Multi-turn Agent Attacks
safety and guardrailstest

Speculative Safety Honeypot: Toward Proactive Defense Against Multi-turn Agent Attacks

SSH replaces purely retrospective, context-based multi-turn attack detection with an action-level speculate-and-verify workflow in which small LLMs simulate a trajectory tree of the target agent's future behavior and prune it against observed actions. The cost is substantial additional inference per monitored interaction: an asynchronous multi-agent simulation maintained alongside production traffic, plus integration and tree-management complexity, all of which multiply latency and compute on a single-GPU or API-budget setup. It can fail through speculative divergence, where simulated trajectories poorly predict the real agent's behavior and either miss split-intent attacks or generate false-positive warnings that erode operator trust, and the abstract provides no quantified detection or lead-time results to bound this risk.

observed signal
0impl.
10stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39601
First page of GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
otherwatch

GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

GroundingPI replaces general-purpose vision-language and video-generation backbones inside VLA and world-action pipelines with a 4B model pretrained specifically to emit points and boxes as quantized coordinates. The cost is the full training stack the reader cannot afford: multimodal and spatial pretraining, supervised fine-tuning, and GRPO reinforcement learning over dedicated data engines, plus integration work to swap the perception backbone of an existing manipulation or driving stack; inference of a 4B model is feasible on one 24 GB GPU only if weights are released. What can fail is generalization to the reader's own sensors, robots, and domains, since the reported gains are benchmark-specific and the model may inherit grounding errors in clutter, tiny-object cases, or real-world distribution shift not covered by RoboTwin, RoboCasa, or nuScenes.

observed signal
0impl.
214stars
—citations
1.248xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39613
First page of Hybrid Methods for Robust Tabular Data Imputation
otheradopt

Hybrid Methods for Robust Tabular Data Imputation

The method replaces MissForest's iterative random-forest cycles with a one-shot pipeline: an SVT or SoftImpute low-rank initialization followed by a single non-iterative Random Forest refinement. Cost is modest and CPU-feasible: one matrix-completion pass plus one forest fit, well within a 24 GB GPU budget and requiring no cluster, though it adds the implementation complexity of tuning the low-rank step (rank threshold, adaptive step size). It can fail when the tabular data lack approximate low-rank structure, when missingness is strongly MNAR and the global covariance warm start is biased, or on very high-dimensional or predominantly categorical tables where the SVD-based initialization degrades.

observed signal
0impl.
0stars
—citations
9.52xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39678
First page of Aletheia: Permission-Minimality Testing for Coding-Agent Rules
safety and guardrailstest

Aletheia: Permission-Minimality Testing for Coding-Agent Rules

It replaces manual review and static filtering of repository instruction files by executing the unchanged rule and task in synthesized sandboxes, removing one permission at a time to produce dispensability witnesses for suspicious authority requests. The cost is an executable sandbox environment, typed permission synthesis, and multiple full agent runs per rule under independent restrictions, which multiplies evaluation compute and adds pipeline complexity. It can fail when malicious behavior only manifests with permissions that functional tests also require, when functional tests are too weak to expose lost capability, or when benign rules are flagged (3.75% false positive rate), and the detection results come from a single shared refactoring task and one attack corpus.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39679
First page of SE-ADD: Self-Evolving Audio Deepfake Detection with Mistake-Driven Supervision
efficient trainingtest

SE-ADD: Self-Evolving Audio Deepfake Detection with Mistake-Driven Supervision

Replaces static, label-only fine-tuning of audio-language-model deepfake detectors with iterative LoRA updates where misclassified samples receive extra supervision from the model's own self-generated forensic cues. Costs are modest in memory (LoRA adapters fit on a 24 GB GPU) but the iterative verdict-and-cue regeneration loop adds training complexity and latency compared to a single fine-tune. It can fail if self-generated forensic cues are wrong, since supervising a model on its own errors risks reinforcing confident mistakes, and gains are reported only for two specific ALMs on the authors' evolving-attack benchmark.

observed signal
0impl.
0stars
—citations
4.88xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39801
First page of RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models
quantizationtest

RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models

RATIO replaces fixed, predefined overthinking-marker interventions with model-specific penalty tokens identified by comparing a quantized model against its full-precision counterpart (QRBA), with per-token penalties derived from full-precision guidance (TSPD) and no additional training. The cost is a one-time analysis pass requiring access to the full-precision model for calibration, plus a modified decoding loop; penalties must be recomputed per model and quantization configuration. It can fail when a full-precision reference is unavailable (as with API-only models), when the identified overthinking tokens do not transfer across tasks or quantization settings, or when accuracy gains do not replicate on the reader's specific model and workload, since results are benchmark-reported and code availability is prospective.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39816
First page of Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication
quantizationwatch

Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication

The method replaces the standard row-local int8 matmul kernel with a certified two-level Strassen realization on bounded integer codes that quantizes token rows independently and cancels exactly before rescaling. It costs implementation complexity: custom integer kernels are required, the claimed gain is a 49-versus-64 block-multiplication count rather than a measured wall-clock speedup, and the certificate only holds at the exact prescribed quantization specification. After adoption it can fail if the deployed hardware does not execute the integer arithmetic at the assumed precision, if quantization error at the int8 spec degrades task quality, or if accuracy-only evaluation masks the prefix-invariance violations the paper shows affect 5.83-10.00% of likelihood-scored answers in uncertified fast realizations.

observed signal
0impl.
0stars
—citations
1.31xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39933
First page of ConflictGuide: AutoResearch Improves When Competing Behaviors Are Made Visible
coding agentstest

ConflictGuide: AutoResearch Improves When Competing Behaviors Are Made Visible

ConflictGuide replaces single scalar task-performance feedback in LLM-driven AutoResearch loops with a two-stage regime: scalar exploration first, then probe-based feedback on competing model behaviors once gains plateau, with a reusable skill that identifies conflicts and specifies probes for a code agent to implement as metrics. It costs the manual or agent-assisted effort of identifying behavior conflicts and building probe metrics per model, plus additional evaluation compute per proposal, and it adds pipeline complexity in the form of a staged search schedule. It can fail when the chosen competing behaviors or probes are misidentified, when probe thresholds for retaining marginal-gain edits are miscalibrated, or in workloads where no meaningful behavior trade-off exists, in which case Stage II adds overhead without benefit.

observed signal
0impl.
121stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39957
First page of Learning When and How to Intervene: A Hindsight-Distilled Sentinel for Coding Agents
coding agentstest

Learning When and How to Intervene: A Hindsight-Distilled Sentinel for Coding Agents

Replaces reactive execution-feedback recovery and ad-hoc rule-based action checks with a lightweight 0.6B/1.7B classifier that, before each agent action, decides to allow, redirect, or escalate to a human and supplies corrective feedback. Costs an extra small-model inference per action plus the effort of obtaining or reproducing the SWE-Intervene hindsight-labeled trajectories and training the student, adding latency and an orchestration dependency to the agent loop. Can fail if the sentinel's intervention judgments do not transfer to the reader's repositories, task distribution, or agent family, and false interventions may block correct actions or inflate human escalation load.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.39972
First page of UBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decoding
speculative decodingtest

UBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decoding

UBTree replaces single-path autoregressive drafters (and prior parallel drafters such as DARTree) with a tree drafter in which a unigram proposer generates per-position candidates and a bigram selector, trained with renormalized KL on high-temperature data, scores transitions between adjacent candidates. The cost is training and serving two additional lightweight draft components, extra tree-construction and verification logic in the inference path, and the overhead of candidate scoring per step. The approach can fail on high-entropy workloads if the tree still lacks sufficient branch diversity, if the proposer-selector training distribution diverges from production traffic, or if per-step drafting overhead erodes the speedup at small batch sizes typical of a single 24 GB GPU.

observed signal
0impl.
121stars
—citations
6.94xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.40063
First page of LARC: Low-Rank Adaptive Residual Connections for Learning in Frozen Models
efficient trainingwatch

LARC: Low-Rank Adaptive Residual Connections for Learning in Frozen Models

LARC replaces full fine-tuning or prompt-based adaptation with a 12,288-parameter rank-4 residual (h+BAh) on a frozen 1B model, using a slow learned initialization and a fast state that updates from feedback and resets. The compute and memory cost is negligible, but it requires a feedback-gradient signal at inference time, reset-policy management, and a separate selection rule. It can fail through online-update degradation: retaining updates worsened downstream Brier loss, updates showed same-batch non-descent, and benefits were inconsistent across only three paired seeds.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗

Research signals

Three views of the same evidence: adoption against attention, publication cadence by research area, and reported gains when coverage is sufficient.

color = research area

01 · attention × adoption

The research frontier

Look to the upper left for independent implementation before mass attention.

compare by
Each point represents one paper. Higher means more independent implementations; farther right means more GitHub stars.The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents: 0 GitHub stars; 2 independent implementationsProtocol-Preserving Context Trimming for Agentic Workflows: Benefits, Failure Regimes, and Budget Guardrails: 3 GitHub stars; 2 independent implementationsDistributionally robust linear regression through the lens of adversarial training: 0 GitHub stars; 0 independent implementationsTowards Robust Time Series Learning via Capacity-Centric Modulation: 0 GitHub stars; 0 independent implementationsComparative study of adapting pre-trained models for driving behavior video captioning: 0 GitHub stars; 0 independent implementationsSpeculative Safety Honeypot: Toward Proactive Defense Against Multi-turn Agent Attacks: 10 GitHub stars; 0 independent implementationsRankEvolve: A Reliable Multi-Agent Auto-Research Harness for Evolving Ranking Models: 131 GitHub stars; 1 independent implementationA2Z GameSpec-Bench: How Faithfully Can Coding Agents Generate Games from Game Design Specifications?: 136 GitHub stars; 3 independent implementationsThinking Outside the Box: Can Language Models Rely on External Guidance Selectively?: 121 GitHub stars; 1 independent implementationGroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives: 214 GitHub stars; 0 independent implementationsHybrid Methods for Robust Tabular Data Imputation: 0 GitHub stars; 0 independent implementationsAletheia: Permission-Minimality Testing for Coding-Agent Rules: 0 GitHub stars; 0 independent implementationsSE-ADD: Self-Evolving Audio Deepfake Detection with Mistake-Driven Supervision: 0 GitHub stars; 0 independent implementationsMemCodex: Self-Programming Hierarchical Memory for Language Agents: 131 GitHub stars; 1 independent implementationRATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models: 0 GitHub stars; 0 independent implementationsBeyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication: 0 GitHub stars; 0 independent implementationsOSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software: 135 GitHub stars; 3 independent implementationsConflictGuide: AutoResearch Improves When Competing Behaviors Are Made Visible: 121 GitHub stars; 0 independent implementationsLearning When and How to Intervene: A Hindsight-Distilled Sentinel for Coding Agents: 0 GitHub stars; 0 independent implementationsBetter Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking: 131 GitHub stars; 1 independent implementationUBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decoding: 121 GitHub stars; 0 independent implementationsMid-Harness: Scaling Actions Between Model and Harness for Terminal Agents: 131 GitHub stars; 3 independent implementationsWho Verifies the Graph? Misspecification Attacks on Causal Action Verification for Language Agents: 131 GitHub stars; 1 independent implementationLARC: Low-Rank Adaptive Residual Connections for Learning in Frozen Models: 0 GitHub stars; 0 independent implementationsAccelerated Algorithm for Sparse Regularized Partial Optimal Transport: 0 GitHub stars; 0 independent implementationsMeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversion: 1 GitHub star; 0 independent implementationsAutoDataBench: A Data-centric Testbed for Accelerating Auto Research: 123 GitHub stars; 1 independent implementationJuryFlow: Disagreement-Guided Human-in-the-Loop Multi-Agent Evaluation: 10 GitHub stars; 0 independent implementationsAgent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training: 131 GitHub stars; 2 independent implementationsPersistent Context Graphs for Efficient Memory Compaction in LLM Agents: 131 GitHub stars; 1 independent implementationLearning Functional Subspaces for Neural Network Compression: 1 GitHub star; 1 independent implementationReinforcement Learning-Guided Graph Transformations for SpTRSV Optimization: 1 GitHub star; 0 independent implementationsLearning from Research: Toward Lifelong Agent Harness Evolution: 131 GitHub stars; 1 independent implementationSCB: SpeechConversationBench for Evaluating Multi-Turn Reasoning in Speech-to-Speech Models: 1 GitHub star; 0 independent implementationsPhantomEnvironments: Training LLM Agents in Fictional Worlds: 10 GitHub stars; 0 independent implementationsComputerSD: Online Self-Distillation from Real-Time Feedback for Computer-Use Agents: 2 GitHub stars; 0 independent implementationsOpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoning: 1 GitHub star; 0 independent implementationscua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents: 10 GitHub stars; 0 independent implementationsHow Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?: 131 GitHub stars; 1 independent implementationCompression Footprints as Security Signals for Model-Poisoning Defense in Federated Learning: 1 GitHub star; 0 independent implementationsScaling Laws for Looped Mixture of Experts: 0 GitHub stars; 0 independent implementationsCogentic: Multi-Agent Orchestration for Automated Proof Discovery: 10 GitHub stars; 0 independent implementations

Papers above 1000 stars or 200 citations are not scored. They have already broken through the attention threshold, while Paperraft is designed to identify earlier signals.

02 · cadence

Research areas over time

Compare monthly publication volume on a shared scale and calendar.

coding agentsmax 7Jan 26: 0 papersSep 26: 3 papersJan 26Sep 26efficient architecturesmax 7Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26agent evaluationmax 7Jan 26: 1 paperSep 26: 7 papersJan 26Sep 26speculative decodingmax 7Jan 26: 0 papersSep 26: 1 paperJan 26Sep 26distillationmax 7Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26memory and contextmax 7Jan 26: 0 papersSep 26: 3 papersJan 26Sep 26multi-agent orchestrationmax 7Jan 26: 0 papersSep 26: 3 papersJan 26Sep 26othermax 7Jan 26: 0 papersSep 26: 6 papersJan 26Sep 26quantizationmax 7Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26failure recoverymax 7Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26safety and guardrailsmax 7Jan 26: 0 papersSep 26: 4 papersJan 26Sep 26efficient trainingmax 7Jan 26: 0 papersSep 26: 5 papersJan 26Sep 26tool usemax 7Jan 26: 0 papersSep 26: 1 paperJan 26Sep 26

Every panel uses the same calendar and vertical scale. An empty column means no papers were recorded.

Method under scrutiny

Inspect the highest-ranked paper, every repository behind its signal, and the rule used to classify each one.

The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents

agent evaluation · test · 2 of 2 implementations are independent

Replaces single-language neutrality evaluation with semantically paired prompts in different languages. It needs paired prompts and extra API calls, with human interpretation because contested questions lack a definitive answer key. The study reports that ChatGPT 5.2 and Claude Opus 4.5 adopt the prompt language's dominant framing, but examines a single Ukrainian document and does not establish generalization to other language pairs or topics.

Exclusions and controls

Every exclusion is counted and published.

Exclusion counts were not recorded for this edition.

0 previously indexed papers were rechecked in this run.

Paperraft tracks one signal: how many independent implementations a paper attracts on GitHub after author-owned repositories are removed. This is a discovery signal to investigate, not a measure of implementation quality.

It does not claim that a method works. Paperraft runs no reproduction benchmarks. Reported gains remain author claims until independently tested. Papers above the attention threshold are excluded because this publication is designed to identify research before consensus forms.