Skip to content

archived edition · 2026-09-21

PaperraftFind the AI research
worth testing.

Understand what a paper changes, what it costs, and whether it fits your next build. Explore research on faster LLM inference, agent memory, and evaluation, with the original sources in reach.

Explore the researchHow papers are assessed

Start with a brief. Follow the evidence. Choose a test.

edition
2026-09-21
briefs
42
research areas
16
independent implementations
20

31 of 42 papers in the index (74%) have no independent implementations.

Research index

Search by topic, browse a research area, or sort by date. Read a brief, open the original paper, and inspect the available evidence before deciding what to test.

showing30 of 42
sort by
arXiv 2601.12164
First page of The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents
agent evaluationtest

The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents

Replaces single-language neutrality evaluation with semantically paired prompts in different languages. It needs paired prompts and extra API calls, with human interpretation because contested questions lack a definitive answer key. The study reports that ChatGPT 5.2 and Claude Opus 4.5 adopt the prompt language's dominant framing, but examines a single Ukrainian document and does not establish generalization to other language pairs or topics.

observed signal
2impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.19122
First page of Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation
efficient architecturestest

Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation

The method replaces manually tuned, fixed sparsity coefficients in convolutional sparse coding front-ends with a differentiable coefficient learned jointly with the network via unfolded FISTA iterations, plus a label-free post-training pass that readjusts compression for corrupted inputs. It costs additional inference compute from the iterative FISTA unrolling inside the network, extra training complexity, and an added reconstruction objective that must be balanced against task loss. It can fail if the learned compression strength does not transfer to corruption types unseen during the post-training adjustment, if the iterative unrolling inflates latency beyond budget, or if gains shown on CIFAR/ImageNet classification do not generalize to the reader's task.

observed signal
2impl.
1stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.14987
First page of ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents
safety and guardrailstest

ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents

ActGuard replaces coarse prompt-hardening and content-filtering defenses with a per-step audit that predicts a local tool prior, flags deviations in tool choice and parameters, and masks only evidence-verified malicious spans before regenerating the action. Its cost is additional inference at every agent step: a tool-prior prediction, contrastive analysis, and a verifier pass, which increases latency and API spend roughly proportional to trajectory length. It can fail when the injected instruction produces an action consistent with the tool prior (parameter-only manipulation that evades localization), when the verifier over- or under-masks evidence, or when legitimate but unusual tool sequences are treated as deviations.

observed signal
2impl.
2stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.16461
First page of Protocol-Preserving Context Trimming for Agentic Workflows: Benefits, Failure Regimes, and Budget Guardrails
memory and contexttest

Protocol-Preserving Context Trimming for Agentic Workflows: Benefits, Failure Regimes, and Budget Guardrails

The method replaces unconstrained context growth and naive trimming (recency, relevance, or summarization-based) with protocol-aware selection plus adaptive retained-context budgets that protect protocol-critical state in agentic workflows. It costs implementation complexity: the system must identify protocol-critical state (instructions, tool states, unresolved dependencies) and maintain budget-enforcement logic, and it retains roughly 44% more tokens than the most aggressive conventional trimming. Failure risk concentrates below 25% retained-context budgets, where failure odds rise 10.92-fold, and critical thresholds increase with workflow complexity, so a static configuration can silently degrade on harder tasks.

observed signal
2impl.
2stars
0citations
2.11xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.22000
First page of RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents
agent evaluationwatch

RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

RecreationWorld replaces ad hoc, single-modality CUA evaluation with a five-platform benchmark where a running reference application acts as an oracle for hidden behavioral tests. Adoption costs substantial infrastructure: reproducible Ubuntu, macOS, Windows, Android, and Web environments, a unified GUI-plus-code harness, and trajectory generation at scale, none of which fits a single 24 GB GPU or a limited cloud budget. It can fail as a decision signal because reference-grounded assertions may overfit to the frozen test suites, and the low 2.8% full-pass rate indicates the benchmark mainly discriminates frontier-scale agents rather than small-team systems.

observed signal
3impl.
46stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21967
First page of NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities
tool usewatch

NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities

The technique replaces the conventional cascaded voice-agent pipeline (streaming ASR, LLM with tool calling, and TTS chained as separate services) with a single unified full-duplex speech-to-speech model that handles interruption, backchanneling, and function calls natively. It costs the deployment burden of a multi-component streaming model (speech encoder, decoder LM, RNN-T branch, streaming TTS decoder) whose total footprint is not stated in the abstract and may exceed a single 24 GB GPU, plus the engineering cost of integrating streaming audio I/O and tool runtimes. It can fail on tool execution specifically, since the authors state argument accuracy and end-to-end tool execution remain areas for improvement, and any regression in full-duplex timing degrades conversational quality in ways the text-only stack would not.

observed signal
2impl.
121stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.20995
First page of Voice-Light: A Full-Duplex Cascaded Voice Agent with Causal Turn-Taking and Speculative Generation
othertest

Voice-Light: A Full-Duplex Cascaded Voice Agent with Causal Turn-Taking and Speculative Generation

This replaces a half-duplex ASR-LLM-TTS pipeline with a cascaded full-duplex stack combining streaming ASR with a shared causal adapter, speculative private response generation, reversible playback, concurrent tool calls with bridge speech, and browser acknowledgments that gate durable history. The cost is substantial integration complexity: shared encoders, playback reversal, concurrency control, and history-authority logic must all be engineered and maintained, with latency gains unproven against a controlled baseline. The learned turn-taking model failed its own locked evaluation (12.53% end-of-turn recall), so the system depends on a heuristic Silero timing policy that can still cut off speakers (2.70% false-cutoff rate) and may misbehave on real user populations.

observed signal
2impl.
678stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.22068
First page of CodeMidas: Scaling Agentic Coding RL Environments from Code Itself
coding agentswatch

CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

The method replaces RL environment construction that depends on development artifacts (issues, commits) with an agentic pipeline that derives behavioral specifications, tests, and validated tasks directly from source code. It costs substantial agentic compute at every construction stage plus the infrastructure to run GRPO training on a large model, which exceeds a single 24 GB GPU and a limited budget. It can fail through unreliable auto-generated verifiers, tasks that pass spuriously under repeated rollouts, and quality degradation if the execution-based filtering is insufficient.

observed signal
2impl.
130stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.20997
First page of MOSAIC-SR: Transformer-Guided Symbolic Regression for Scientific Equation Recovery
othernot applicable

MOSAIC-SR: Transformer-Guided Symbolic Regression for Scientific Equation Recovery

MOSAIC-SR replaces randomly initialized combinatorial search for equation recovery with a pretrained Transformer that proposes candidate sketches, which are then refined by scale-aware constant optimization and local symbolic repair. The cost is running the pretrained proposer plus several parallel local searches, so it is more expensive per query than pure neural generation and requires maintaining the search and optimization stack. It can fail when the true equation lies outside the Transformer's learned prior, when constants are numerically ill-conditioned, or when the pretrained model was trained on a different distribution of expressions than the target domain.

observed signal
1impl.
121stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21109
First page of Talk to Me, Jarvis: An Open-Source Edge-Deployable Voice Assistant Framework for Autonomous Racecars
efficient trainingadopt

Talk to Me, Jarvis: An Open-Source Edge-Deployable Voice Assistant Framework for Autonomous Racecars

Replaces network-dependent, online-hosted large models for voice command intent classification with a locally deployed Mistral 7B fine-tuned on a domain-specific command dataset, removing network dependency and variable API latency. Costs one-time fine-tuning effort and enough local compute to serve a 7B model (roughly 5-16 GB depending on quantization), plus ongoing dataset maintenance as the command set evolves; latency of about 1.4 s end-to-end is acceptable for high-level commands but not for reflex-level control. Can fail on out-of-distribution utterances, accents or noisy audio degrading the upstream speech recognizer, and command classes not covered in the fine-tuning data, with no fallback behavior specified if the classifier mispredicts.

observed signal
1impl.
130stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21378
First page of ArenaFlow: From Trajectory Ranking to Hierarchical Credit Propagation for Open-Ended Agent RL
otherwatch

ArenaFlow: From Trajectory Ranking to Hierarchical Credit Propagation for Open-Ended Agent RL

ArenaFlow replaces single trajectory-level scalar rewards in open-ended agent RL with tournament-based relative ranking, reflective step-level credit propagation, and a utility-maintained skill memory used as a policy prior. It costs substantial extra inference per training iteration (pairwise tournament comparisons plus structured reflective LLM evaluations), a full RL training loop, and engineering for skill storage, attribution, pruning, and retrieval, all of which exceed a single 24 GB GPU and a limited API budget. It can fail through judge-induced reward bias in pairwise evaluation, misattributed credit to spurious pivotal steps, and skill-memory drift if utility estimates reward frequent-but-low-value skills.

observed signal
1impl.
130stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.14759
First page of Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Families
safety and guardrailswatch

Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Families

The paper does not propose a deployable technique; it is a causal interpretability study showing that refusal behavior in aligned models is implemented as a shallow, low-rank post-training addition that reads a narrow harm direction rather than the model's broader moral comprehension, which explains why a rank-one activation edit can strip refusal from open-weight models. There is nothing to adopt or pay for directly, but the operational implication is that a production system cannot rely on a fine-tuned open-weight model's internal refusal as its only safety layer, since that layer is editable and reads only a slice of what the model knows. What can fail is exactly this: a downstream fine-tune, adapter, or activation intervention can silently remove refusal while leaving general capability and moral comprehension intact, so the failure mode is undetectable through ordinary quality evaluation.

observed signal
0impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.14795
First page of Mind Which Bird You Favour: Parameterizing Adequacy-Fluency Balance in Meta-Evaluation of Machine Translation
agent evaluationwatch

Mind Which Bird You Favour: Parameterizing Adequacy-Fluency Balance in Meta-Evaluation of Machine Translation

The method replaces uniform weighting of translation systems in MT meta-evaluation datasets with optimized per-system weights that achieve a chosen adequacy-fluency balance while staying close to uniform. It costs an exact optimization step with pruning over the system set, plus a scorer-augmentation validation framework, adding methodological complexity but negligible compute. It can fail if the available system pool is too small or skewed to support the target balance without distorting weights, and its conclusions do not transfer directly to non-MT evaluation settings.

observed signal
0impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.14985
First page of Converting Sequenced Fuzzy Cognitive Maps to Causal Virtual Worlds with Large Video Generators
multi-agent orchestrationnot applicable

Converting Sequenced Fuzzy Cognitive Maps to Causal Virtual Worlds with Large Video Generators

The method replaces hand-authored scripting and storyboarding of video scenes with an orchestrated pipeline in which a fuzzy cognitive map encodes causal rules, its feedback dynamics are converted into if-then meta-rules, an LLM writes a script from them, and a large video model renders each scene. It costs nothing locally in model training but depends entirely on paid third-party APIs (Gemini 3.1, Veo 3.1), adds the complexity of authoring and validating an FCM per world, and offers no measurable quality, speed, or cost advantage over direct prompting. Failure modes include an incorrectly specified FCM producing incoherent causal sequences, the LLM mis-translating meta-rules into the script, and the video generator not preserving causal or entity consistency across scenes, none of which are evaluated in the paper.

observed signal
0impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.14992
First page of MTAC-IFBench: Benchmarking Instruction-Following in Multi-Turn Agentic Coding
agent evaluationtest

MTAC-IFBench: Benchmarking Instruction-Following in Multi-Turn Agentic Coding

MTAC-IFBench replaces ad hoc or single-turn evaluation of coding agents with a multi-turn benchmark averaging 7.04 turns and 91.33 constraints per instance, using per-item checklists verified by scripts and judge agents. Adopting it costs benchmark setup, API or GPU inference for the agents under test, and reliance on judge agents whose own reliability can distort scores. It can fail as an adoption decision if the reader's production workload differs from its 6 primary and 18 secondary constraint categories, since reported degradation over long sessions may not transfer.

observed signal
0impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.15011
First page of Semantic-TVM: Structure-Preserving Trustworthy Virtual Memory for Memory-Augmented and Tool-Using Agents
safety and guardrailstest

Semantic-TVM: Structure-Preserving Trustworthy Virtual Memory for Memory-Augmented and Tool-Using Agents

The method replaces whole-field masking of private values sent to remote LLMs with replacement of only the sensitive spans identified by a trusted local model, keeping exact values in a local runtime that re-injects them for tool execution. It costs a local trusted model for span prediction, added runtime complexity for handle resolution and closed-loop value restoration, and residual latency from the local inference step. It can fail if the local span predictor misses sensitive values (exposure), over-masks (utility loss approaching Rule-TVM), or if later observations re-leak values the projection assumed were protected.

observed signal
0impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.15013
First page of Overflip: Repetition-Induced Label Flips in Guardrail Models
safety and guardrailstest

Overflip: Repetition-Induced Label Flips in Guardrail Models

Overflip replaces attention-dilution attacks (padding, shuffling) with simple prompt repetition to flip compact guardrail classifiers from malicious to benign, exploiting bucketed positional encodings beyond 512-token training windows. It costs the attacker nothing beyond extra tokens, and defending costs the reader additional evaluation and mitigation work such as repetition detection, input length limits, or chunking. It fails against length-robust or long-context guardrails (4 of 9 tested models were unaffected), and mitigations like naive truncation may reintroduce evasion or break legitimate long inputs.

observed signal
0impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.20883
First page of Sparse Priors for Efficient Distribution Learning
othernot applicable

Sparse Priors for Efficient Distribution Learning

This is a statistical learning theory paper that provides Bayesian risk bounds (Omega(sqrt(k/n)) lower bound, matching upper bound in TV) for distribution learning under a hypothesized k-sparse prior, rather than an implementable method that replaces any production component. It proposes no algorithm, model, kernel, or serving technique, so there is no memory, latency, or integration cost to assess for deployment. Adoption risk is moot; the practical failure mode would be misreading theoretical sample-complexity bounds as evidence that a deployed generative model will escape the curse of dimensionality without verifying that the data actually satisfies the sparse-prior assumption.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.20888
First page of Elastic Threshold Attention: Learned Contextual Sparsity for Long-Context Decoding
attention and kernelswatch

Elastic Threshold Attention: Learned Contextual Sparsity for Long-Context Decoding

ETA replaces fixed-heuristic sparse attention (and dense FlashAttention decoding) with per-query learned thresholds that hard-prune KV blocks, trained via multiplicative suppression of sub-threshold logits rather than token deletion. The cost is substantial: the policy must be learned during pretraining, inference requires a custom Triton decode kernel, and deployment-quality thresholds need offline per-domain calibration to remove predictor overhead. It can fail on workloads unlike the training distribution, where predicted thresholds may prune context needed for retrieval or reasoning, and the reported quality parity is demonstrated only at 1.45B scale, so transfer to larger or API-served models is unverified.

observed signal
0impl.
120stars
—citations
2.5xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.20957
First page of An Approximate Queueing Model of LLM Inference Serving for SLO-Driven Autoscaling
serving and batchingwatch

An Approximate Queueing Model of LLM Inference Serving for SLO-Driven Autoscaling

It replaces reactive or throughput-only autoscalers with an approximate three-parameter queueing model that predicts TTFT and ITL to set replica counts against explicit latency SLOs. The cost is per-model-accelerator parameter calibration from latency measurements, a state-dependent Markov chain solver in the control loop, and the multi-replica GPU cluster the controller exists to manage. It can fail when load exceeds the validated light-to-moderate regime, when Markovian assumptions break under bursty or non-stationary traffic, or when parameters drift across model, hardware, or serving-stack changes.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.20981
First page of CaLR: Causal Latent Revision for Robust Diffusion Reasoning
planning and decompositionwatch

CaLR: Causal Latent Revision for Robust Diffusion Reasoning

CaLR replaces standard autoregressive step-by-step reasoning with parallel diffusion-based generation augmented by gradient-guided revision of intermediate latent states, using a causal topology matrix distilled from an expert model and implicit differentiation to enforce logical consistency. The cost is substantial: it presupposes a diffusion language model backbone (not a mainstream API-served model), an expert model to supply the causal topology, and additional optimization machinery at inference time, none of which fits a single 24 GB GPU plus API budget without dedicated training. It can fail through dependence on the quality and transferability of the expert-derived causal structure, sensitivity of implicit-differentiation-based revision to hyperparameters, and the general immaturity of diffusion LM tooling relative to autoregressive serving stacks.

observed signal
0impl.
120stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21039
First page of Stiefel-AdamW: Geometry-Aware AdamW for Linear Factorization Blocks
efficient trainingtest

Stiefel-AdamW: Geometry-Aware AdamW for Linear Factorization Blocks

Stiefel-AdamW replaces AdamW specifically for the factor matrices of linear factorization blocks (LoRA adapters, low-rank layers, query-key products), constraining one factor to the Stiefel manifold while keeping AdamW's diagonal preconditioning in ambient space with only a tangent projection and retraction added. The claimed cost is minimal implementation and compute overhead over AdamW, with no extra memory of significance, fitting a single 24 GB GPU workflow. Failure modes include the unquantified gains not transferring to the reader's specific fine-tuning workload, retraction/projection bugs in custom implementations, and possible interaction issues with existing training stacks (gradient clipping, mixed precision, sharded optimizers) that the paper's benchmark setup may not cover.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21126
First page of Layerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers
sparsity and pruningtest

Layerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers

The method replaces joint, coupled penalization of all layers with a sequential layerwise scheme that extracts shallow two-layer subnetworks, normalizes inner weights, and applies a structured group penalty to prune neurons and reduce layer width. The cost is additional implementation complexity in the projected/proximal formulation and an offline pruning pass per layer, with no claimed reduction in inference latency or deployment memory unless the pruned widths are exploited by downstream kernels. It can fail when the activation is not positively homogeneous (the equivalence proof depends on this), when regularization strengths transfer poorly across architectures, or when structured width reduction in FFNs does not translate into actual hardware speedup.

observed signal
0impl.
1stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21137
First page of A Multi-Engine Dataflow for MoE Decoding on Scratchpad-Based Tensor Accelerators
quantizationwatch

A Multi-Engine Dataflow for MoE Decoding on Scratchpad-Based Tensor Accelerators

The method replaces dense BF16 expert-weight streaming during MoE decoding with a vector-quantized plus shared-basis low-rank representation, co-designed with a multi-engine dataflow that overlaps DMA transfers with compute on scratchpad-based accelerators. The cost is a custom weight conversion and a hardware-specific decoding pipeline tied to AWS Trainium3's scratchpad and multi-engine architecture, plus implementation complexity well beyond a standard inference stack. It can fail to transfer: the speedups depend on Trainium3's DMA and engine structure, the quality results are measured only against perplexity, and neither the representation nor the dataflow is available on a single 24 GB GPU or through third-party APIs.

observed signal
0impl.
0stars
—citations
1.31xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21145
First page of Scaling Forced Alignment to End-User Devices
otheradopt

Scaling Forced Alignment to End-User Devices

The method replaces quadratic-time-and-space Viterbi forced alignment (as in torchaudio) with a Hirschberg-based in-place variant plus constrained-random-walk search-space pruning, enabling alignment of multi-hour audio on CPU-class hardware. The cost is implementation complexity beyond a standard Viterbi decoder and reliance on an existing acoustic model to supply frame-level emissions; the alignment quality itself is unchanged without pruning. With pruning enabled, alignment accuracy can degrade in roughly 2% of tested cases, particularly where transcriptions contain errors that the random-walk model does not capture.

observed signal
0impl.
0stars
—citations
28000xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21157
First page of Can Agents Design Better Chips with a Higher Level Abstraction?
othernot applicable

Can Agents Design Better Chips with a Higher Level Abstraction?

The method replaces direct LLM-agent generation of RTL with a workflow in which agents write high-level synthesis (HLS) code and then refine the post-synthesis RTL, targeting hardware chip design rather than model inference or training. It costs the additional toolchain complexity of an HLS compiler plus a refinement loop, and requires access to FPGA or ASIC synthesis infrastructure rather than just GPU inference capacity. It can fail on designs where HLS abstractions prevent the agent from reaching lower-level optimizations, and the reported gains are limited to an 11-task FPGA benchmark that may not transfer to production silicon design.

observed signal
0impl.
10stars
—citations
2.6xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21172
First page of TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching
KV cachewatch

TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching

TierKV replaces reactive token eviction, uniform low-rank compression, and naive flash offloading with a pre-decode predictor that assigns tokens to exact, low-rank, or flash tiers via a closed-form solver under memory and accuracy budgets. It costs a prediction pass over prefill hidden states, runtime solver integration into the inference stack, and a reported minor accuracy degradation from low-rank and offloaded tiers. It can fail if the demand predictor mispredicts which tokens matter, and its gains are demonstrated on mobile SoCs, so throughput and memory results may not transfer to a 24 GB GPU serving setup.

observed signal
0impl.
0stars
—citations
17.6xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21257
First page of Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale
agent evaluationwatch

Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale

EvoPilot replaces ungated autoresearch loops, in which LLM agents propose, implement, and evaluate model changes autonomously, with role-specific agents whose rounds pass through versioned domain skills, typed adapters, durable records, and deterministic checks gated by human review. The cost is substantial supporting infrastructure: durable experiment state, replay and mutation testing of comparisons, deterministic enforcement of recorded lessons, and an hourly refreshed index over hundreds of millions of videos, sustained over a 37-day campaign. It can fail when the verification harness itself misses defects such as no-op code changes, data-window leakage, evaluator semantic drift, or divergent serving funnels, and its gating discipline adds human latency to every iteration.

observed signal
0impl.
10stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21267
First page of Efficient Benchmarking in Production: A Study of an Evolving LLM Agent
agent evaluationadopt

Efficient Benchmarking in Production: A Study of an Evolving LLM Agent

The method replaces full benchmark reruns for recurring production-agent evaluation with a small fixed subset of questions stratified by difficulty, selected once from historical runs. It costs a one-time calibration over past evaluation data and accepts a small score-estimation error (around 1 percentage point MAE for the best variant) in exchange for roughly 60% fewer executions per evaluation cycle. It can fail if the agent's behavior drifts so that the fixed subset's difficulty profile no longer discriminates, or if historical calibration data are unavailable for a new agent family, although the paper reports transfer across five families and stability with one-day calibration windows.

observed signal
0impl.
10stars
—citations
2.6xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.21341
First page of What Stops a Small Language Model From Driving a Database Agent
tool usetest

What Stops a Small Language Model From Driving a Database Agent

The work replaces the assumption that small-model agent failures reflect reasoning deficits with a failure-attribution methodology: capture model tool-call arguments (production ledgers record only refusal codes), classify losses into transport, capability, and related classes, and fix server-side defects before blaming the model. The cost is infrastructure rather than compute: logging tool arguments, building a scorer and failure taxonomy, and running clustered resampling by model, plus attention to a confound where an uncapped 262k-token context window let a 7.1 GB model hold 51 GB and produce runs indistinguishable from timeouts. After adoption, the main risks are that the transport-over-capability ordering is corpus-specific (holding in only 74.5% of model-clustered resamples) and that conclusions drawn without argument-level logging will misattribute server defects to model incapacity.

observed signal
0impl.
0stars
—citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗

Research signals

Three views of the same evidence: adoption against attention, publication cadence by research area, and reported gains when coverage is sufficient.

color = research area

01 · attention × adoption

The research frontier

Look to the upper left for independent implementation before mass attention.

compare by
Each point represents one paper. Higher means more independent implementations; farther right means more GitHub stars.The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents: 0 GitHub stars; 2 independent implementationsRefusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Families: 0 GitHub stars; 0 independent implementationsMind Which Bird You Favour: Parameterizing Adequacy-Fluency Balance in Meta-Evaluation of Machine Translation: 0 GitHub stars; 0 independent implementationsConverting Sequenced Fuzzy Cognitive Maps to Causal Virtual Worlds with Large Video Generators: 0 GitHub stars; 0 independent implementationsActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents: 2 GitHub stars; 2 independent implementationsMTAC-IFBench: Benchmarking Instruction-Following in Multi-Turn Agentic Coding: 0 GitHub stars; 0 independent implementationsSemantic-TVM: Structure-Preserving Trustworthy Virtual Memory for Memory-Augmented and Tool-Using Agents: 0 GitHub stars; 0 independent implementationsOverflip: Repetition-Induced Label Flips in Guardrail Models: 0 GitHub stars; 0 independent implementationsProtocol-Preserving Context Trimming for Agentic Workflows: Benefits, Failure Regimes, and Budget Guardrails: 2 GitHub stars; 2 independent implementationsTraining-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation: 1 GitHub star; 2 independent implementationsSparse Priors for Efficient Distribution Learning: 0 GitHub stars; 0 independent implementationsElastic Threshold Attention: Learned Contextual Sparsity for Long-Context Decoding: 120 GitHub stars; 0 independent implementationsAn Approximate Queueing Model of LLM Inference Serving for SLO-Driven Autoscaling: 0 GitHub stars; 0 independent implementationsCaLR: Causal Latent Revision for Robust Diffusion Reasoning: 120 GitHub stars; 0 independent implementationsVoice-Light: A Full-Duplex Cascaded Voice Agent with Causal Turn-Taking and Speculative Generation: 678 GitHub stars; 2 independent implementationsMOSAIC-SR: Transformer-Guided Symbolic Regression for Scientific Equation Recovery: 121 GitHub stars; 1 independent implementationStiefel-AdamW: Geometry-Aware AdamW for Linear Factorization Blocks: 0 GitHub stars; 0 independent implementationsTalk to Me, Jarvis: An Open-Source Edge-Deployable Voice Assistant Framework for Autonomous Racecars: 130 GitHub stars; 1 independent implementationLayerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers: 1 GitHub star; 0 independent implementationsA Multi-Engine Dataflow for MoE Decoding on Scratchpad-Based Tensor Accelerators: 0 GitHub stars; 0 independent implementationsScaling Forced Alignment to End-User Devices: 0 GitHub stars; 0 independent implementationsCan Agents Design Better Chips with a Higher Level Abstraction?: 10 GitHub stars; 0 independent implementationsTierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching: 0 GitHub stars; 0 independent implementationsVerify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale: 10 GitHub stars; 0 independent implementationsEfficient Benchmarking in Production: A Study of an Evolving LLM Agent: 10 GitHub stars; 0 independent implementationsWhat Stops a Small Language Model From Driving a Database Agent: 0 GitHub stars; 0 independent implementationsArenaFlow: From Trajectory Ranking to Hierarchical Credit Propagation for Open-Ended Agent RL: 130 GitHub stars; 1 independent implementationUnderstanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals: 120 GitHub stars; 0 independent implementationsHE-Guardrail: A Homomorphic Guardrail Against Jailbreak Attacks for Encrypted Large Language Model Inference: 0 GitHub stars; 0 independent implementationsWhat Must Survive? Exact Task-Information--State Frontiers for Resource-Sufficient Learning: 0 GitHub stars; 0 independent implementationsGameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions: 10 GitHub stars; 0 independent implementationsWeighted Quantum Signal Processing: Low-Depth Polynomial Approximation with Applications to Kolmogorov-Arnold Networks: 1 GitHub star; 0 independent implementationsMulti-Domain Clustering via Measure Quantization: 0 GitHub stars; 0 independent implementationsCIPL: A Channel-Aware Framework for Recoverable Privacy Leakage in LLM Agents: 10 GitHub stars; 0 independent implementationsSpecQuant: Speculative Decoding with Multi-Parent Quantization for Adaptive LLM Inference: 120 GitHub stars; 0 independent implementationsRheoSampling: Resolving the One-Hot Dilemma in Stochastic Dynamic-Tree Speculative Decoding: 120 GitHub stars; 0 independent implementationsEnterpriseVal: Quantifying the Efficacy, Reliability and Value of Generative AI in the Enterprise: 0 GitHub stars; 0 independent implementationsAutoViewMem: Self-Configuring Orthogonal Views for Conversational Long-Term Memory: 120 GitHub stars; 0 independent implementationsNemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities: 121 GitHub stars; 2 independent implementationsBayesian Belief Layer for Controllable Opinion Dynamics in LLM Agents: 10 GitHub stars; 0 independent implementationsRecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents: 46 GitHub stars; 3 independent implementationsCodeMidas: Scaling Agentic Coding RL Environments from Code Itself: 130 GitHub stars; 2 independent implementations

Papers above 1000 stars or 200 citations are not scored. They have already broken through the attention threshold, while Paperraft is designed to identify earlier signals.

02 · cadence

Research areas over time

Compare monthly publication volume on a shared scale and calendar.

coding agentsmax 8Jan 26: 0 papersSep 26: 1 paperJan 26Sep 26efficient architecturesmax 8Jan 26: 0 papersSep 26: 1 paperJan 26Sep 26agent evaluationmax 8Jan 26: 1 paperSep 26: 7 papersJan 26Sep 26KV cachemax 8Jan 26: 0 papersSep 26: 1 paperJan 26Sep 26speculative decodingmax 8Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26sparsity and pruningmax 8Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26attention and kernelsmax 8Jan 26: 0 papersSep 26: 1 paperJan 26Sep 26memory and contextmax 8Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26multi-agent orchestrationmax 8Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26othermax 8Jan 26: 0 papersSep 26: 8 papersJan 26Sep 26planning and decompositionmax 8Jan 26: 0 papersSep 26: 1 paperJan 26Sep 26quantizationmax 8Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26safety and guardrailsmax 8Jan 26: 0 papersSep 26: 6 papersJan 26Sep 26serving and batchingmax 8Jan 26: 0 papersSep 26: 1 paperJan 26Sep 26efficient trainingmax 8Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26tool usemax 8Jan 26: 0 papersSep 26: 2 papersJan 26Sep 26

Every panel uses the same calendar and vertical scale. An empty column means no papers were recorded.

Method under scrutiny

Inspect the highest-ranked paper, every repository behind its signal, and the rule used to classify each one.

The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents

agent evaluation · test · 2 of 2 implementations are independent

Replaces single-language neutrality evaluation with semantically paired prompts in different languages. It needs paired prompts and extra API calls, with human interpretation because contested questions lack a definitive answer key. The study reports that ChatGPT 5.2 and Claude Opus 4.5 adopt the prompt language's dominant framing, but examines a single Ukrainian document and does not establish generalization to other language pairs or topics.

Exclusions and controls

Every exclusion is counted and published.

Exclusion counts were not recorded for this edition.

0 previously indexed papers were rechecked in this run.

Paperraft tracks one signal: how many independent implementations a paper attracts on GitHub after author-owned repositories are removed. This is a discovery signal to investigate, not a measure of implementation quality.

It does not claim that a method works. Paperraft runs no reproduction benchmarks. Reported gains remain author claims until independently tested. Papers above the attention threshold are excluded because this publication is designed to identify research before consensus forms.