Skip to content

archived edition · 2026-09-13

PaperraftFind the AI research
worth testing.

Understand what a paper changes, what it costs, and whether it fits your next build. Explore research on faster LLM inference, agent memory, and evaluation, with the original sources in reach.

Explore the researchHow papers are assessed

Start with a brief. Follow the evidence. Choose a test.

edition
2026-09-13
briefs
6
research areas
3
independent implementations
6

3 of 6 papers in the index (50%) have no independent implementations.

Research index

Search by topic, browse a research area, or sort by date. Read a brief, open the original paper, and inspect the available evidence before deciding what to test.

showing6 of 6
sort by
arXiv 2601.12164
First page of The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents
agent evaluationtest

The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents

Replaces single-language neutrality evaluation with semantically paired prompts in different languages. It needs paired prompts and extra API calls, with human interpretation because contested questions lack a definitive answer key. The study reports that ChatGPT 5.2 and Claude Opus 4.5 adopt the prompt language's dominant framing, but examines a single Ukrainian document and does not establish generalization to other language pairs or topics.

observed signal
2impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2605.13790
First page of Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
othernot applicable

Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations

Replaces classical numerical and neural PDE solvers under fewer than 3% observations with a VAE and latent diffusion. Training requires the full VAE, diffusion, and contrastive framework using domain-specific simulation data. It is relevant to physical simulation and PDEs and depends on domain data and validation that a general LLM engineer may not have.

observed signal
3impl.
45stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.03430
First page of Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
KV cacheadopt

Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

Random Attention replaces score-based KV cache eviction policies (such as H2O-style importance scoring) with uniform random eviction within each attention head, while always retaining the prompt, eliminating all per-token scoring computation. It costs nothing in memory or latency beyond the eviction itself and adds almost no implementation complexity, since the selection signal is removed entirely; quality is reported to match the best prior evictor on reasoning workloads. It can fail on workloads where the reasoning trace does not restate needed information or where redundancy across heads is insufficient, and on non-reasoning tasks or cache configurations where the prompt is not the dominant fragile component, since the findings are validated on reasoning traces with long chains of thought.

observed signal
1impl.
61stars
0citations
1.43xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2608.28911
First page of SemKV: Semantic Mixed-Precision KV Cache Quantization Guided by the Quality Cliff for Long-Context LLM Inference
KV cachetest

SemKV: Semantic Mixed-Precision KV Cache Quantization Guided by the Quality Cliff for Long-Context LLM Inference

SemKV replaces uniform low-bit KV cache quantization and FP16 token pruning with full token retention plus two adjacent above-cliff precisions assigned by a model-internal importance score. The cost is a per-deployment calibration step to locate the quality cliff, mixed-precision packing and dequantization logic in the inference path, and storage overhead for scores and per-token precision assignments, all inference-time with no retraining. It can fail if the cliff shifts under a different model, quantizer, or serving regime (generation-time quantization, multi-turn dialogue), since the reported thresholds were validated on Llama-3.1-8B-Instruct and Mistral-7B, and the no-loss claim is statistical rather than a guarantee of per-example equivalence.

observed signal
0impl.
0stars
0citations
6xclaimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.04263
First page of Quality Recovery for Quantized KV Caches via Low-Rank Attention Adaptation
KV cachetest

Quality Recovery for Quantized KV Caches via Low-Rank Attention Adaptation

The method replaces retraining or changing the KV-cache quantizer with distillation of the floating-point-cache model into low-rank Q/K/V projection updates, applied while the student runs a physically packed low-bit incremental cache. It costs an offline distillation run per model-quantizer pair, a small low-rank parameter overhead in the projections, and the validation effort needed to select ranks and runs. Perplexity recovery does not guarantee long-context retrieval recovery: the paper shows perplexity nearly restored at 2-bit while associative retrieval remains almost entirely broken, so quality must be validated on retrieval-style workloads, not perplexity alone.

observed signal
0impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗
arXiv 2609.04280
First page of EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?
agent evaluationwatch

EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?

EvoHarnessBench replaces ad hoc or fixed-harness agent evaluation with a controlled benchmark of 17 multi-stage harness streams (802 tasks, 520 tools, 42 skills, 62 agents) that isolates retention and adaptation as tools, skills, and sub-agents are added. Adopting it costs evaluation compute and engineering to wire one's own agent stack into its verifier-based environments, and as a benchmark it delivers no direct capability gain. Its practical warning—that harness expansion alone can degrade previously solved tasks (harness-induced forgetting), and that self-evolving memory gains are inconsistent—can fail to transfer if the reader's harness, model, or task distribution differs from the benchmark's deterministic streams.

observed signal
0impl.
0stars
0citations
—claimed gain
abstract onlyReview decisionDownload skill ↓Original paper ↗

Research signals

Three views of the same evidence: adoption against attention, publication cadence by research area, and reported gains when coverage is sufficient.

color = research area

01 · attention × adoption

The research frontier

Look to the upper left for independent implementation before mass attention.

compare by
Each point represents one paper. Higher means more independent implementations; farther right means more GitHub stars.The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents: 0 GitHub stars; 2 independent implementationsDi-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations: 45 GitHub stars; 3 independent implementationsSemKV: Semantic Mixed-Precision KV Cache Quantization Guided by the Quality Cliff for Long-Context LLM Inference: 0 GitHub stars; 0 independent implementationsRandom Attention: Rethinking KV Cache Eviction for Efficient Reasoning: 61 GitHub stars; 1 independent implementationQuality Recovery for Quantized KV Caches via Low-Rank Attention Adaptation: 0 GitHub stars; 0 independent implementationsEVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?: 0 GitHub stars; 0 independent implementations

Papers above 1000 stars or 200 citations are not scored. They have already broken through the attention threshold, while Paperraft is designed to identify earlier signals.

02 · cadence

Research areas over time

Compare monthly publication volume on a shared scale and calendar.

agent evaluationmax 2Jan 26: 1 paperMay 26: 0 papersAug 26: 0 papersSep 26: 1 paperJan 26Sep 26KV cachemax 2Jan 26: 0 papersMay 26: 0 papersAug 26: 1 paperSep 26: 2 papersJan 26Sep 26othermax 2Jan 26: 0 papersMay 26: 1 paperAug 26: 0 papersSep 26: 0 papersJan 26Sep 26

Every panel uses the same calendar and vertical scale. An empty column means no papers were recorded.

Method under scrutiny

Inspect the highest-ranked paper, every repository behind its signal, and the rule used to classify each one.

The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents

agent evaluation · test · 2 of 2 implementations are independent

Replaces single-language neutrality evaluation with semantically paired prompts in different languages. It needs paired prompts and extra API calls, with human interpretation because contested questions lack a definitive answer key. The study reports that ChatGPT 5.2 and Claude Opus 4.5 adopt the prompt language's dominant framing, but examines a single Ukrainian document and does not establish generalization to other language pairs or topics.

Exclusions and controls

Every exclusion is counted and published.

Exclusion counts were not recorded for this edition.

0 previously indexed papers were rechecked in this run.

Paperraft tracks one signal: how many independent implementations a paper attracts on GitHub after author-owned repositories are removed. This is a discovery signal to investigate, not a measure of implementation quality.

It does not claim that a method works. Paperraft runs no reproduction benchmarks. Reported gains remain author claims until independently tested. Papers above the attention threshold are excluded because this publication is designed to identify research before consensus forms.