Repair Changes Which Layers to Remove: Exact Optima for Depth Pruning of Language Models

A self-hosted model uses more compute or memory than your workload can justify.

What you would change

Check for a usable pruned checkpoint and runtime support before attempting model surgery.

Check for a usable pruned checkpoint and runtime support before attempting model surgery.

Compare against

The original unpruned model at the same precision, context, and load.

1. Locate the paper's implementation and check its license, model support, data needs, and hardware requirements. Stop here if you cannot obtain the required artifacts.
2. Save a small set of representative product tasks and expected outcomes. Keep evaluation tasks out of any training or tuning, and choose quality and cost limits before running.
3. Run the baseline and the proposed change on identical inputs. Keep other settings fixed and repeat paired runs to expose variation.
4. Save per-task outputs, failures, timings, and all usage costs. Review regressions before expanding the test.

What to measure

Task quality, actual GPU memory, serving throughput, and preparation cost.

When to keep it

Keep the change only if it meets your preselected quality and latency limits and improves the metric you care about after setup and operating costs. Otherwise retain the baseline. This is an exploration plan, not a validated recommendation.

recommendation

test

research area

sparsity and pruning

technical change

Repair-aware greedy depth pruning of transformer layers

published

2026-10-22

Depth pruning with training-free linear repair is executable on a single 24 GB GPU for small-to-mid models and the paper shows repair-aware selection materially outperforms heuristic baselines, but gains are model- and task-dependent, so the reader should validate the greedy or second-order selection on their own workload before production adoption.

1. claim 01Paperraft inference
   Read the brief interpretation

   The method replaces heuristic, repair-agnostic layer-removal scores (as in SLEB and ReplaceMe) with a greedy search that selects layers under the linear repair that will actually be applied, plus a second-order loss score that identifies the repaired block from one forward and one backward pass of the intact model. Costs are modest: no retraining, but the greedy search requires repeated evaluations of pruned-and-repaired variants (the paper itself ran roughly 53,000 evaluations for the exact study), and the linear repair map must be computed per removal block. It can fail when the optimal pruned layout differs sharply across model families or downstream tasks, when the linear repair poorly approximates removed-layer interactions for non-contiguous removals, and when perplexity gains do not transfer to the reader's specific task metrics.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source
2. claim 02Paperraft inference

   Greedy repair-aware layer selection lowers perplexity by 20% to 66% relative to ReplaceMe on five models up to 27B parameters, and with a task objective is up to 25 points more accurate given the same data.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source