Cross-Architecture Foundation-Model Distillation for Edge Flood Segmentation

A narrow, repeated product task may not need your largest model.

What you would change

Check whether a released student model fits the task before generating training data.

Check whether a released student model fits the task before generating training data.

Compare against

Your current model and an off-the-shelf small model on the same held-out tasks.

1. Locate the paper's implementation and check its license, model support, data needs, and hardware requirements. Stop here if you cannot obtain the required artifacts.
2. Save a small set of representative product tasks and expected outcomes. Keep evaluation tasks out of any training or tuning, and choose quality and cost limits before running.
3. Run the baseline and the proposed change on identical inputs. Keep other settings fixed and repeat paired runs to expose variation.
4. Save per-task outputs, failures, timings, and all usage costs. Review regressions before expanding the test.

What to measure

Student quality, serving cost, teacher calls, training cost, and failures on unusual inputs.

When to keep it

Keep the change only if it meets your preselected quality and latency limits and improves the metric you care about after setup and operating costs. Otherwise retain the baseline. This is an exploration plan, not a validated recommendation.

recommendation

adopt

research area

distillation

technical change

Cross-architecture foundation-model distillation for edge segmentation

published

2026-09-17

Knowledge distillation from a large teacher into a quantized small student is a proven, low-infrastructure path to deployable models, and here it also stretches a fixed annotation budget, which directly matches the reader's constraints.

1. claim 01Paperraft inference
   Read the brief interpretation

   The method replaces direct supervised training of a small model on a fixed labeled set, and replaces edge deployment of a large foundation model, with teacher-generated pseudo-labels on unlabeled imagery used to train a compact student. The cost is a full teacher fine-tuning and inference pass over unlabeled data plus quantization-aware training and TensorRT export, with a residual quality gap on the primary benchmark (0.787 vs 0.822 water IoU). Failure modes include the student remaining below the teacher on the harder WorldFloods-v2 distribution, pseudo-label noise propagating at larger pool scales, and the possibility that a fixed MNDWI threshold matches the learned models on clean external data, limiting the practical advantage of the model.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source
2. claim 02Paperraft inference

   Distills a 300M-parameter Prithvi-EO-2.0 teacher into a 0.7M-parameter EfficientViT-B0 student (roughly 428x smaller), deployed as a 1.5 MB INT8 TensorRT engine at 5.57 ms per 512x512 image with ~14 MB runtime device memory; student reaches 0.787 water IoU versus 0.822 for the teacher on Sen1Floods11.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source