Advisor(s)

Qing Tian

Committee Member(s)

Tianyang Wang
Xi Li

Document Type

Thesis

Date of Award

6-18-2026

Degree Name

Master of Science (MS)

School

College of Arts and Sciences

Department

Computer and Information Sciences

Abstract

Dense prediction tasks, including object detection and semantic segmentation, require models to produce structured predictions and are widely used in real-world vision applications. Although deep networks have achieved strong performance on these tasks, their high computational cost limits deployment on resource-constrained systems such as autonomous-driving vehicles. Knowledge distillation (KD) addresses this issue by transferring knowledge from a large teacher model to a compact student model. However, existing distillation methods for dense prediction face greater challenges than those for classification due to the more complex task requirements. To overcome the challenges, this thesis presents a unified study of adaptive distillation for dense prediction, introducing two novel KD frameworks. The central hypothesis is that emphasizing task-specific informative structures through learnable and student-aware distillation may improve the effectiveness of knowledge transfer. For object detection, Learnable Instance Attention Filtering for Knowledge Distillation (LIAF-KD) introduces learnable instance selectors to estimate instance importance from RoI-aligned features. During distillation, both teacher and student features interact with the learned selectors, producing instance attention scores that are projected into spatial masks for feature reweighting. This allows the student to participate in determining the distillation focus rather than passively receiving teacher-driven supervision. For semantic segmentation, SABER-KD (Student-Aware Boundary Evaluation and Refinement for Semantic Segmentation Distillation) extends structured attention to boundary-aware distillation. Since foreground-background distinction is insufficient for pixel-level prediction, SABER-KD focuses on boundary regions where classification errors are more likely to occur. Two ground-truth-supervised evaluators are pre-trained to estimate teacher-side and student-side boundary weights. During distillation, these evaluators interact with the current teacher and student to generate adaptive boundary weight maps, which are used to reweigh the boundary-alignment loss. Crucially, the student-side evaluator remains learnable during distillation and co-adapts with the student, so the supervision signal tracks the student's evolving difficulties. SABER-KD further introduces a dual-mask logit distillation objective that aligns the student to the teacher under teacher- and student-derived masks separately, allowing the student to learn from regions emphasized by either side rather than only their intersection. Extensive experiments on KITTI, COCO, Cityscapes, and Pascal VOC 2012 show that LIAF-KD and SABER-KD each outperform state-of-the-art distillation methods across multiple teacher-student pairs, without adding any inference-time cost. Together, the two frameworks make compact dense-prediction models more deployable on resource-constrained platforms such as autonomous-driving vehicles.

Keywords

Autonomous Driving;Knowledge Distillation;Object Detection;Semantic Segmentation

Download

Share

COinS