همایش بین المللی هوش مصنوعی

صفحه اصلی / دومین همایش بین المللی هوش مصنوعی

AdaKD-LoRA: Adaptive Knowledge Distillation with Low-Rank Adaptation for Rehearsal Free Continual Learning

نویسندگان :

Mohammad Ghabel rahmat¹ Samira Vejdanparast² Azam Bastan Fard³ Abbas Jalilvand⁴

1- دانشگاه آزاد اسلامی واحد کرج 2- دانشگاه آزاد اسلامی واحد کرج 3- دانشگاه آزاد اسلامی واحد کرج 4- دانشگاه آزاد اسلامی واحد کرج

کلمات کلیدی :

Continual Learning،Rehearsal-Free،Adaptive Knowledge Distillation،LoRA،Class-Incremental Learning،Catastrophic Forgetting

چکیده :

Rehearsal free continual learning requires models to acquire new task-specific knowledge without erasing previously learned information, despite the absence of stored exemplars. However, many distillation-based strategies employ a static knowledge-distillation weight, applying a fixed regularization strength across all tasks. This static formulation is unable to reflect the varying degrees of model drift that occur during incremental learning, often resulting in either excessive rigidity or insufficient retention. To address this limitation, we introduce AdaKD-LoRA, a parameter efficient framework that combines low-rank adaptation with dynamically regulated distillation. In this approach, a pretrained backbone remains frozen to preserve general representations, while lightweight LoRA modules encode task-specific updates. The strength of the distillation term is adaptively adjusted based on the divergence between consecutive task models, allowing the method to increase emphasis on stability when drift is high and to maintain flexibility when new classes require stronger plasticity. Experiments on a ten-task Class-Incremental Learning benchmark show that this adaptive formulation substantially reduces forgetting and achieves more stable knowledge preservation compared with Fine-Tuning, LwF, and LoRA-only baselines. AdaKD-LoRA delivers higher accuracy on earlier tasks, more favorable backward transfer, and competitive overall performance, demonstrating that replacing static distillation with an adaptive mechanism provides a more effective and scalable solution for rehearsal-free continual learning.