Machine learning, especially deep learning, has become essential in many application domains. However, deep learning relies on artificial neural networks that often face resource-related limitations. For instance, data is often proprietary, model training can be costly, and using these models may be constrained by limited computational or storage resources. Transfer learning offers a solution to overcome these constraints by “transferring” a model from a source domain to a target domain, potentially in a different context. This transfer takes various forms: models can be adapted with minor structural changes (e.g., “fine-tuning”), reduced in size (e.g., “knowledge distillation”), or retrained with modified training and testing datasets (e.g., “domain adaptation”). This paper first motivates the need for a generic definitional framework and implementation support for transfer learning through a literature review. We then introduce Generic Transfer Learning (GTL), our proposal of such a framework. GTL supports the declarative definition of transfers through network transformations and dataset manipulations and includes corresponding Python implementation support. We finally present a case study demonstrating how to define and implement a transfer using GTL in the health domain.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Declarative, Generic Definition and Effective Implementation of Transfer Learning Algorithms

  • Tayeb Diab,
  • Mohamed Graiet,
  • Mario Sudholt

摘要

Machine learning, especially deep learning, has become essential in many application domains. However, deep learning relies on artificial neural networks that often face resource-related limitations. For instance, data is often proprietary, model training can be costly, and using these models may be constrained by limited computational or storage resources. Transfer learning offers a solution to overcome these constraints by “transferring” a model from a source domain to a target domain, potentially in a different context. This transfer takes various forms: models can be adapted with minor structural changes (e.g., “fine-tuning”), reduced in size (e.g., “knowledge distillation”), or retrained with modified training and testing datasets (e.g., “domain adaptation”). This paper first motivates the need for a generic definitional framework and implementation support for transfer learning through a literature review. We then introduce Generic Transfer Learning (GTL), our proposal of such a framework. GTL supports the declarative definition of transfers through network transformations and dataset manipulations and includes corresponding Python implementation support. We finally present a case study demonstrating how to define and implement a transfer using GTL in the health domain.