Back to Search Start Over

Continual Learning for End-to-End ASR by Averaging Domain Experts

Authors :
Plantinga, Peter
Yoo, Jaekwon
Dhir, Chandra
Publication Year :
2023

Abstract

Continual learning for end-to-end automatic speech recognition has to contend with a number of difficulties. Fine-tuning strategies tend to lose performance on data already seen, a process known as catastrophic forgetting. On the other hand, strategies that freeze parameters and append tunable parameters must maintain multiple models. We suggest a strategy that maintains only a single model for inference and avoids catastrophic forgetting. Our experiments show that a simple linear interpolation of several models' parameters, each fine-tuned from the same generalist model, results in a single model that performs well on all tested data. For our experiments we selected two open-source end-to-end speech recognition models pre-trained on large datasets and fine-tuned them on 3 separate datasets: SGPISpeech, CORAAL, and DiPCo. The proposed average of domain experts model performs well on all tested data, and has almost no loss in performance on data from the domain of original training.<br />Comment: Submitted to INTERSPEECH 2023

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2305.09681
Document Type :
Working Paper