(1) Objective: Our objective was to conduct a meta-analysis of randomized controlled trials that have evaluated the benefits of exercise training for elderly pulmonary fibrosis (PF) patients. (2) Methods: Studies in either English or Chinese were retrieved from the China National Knowledge Infrastructure (CNKI) and the Wanfang, PubMed, Web of Science and SPORTDiscus databases from inception until the first week of April 2021. Age, body mass index (BMI), and exercise frequency, intensity, type, and duration were considered for each participant. The specific data recorded were the six-minute walk distance (6MWD), maximal rate of oxygen consumption (peak VO2), predicted forced vital capacity (FVC% pred), predicted diffusing capacity of the lung for carbon monoxide (DLCO% pred), predicted total lung capacity (TLC% pred), St. George’s respiratory questionnaire (SGRQ) total score and a modified medical research council score (mMRC). (3) Results: Thirteen studies comprised this meta-analysis (eleven randomized controlled trials and two prospective studies design), wherein 335 patients were exercised and 334 were controls. The results showed that exercise training increased the 6MWD (Cohen’s d = 0.77, MD = 34.04 (95% CI, 26.50–41.58), p < 0.01), peak VO2 (Cohen’s d = 0.45, MD = 1.13 (95% CI, 0.45–1.82), p = 0.0001) and FVC% pred (Cohen’s d = 0.42, MD = 3.94 (95% CI, 0.91–6.96), p = 0.01). However, exercise training reduced scores for the SGRQ (Cohen’s d = 0.89, MD = −8.79 (95% CI, −10.37 to −7.21), p < 0.01) and the mMRC (Cohen’s d = 0.64, MD = −0.58 (95% CI, −0.79 to −0.36), p < 0.01). In contrast, exercise training could not increase DLCO% pred (Cohen’s d = 0.16, MD = 1.86 (95% CI, −0.37–4.09), p = 0.10) and TLC% pred (Cohen’s d = 0.02, MD = 0.07 (95% CI, −6.53–6.67), p = 0.98). Subgroup analysis showed significant differences in frequency, intensity, type, and age in the 6MWD results (p < 0.05), which were higher with low frequency, moderate intensity, aerobic–resistance–flexibility–breathing exercises and age ≤ 70. Meanwhile, the subgroup analysis showed significant differences in exercise intensity and types in the mMRC results (p < 0.05), which were lower with moderate intensity and aerobic–resistance exercises. (4) Conclusions: Exercise training during pulmonary rehabilitation can improved cardiopulmonary endurance and quality of life in elderly patients with PF. The 6MWDs were more noticeable with moderate exercise intensity, combined aerobic–resistance–flexibility–breathing exercises and in younger patients, which all were not affected by BMI levels or exercise durations. As to pulmonary function, exercise training can improve FVC% pred, but has no effect on DLCO% pred and TLC% pred.