Background Mediastinal follicular dendritic cell sarcoma (FDCS) is extremely rare. Due to potential under-recognization of this disease, it happens to be misdiagnosed, especially on core needle biopsy. We report 3 cases of mediastinal FDCS and provide a literature review to improve better understanding of the tumor and to reduce misdiagnosis. Methods Three cases of mediastinal FDCS in our clinic practice were studied, including their core needle biopsy and resected specimens, and those cases reported previously in English literature were retrieved and analyzed. Results The core needle biopsy of case 1 showed a tumor reminiscent of classical Hodgkin’s lymphoma (CHL), while the resected mass was finally diagnosed with FDCS combined with hyaline-vascular Castleman’s disease. Both the biopsy and resected tissue of case 2 were constitutive of the clear epithelioid cells with marked atypia. In both cases, definitive diagnoses were not made on core needle biopsy. In case 3, there were some areas morphologically similar to CHL, and some areas contained ovoid to spindle-shaped tumor cells with fascicular pattern. The analysis of 43 cases of mediastinal FDCS showed the age of patients were from 16 to 76 years old, the male to female ratio was 1.5:1, the maximal tumor diameters were 3–17 cm. 18 cases were underwent preoperative biopsy, whereas 15 (83.3%) of which were misdiagnosed initially, often as lymphoma. 32 patients had available follow-up data, the rates of recurrence, metastasis, and mortality were 12.5, 18.8 and 28.1%, respectively. Current limited data suggested no statistical differences between adverse prognosis and gender, age, tumor size, necrosis, or different therapeutics, respectively. Conclusions Mediastinal FDCS is a rare malignancy that has yet not been fully understood and been often misdiagnosed, particularly when making a diagnosis on core needle biopsy. Increased awareness of this enigmatic tumor is crucial to avoid diagnostic pitfalls.