Purpose To evaluate the contralateral lymph node recurrence rate (clLNRR) of stage IVA to IVB well-lateralized oral cavity cancer. To evaluate the risk factors of clLNRR. Materials and methods Pathologic stage IVA-B squamous cell carcinoma of oral cavity, originating from buccal mucosa, gingiva, or retromolar trigone were retrospectively recruited. Those who did not receive definitive surgery, with previous cancer history, or with contralateral nodal metastasis at diagnosis were excluded. Results From 2010 to 2017, 120 cases were enrolled, including 103 pT4 and 38 pN2. Thirty-one patients underwent contralateral neck dissection, and 18 had contralateral elective nodal irradiation. After median follow up of 35.1 months, the 3-year clLNRR was 15.7% (95% CI: 8.8 - 22.6%) as first event and was 17.1% (95% CI: 9.8 - 24.4%) for overall recurrences. The 3-year disease-free survival and overall survival were 52.8% and 63.1%, respectively. In multivariate analysis, positive nodal metastasis, gingival origin, and perineural invasion were associated with significantly higher clLNRR. Nodal metastasis was the strongest prognostic factor for clLNRR (pN1, HR: 17.1, p = 0.010; pN2, HR: 16.7, p = 0.004, comparing to pN0). The 3-year clLNRR were 2.9% for pN0 (n = 71, 95% CI: 0 - 6.8%), 37.7% for pN1 (n = 11, 95% CI: 8.3 - 67.1%), and 38.4% for pN2 (n = 38, 95% CI: 19.2 - 57.6%). Advanced T classification, elective contralateral neck dissection, and contralateral nodal irradiation did not have significant impact on clLNRR. Conclusions Positive homolateral nodal metastasis, gingival origin, and perineural invasion were risk factors correlated with significantly higher clLNRR. For patient without nodal metastasis, the clLNRR was low and elective contralateral neck management might be safely omitted. For patients with homolateral nodal disease, the contralateral nodal recurrence was not unusual. The optimal treatment for these high risk patients warrant further research.