The nonlinear resonance behavior of a rigid-flexible space antenna deployed from the main body of a satellite is studied in this paper. At first, the nonlinear dynamic equations of in-plane motion of the antenna system of four degree-of-freedom are derived by using the Lagrange equation and the assumed modes method. Then the resonance and non-resonance regions are obtained. Afterwards, the universal expressions of the approximate analytic solutions under 2:1 or 3:1 internal resonance are obtained by using the method of multiple scales expressed in matrix form. Then, the stabilities of the approximate solutions are studied. Furthermore, the bifurcation characteristics of frequency-amplitude response of the antenna system are analyzed via the singularity theory. The transition set and the bifurcation diagrams in various regions of the unfolding parameters plane are discussed. To validate the accuracy of the approximate solutions, the numerical solutions and an experimental investigation are presented and discussed, respectively. The results show that nonlinear mode-coupled solutions exist over a wide range of the detuning parameter, both the numerical and the experimental results agree well with the analytical one. In addition, the bifurcation analysis and the experimental study provide a good reference for designing such a kind of antenna system. [ABSTRACT FROM AUTHOR]