A second-order three-wave interaction system and its rogue wave solutions

On the basis of a $$3\times 3$$ matrix spectral problem, a second-order three-wave interaction system is proposed. A one-fold Darboux transformation is derived by using the gauge transformation between the $$3\times 3$$ matrix spectral problems. The compact determinant form of the N-fold Darboux transformation is obtained by iterating the onefold Darboux transformation and solving a complex linear algebraic system. Furthermore, the N-fold Darboux transformation and Taylor expansion are used to construct multifold generalized Darboux transforms of the second-order three-wave interaction system. As applications, the solutions of the interaction between a dark-bright soliton and a rogue wave (RW), the solutions of the interaction between two dark-bright solitons and a second-order rogue wave, the solutions of the eye-shaped rogue wave and triangle rogue wave are obtained.