Viscoelasticity and roughness are among the possible causes of the adhesive hysteresis displayed by soft contacts. Viscoelasticity causes an increased effective work of adhesion due to stiffening of the contact, while roughness is responsible for elastic instabilities. Herein, we explore the interplay between viscoelasticity and roughness by simulating in two dimensions the retraction of a rigid cylinder, with wavy surface profile, from a viscoelastic half-space. The wave amplitude and length are varied to induce instabilities in the load–to–area response, while the retraction velocity is increased to promote viscoelasticity. Results show that, in the regime where viscoelasticity is confined to the edges of the wavy contact, the contributions of viscoelasticity and waviness to adhesive hysteresis are nearly independent and additive. At low retraction rates, the instabilities in the load-area curve typical of rough elastic contacts are suppressed by viscoelasticity: the contact stiffens to promote a stable decrease of the contact area with load. This occurs with a minimal change in work of adhesion. However, when the instantaneous limit is met at high retraction rates, mechanical instabilities appear. LN acknowledges funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement no. 681813).