Performance Analysis of Coherent and Noncoherent Modulation under I/Q Imbalance

In-phase/quadrature-phase Imbalance (IQI) is considered a major performance-limiting impairment in direct-conversion transceivers. Its effects become even more pronounced at higher carrier frequencies such as the millimeter-wave frequency bands being considered for 5G systems. In this paper, we quantify the effects of IQI on the performance of different modulation schemes under multipath fading channels. This is realized by developing a general framework for the symbol error rate (SER) analysis of coherent phase shift keying, noncoherent differential phase shift keying and noncoherent frequency shift keying under IQI effects. In this context, the moment generating function of the signal-to-interference-plus-noise-ratio is first derived for both single-carrier and multi-carrier systems suffering from transmitter (TX) IQI only, receiver (RX) IQI only and joint TX/RX IQI. Capitalizing on this, we derive analytic expressions for the SER of the different modulation schemes. These expressions are corroborated by comparisons with corresponding results from computer simulations and they provide insights into the dependence of IQI on the system parameters. We demonstrate that the effects of IQI differ considerably depending on the considered system as some cases of single-carrier transmission appear robust to IQI, whereas multi-carrier systems experiencing IQI at the RX require compensation in order to achieve a reliable communication link.