Search

In numerous classification problems, class distribution is not balanced. For example, positive examples are rare in the fields of disease diagnosis and credit card fraud detection. General machine learning methods are known to be suboptimal for such imbalanced classification. One popular solution is to balance training data by oversampling the underrepresented (or undersampling the overrepresented) classes before applying machine learning algorithms. However, despite its popularity, the effectiveness of sampling has not been rigorously and comprehensively evaluated. This study assessed combinations of seven sampling methods and eight machine learning classifiers (56 varieties in total) using 31 datasets with varying degrees of imbalance. We used the areas under the precision-recall curve (AUPRC) and receiver operating characteristics curve (AUROC) as the performance measures. The AUPRC is known to be more informative for imbalanced classification than the AUROC. We observed that sampling significantly changed the performance of the classifier (paired t-tests P < 0.05) only for few cases (12.2% in AUPRC and 10.0% in AUROC). Surprisingly, sampling was more likely to reduce rather than improve the classification performance. Moreover, the adverse effects of sampling were more pronounced in AUPRC than in AUROC. Among the sampling methods, undersampling performed worse than others. Also, sampling was more effective for improving linear classifiers. Most importantly, we did not need sampling to obtain the optimal classifier for most of the 31 datasets. In addition, we found two interesting examples in which sampling significantly reduced AUPRC while significantly improving AUROC (paired t-tests P < 0.05). In conclusion, the applicability of sampling is limited because it could be ineffective or even harmful. Furthermore, the choice of the performance measure is crucial for decision making. Our results provide valuable insights into the effect and characteristics of sampling for imbalanced classification., Competing Interests: The authors have declared that no competing interests exist.

The molecular basis of heat shock response (HSR), a cellular defense mechanism against various stresses, is not well understood. In this, the first comprehensive analysis of gene expression changes in response to heat shock and MG132 (a proteasome inhibitor), both of which are known to induce heat shock proteins (Hsps), we compared the responses of normal mouse fibrosarcoma cell line, RIF-1, and its thermotolerant variant cell line, TR-RIF-1 (TR), to the two stresses. The cellular responses we examined included Hsp expressions, cell viability, total protein synthesis patterns, and accumulation of poly-ubiquitinated proteins. We also compared the mRNA expression profiles and kinetics, in the two cell lines exposed to the two stresses, using microarray analysis. In contrast to RIF-1 cells, TR cells resist heat shock caused changes in cell viability and whole-cell protein synthesis. The patterns of total cellular protein synthesis and accumulation of poly-ubiquitinated proteins in the two cell lines were distinct, depending on the stress and the cell line. Microarray analysis revealed that the gene expression pattern of TR cells was faster and more transient than that of RIF-1 cells, in response to heat shock, while both RIF-1 and TR cells showed similar kinetics of mRNA expression in response to MG132. We also found that 2,208 genes were up-regulated more than 2 fold and could sort them into three groups: 1) genes regulated by both heat shock and MG132, (e.g. chaperones); 2) those regulated only by heat shock (e.g. DNA binding proteins including histones); and 3) those regulated only by MG132 (e.g. innate immunity and defense related molecules). This study shows that heat shock and MG132 share some aspects of HSR signaling pathway, at the same time, inducing distinct stress response signaling pathways, triggered by distinct abnormal proteins.