Search

In outlier detection, recent major research has shifted from developing univariate methods to multivariate methods due to the rapid growth of multidimensional data. However, one typical issue of this paradigm shift is that many multidimensional data often mainly contains univariate outliers , in which many features are actually irrelevant. In such cases, multivariate methods are ineffective in identifying such outliers due to the potential biases and the curse of dimensionality brought by irrelevant features. Those univariate outliers might be well detected by applying univariate outlier detectors in individually relevant features. However, it is very challenging to choose a right univariate detector for each individual feature since different features may take very different probability distributions. To address this challenge, we introduce a novel Heterogeneous Univariate Outlier Ensembles (HUOE) framework and its instance ZDD to synthesize a set of heterogeneous univariate outlier detectors as base learners to build heterogeneous ensembles that are optimized for each individual feature. Extensive results on 19 real-world datasets and a collection of synthetic datasets show that ZDD obtains 5%–14% average AUC improvement over four state-of-the-art multivariate ensembles and performs substantially more robustly w.r.t. irrelevant features.

© 2017 IEEE. Revealing complex relations between entities (e.g., items within or between transactions) is of great significance for business optimization, prediction, and decision making. Such relations include not only co-occurrence-based explicit relations but also nonco-occurrence-based implicit ones. Explicit relations have been substantially studied by rule mining-based approaches, including association rule mining and causal rule discovery. In contrast, implicit relations have received much less attention but could be more actionable. In this paper, we focus on the implicit relations between items which rarely or never co-occur while each of them co-occurs with other identical items (link items) with a high probability. A framework integrates both explicit and hidden item dependencies and a corresponding efficient algorithm IRRMiner captures such implicit relations with implicit rule inference. Experimental results show that IRRMiner not only infers implicit rules of various sizes consisting of both frequent and infrequent items effectively, it also runs at least four times faster than IARMiner, a typical indirect association rule mining algorithm which can only mine size-2 indirect association rules between frequent items. IRRMiner is applied to make recommendations and shows that the identified implicit rules can increase recommendation reliability.

Multivariate time series (MTS) such as multiple medical measures in intensive care units (ICU) are irregularly acquired and hold missing values. Conducting learning tasks on such irregular MTS with missing values, e.g., predicting the mortality of ICU patients, poses significant challenge to existing MTS forecasting models and recurrent neural networks (RNNs), which capture the temporal dependencies within a time series. This work proposes a bidirectional coupled MTS learning (BiCMTS) method to represent both forward and backward value couplings within a time series by RNNs and between MTS by self-attention networks; the learned bidirectional intra- and inter-time series coupling representations are fused to estimate missing values. We test BiCMTS on both data imputation and mortality prediction for ICU patients, showing a great potential of leveraging the deep and hidden relations captured in RNNs by the BiCMTS-learned intra- and inter-time series value couplings in MTS.

The detection of, explanation of, and accommodation to anomalies and novelties are active research areas in multiple communities, including data mining, machine learning, and computer vision. They are applied in various guises including anomaly detection, out-of-distribution example detection, adversarial example recognition and detection, curiosity-driven reinforcement learning, and open-set recognition and adaptation, all of which are of great interest to the SIGKDD community. The techniques developed have been applied in a wide range of domains including fraud detection and anti-money laundering in fintech, early disease detection, intrusion detection in large-scale computer networks and data centers, defending AI systems from adversarial attacks, and in improving the practicality of agents through overcoming the closed-world assumption. This workshop is focused on Anomaly and Novelty Detection, Explanation, and Accommodation (ANDEA). It will gather researchers and practitioners from data mining, machine learning, and computer vision communities and diverse knowledge background to promote the development of fundamental theories, effective algorithms, and novel applications of anomaly and novelty detection, characterization, and adaptation. All materials of keynote talks, panel discussion, and accepted papers of the workshop are made available at https://tinyurl.com/andea2021.

This special issue on learning complex couplings and interactions aims to encourage deep research in the above areas and beyond, with a focus on the latest advancements in modeling complex couplings and interactions in big data, complex behaviors, and systems.

Reports on the IEEE DSAA—Transdisciplinary Data Science initiative led by the IEEE The IEEE International Conference on Data Science and Advanced Analytics (IEEE DSAA). This conference aims to be a premier forum for addressing the above ever increasing and important demand, data volume and complexities, and the associated business problems, opportunities, decisions and values in a translational and transdisciplinary approach.

Simple and efficient exploration remains a core challenge in deep reinforcement learning. While many exploration methods can be applied to high-dimensional tasks, these methods manually adjust exploration parameters according to domain knowledge. This paper proposes a novel method that can automatically balance exploration and exploitation, as well as combine on-policy and off-policy update targets through a dynamic weighted way based on value difference. The proposed method does not directly affect the probability of a selected action but utilizes the value difference produced during the learning process to adjust update target for guiding the direction of agent’s learning. We demonstrate the performance of the proposed method on CartPole-v1, MountainCar-v0, and LunarLander-v2 classic control tasks from the OpenAI Gym. Empirical evaluation results show that by integrating on-policy and off-policy update targets dynamically, this method exhibits superior performance and stability than does the exclusive use of the update target.

Background: According to the statistics of WHO/IARC, cervical cancer (CC) has become the fourth malignant cancer of female worldwide and it is one of the main causes of death of women in developing countries. Purpose: Potential plasma and metabolic biomarkers for CC precancerous lesions and cervicitis were indicated by LC-MS techniques, and their underlying mechanisms and functions were analyzed. Methods: Plasma samples were selected from healthy people (control), low-grade squamous intraepithelial lesions (LSIL), high-grade squamous intraepithelial lesions (HSIL), CC, and post-treatment patients. All polypeptide types and sequences were detected by LC-MS/MS and the results were normalized by using Pareto-scaling. Potential metabolic biomarkers were screened by applying MetaboAnalyst 4.0 software and XCMS software, and analysis of variance and enrichment analysis were performed. Metabolic pathway analysis and functional enrichment analysis were used to further investigate the significance and pathological mechanisms of potential biomarkers. Results: Compared with healthy people, 9 differentially expressed metabolites were screened, 4 of which were up-regulated and 5 were down-regulated. LSIL group screened 7 differentially expressed metabolites, 5 of which were up-regulated and 2 were down-regulated; CC group screened 12 differentially expressed metabolites were screened, of which 9 were up-regulated and 3 were down-regulated. Eight differentially expressed metabolites were screened in the IF group, of which 5 showed up-regulation and 3 showed down-regulation. In functional enrichment analysis, differential metabolism was found to be associated with addition and coagulation cascades. Among all potential biomarkers, 2-amino-3-methyl-1-butanol, L-carnitine, Asn Asn Gln Arg, Ala Cys Ser Trp, Soladulcidine, Ala Ile Gln Arg, 2-amino-3 -Methyl-1-butanol, L-carnitine, Asn Asn Gln Arg, Ala Cys Ser Trp, Soladulcidine, Ala Ile Gln Arg can be used as predictors of precancerous lesions at different stages of CC. Among all biomarkers, 6α-fluoro-11β1,17-dihydroxypren-4-ene-3,20-dione has higher expression in the CC and HSIL groups and lower expression in the treatment group. Conclusion: By applying molecular markers to assess the progression of the disease, the accuracy and specificity of the diagnosis can be improved, which has certain prospects in clinical applications.

Mining negative sequential patterns (NSP) is an important tool for nonoccurring behavior analysis, and it is much more challenging than mining positive sequential patterns (PSPs) due to the high computational complexity and huge search space when obtaining the support of negative sequential candidates (NSCs). Very few NSP mining algorithms are available and most of them are very inefficient since they obtain the support of NSC by scanning the database repeatedly. Instead, the state-of-the-art NSP mining algorithm e-NSP only uses the PSP’s information stored in an array structure to ‘calculate’ the support of NSC by equations, without database re-scanning. This makes e-NSP highly efficient, particularly on sparse datasets. However, when datasets become dense, the key process to obtain the support of NSC in e-NSP becomes very time-consuming and needs to be improved. In this paper, we propose a novel and efficient data structure, a bitmap, to obtain the support of NSC. We correspondingly propose a fast NSP mining algorithm, f-NSP, which uses a bitmap to store the PSP’s information and then obtain the support of NSC only by bitwise operations, which is much faster than the hash method in e-NSP. Experimental results on real-world and synthetic datasets show that f-NSP is not only tens to hundreds of times faster than e-NSP, but also saves more than ten-fold the storage spaces of e-NSP, particularly on dense datasets with a large number of elements in a sequence or a small number of itemsets. Further, we find that f-NSP consumes more storage space than e-NSP when PSP’s support is less than a support threshold sdsup, a value obtained through our theoretical analysis of storage space. Accordingly, we propose a self-adaptive storage strategy and a corresponding algorithm f-NSP + to overcome this deficiency. f-NSP + can automatically choose a bitmap or an array structure to store PSP information according to PSP support. Experimental results show that f-NSP + saves more storage spaces of f-NSP, and has similar time efficiency as f-NSP.

In recommendation, both stationary and dynamic user preferences on items are embedded in the interactions between users and items (e.g., rating or clicking) within their contexts. Sequential recommender systems (SRSs) need to jointly involve such context-aware user-item interactions in terms of the couplings between the user and item features and sequential user actions on items over time. However, such joint modeling is non-trivial and significantly challenges the existing work on preference modeling, which either only models user-item interactions by latent factorization models but ignores user preference dynamics or only captures sequential user action patterns without involving user/item features and context factors and their coupling and influence on user actions. We propose a neural time-aware recommendation network (TARN) with a temporal context to jointly model 1) stationary user preferences by a feature interaction network and 2) user preference dynamics by a tailored convolutional network. The feature interaction network factorizes the pairwise couplings between non-zero features of users, items, and temporal context by the inner product of their feature embeddings while alleviating data sparsity issues. In the convolutional network, we introduce a convolutional layer with multiple filter widths to capture multi-fold sequential patterns, where an attentive average pooling (AAP) obtains significant and large-span feature combinations. To learn the preference dynamics, a novel temporal action embedding represents user actions by incorporating the embeddings of items and temporal context as the inputs of the convolutional network. The experiments on typical public data sets demonstrate that TARN outperforms state-of-the-art methods and show the necessity and contribution of involving time-aware preference dynamics and explicit user/item feature couplings in modeling and interpreting evolving user preferences.

Most of existing outlier detection methods assume that the outlier factors (i.e., outlierness scoring measures) of data entities (e.g., feature values and data objects) are Independent and Identically Distributed (IID). This assumption does not hold in real-world applications where the outlierness of different entities is dependent on each other and/or taken from different probability distributions (non-IID). This may lead to the failure of detecting important outliers that are too subtle to be identified without considering the non-IID nature. The issue is even intensified in more challenging contexts, e.g., high-dimensional data with many noisy features. This work introduces a novel outlier detection framework and its two instances to identify outliers in categorical data by capturing non-IID outlier factors. Our approach first defines and incorporates distribution-sensitive outlier factors and their interdependence into a value-value graph-based representation. It then models an outlierness propagation process in the value graph to learn the outlierness of feature values. The learned value outlierness allows for either direct outlier detection or outlying feature selection. The graph representation and mining approach is employed here to well capture the rich non-IID characteristics. Our empirical results on 15 real-world data sets with different levels of data complexities show that (i) the proposed outlier detection methods significantly outperform five state-of-the-art methods at the 95%/99% confidence level, achieving 10%-28% AUC improvement on the 10 most complex data sets; and (ii) the proposed feature selection methods significantly outperform three competing methods in enabling subsequent outlier detection of two different existing detectors., To appear in Data Ming and Knowledge Discovery Journal

In this tutorial we aim to present a comprehensive survey of the advances in deep learning techniques specifically designed for anomaly detection (deep anomaly detection for short). Deep learning has gained tremendous success in transforming many data mining and machine learning tasks, but popular deep learning techniques are inapplicable to anomaly detection due to some unique characteristics of anomalies, e.g., rarity, heterogeneity, boundless nature, and prohibitively high cost of collecting large-scale anomaly data. Through this tutorial, audiences would gain a systematic overview of this area, learn the key intuitions, objective functions, underlying assumptions, advantages and disadvantages of different categories of state-of-the-art deep anomaly detection methods, and recognize its broad real-world applicability in diverse domains. We also discuss what challenges the current deep anomaly detection methods can address and envision this area from multiple different perspectives. Any audience who may be interested in deep learning, anomaly/outlier/novelty detection, out-of-distribution detection, representation learning with limited labeled data, and self-supervised representation learning would find it very helpful in attending this tutorial. Researchers and practitioners in finance, cybersecurity, healthcare would also find the tutorial helpful in practice.

AI in finance broadly refers to the applications of AI techniques in financial businesses. This area has been lasting for decades with both classic and modern AI techniques applied to increasingly broader areas of finance, economy and society. In contrast to either discussing the problems, aspects and opportunities of finance that have benefited from specific AI techniques and in particular some new-generation AI and data science (AIDS) areas or reviewing the progress of applying specific techniques to resolving certain financial problems, this review offers a comprehensive and dense roadmap of the overwhelming challenges, techniques and opportunities of AI research in finance over the past decades. The landscapes and challenges of financial businesses and data are firstly outlined, followed by a comprehensive categorization and a dense overview of the decades of AI research in finance. We then structure and illustrate the data-driven analytics and learning of financial businesses and data. The comparison, criticism and discussion of classic vs. modern AI techniques for finance are followed. Lastly, open issues and opportunities address future AI-empowered finance and finance-motivated AI research., Comment: The paper is in the revision for ACM Computing Surveys, 40 pages

Unsupervised representation learning on mixed data is highly challenging but rarely explored. It has to tackle significant challenges related to common issues in real-life mixed data, including sparsity, dynamics and heterogeneity of attributes and values. This work introduces an effective and efficient unsupervised deep representer called Mix2Vec to automatically learn a universal representation of dynamic mixed data with the above complex characteristics. Mix2Vec is empowered with three effective mechanisms: random shuffling prediction, prior distribution matching, and structural informativeness maximization, to tackle the aforementioned challenges. These mechanisms are implemented as an unsupervised deep neural representer Mix2Vec. Mix2Vec converts complex mixed data into vector space-based representations that are universal and comparable to all data objects and transparent and reusable for both unsupervised and supervised learning tasks. Extensive experiments on four large mixed datasets demonstrate that Mix2Vec performs significantly better than state-of-the-art deep representation methods. We also empirically verify the designed mechanisms in terms of representation quality, visualization and capability of enabling better performance of downstream tasks.

Complex categorical data is often hierarchically coupled with heterogeneous relationships between attributes and attribute values and the couplings between objects. Such value-to-object couplings are heterogeneous with complementary and inconsistent interactions and distributions. Limited research exists on unlabeled categorical data representations, ignores the heterogeneous and hierarchical couplings, underestimates data characteristics and complexities, and overuses redundant information, etc. The deep representation learning of unlabeled categorical data is challenging, overseeing such value-to-object couplings, complementarity and inconsistency, and requiring large data, disentanglement, and high computational power. This work introduces a shallow but powerful UNsupervised heTerogeneous couplIng lEarning (UNTIE) approach for representing coupled categorical data by untying the interactions between couplings and revealing heterogeneous distributions embedded in each type of couplings. UNTIE is efficiently optimized w.r.t. a kernel k-means objective function for unsupervised representation learning of heterogeneous and hierarchical value-to-object couplings. Theoretical analysis shows that UNTIE can represent categorical data with maximal separability while effectively represent heterogeneous couplings and disclose their roles in categorical data. The UNTIE-learned representations make significant performance improvement against the state-of-the-art categorical representations and deep representation models on 25 categorical data sets with diversified characteristics.

© 2014 Elsevier Ltd. Complex applications such as big data analytics involve different forms of coupling relationships that reflect interactions between factors related to technical, business (domain-specific) and environmental (including socio-cultural and economic) aspects. There are diverse forms of couplings embedded in poor-structured and ill-structured data. Such couplings are ubiquitous, implicit and/or explicit, objective and/or subjective, heterogeneous and/or homogeneous, presenting complexities to existing learning systems in statistics, mathematics and computer sciences, such as typical dependency, association and correlation relationships. Modeling and learning such couplings thus is fundamental but challenging. This paper discusses the concept of coupling learning, focusing on the involvement of coupling relationships in learning systems. Coupling learning has great potential for building a deep understanding of the essence of business problems and handling challenges that have not been addressed well by existing learning theories and tools. This argument is verified by several case studies on coupling learning, including handling coupling in recommender systems, incorporating couplings into coupled clustering, coupling document clustering, coupled recommender algorithms and coupled behavior analysis for groups.

User purchase behaviours are complex and dynamic, which are usually observed as multiple choice actions across a sequence of shopping baskets. Most of the existing next-basket prediction approaches model user actions as homogeneous sequence data without considering complex and heterogeneous user intentions, impeding deep under-standing of user behaviours from the perspective of human inside drivers and thus reducing the prediction performance. Psychological theories have indicated that user actions are essentially driven by certain underlying intentions (e.g., diet and entertainment). Moreover, different intentions may influence each other while different choices usually have different utilities to accomplish an intention. Inspired by such psychological insights, we formalize the next-basket prediction as an Intention Recognition, Modelling and Accomplishing problem and further design the Intention2Basket (Int2Ba in short) model. In Int2Ba, an Intention Recognizer, a Coupled Intention Chain Net, and a Dynamic Basket Planner are specifically designed to respectively recognize, model and accomplish the heterogeneous intentions behind a sequence of baskets to better plan the next-basket. Extensive experiments on real-world datasets show the superiority of Int2Ba over the state-of-the-art approaches.

While data science has emerged as a contentious new scientific field, enormous debates and discussions have been made on it why we need data science and what makes it as a science. In reviewing hundreds of pieces of literature which include data science in their titles, we find that the majority of the discussions essentially concern statistics, data mining, machine learning, big data, or broadly data analytics, and only a limited number of new data-driven challenges and directions have been explored. In this paper, we explore the intrinsic challenges and directions inspired by comprehensively exploring the complexities and intelligence embedded in data science problems. We focus on the research and innovation challenges inspired by the nature of data science problems as complex systems, and the methodologies for handling such systems.

The recent booming of AI in FinTech evidences the significant developments and potential of AI for making smart FinTech, economy, finance and society. AI-empowered smart FinTech has emerged as a sexy and increasingly critical area in AI, data science, economics, finance, and other relevant research disciplines and business domains. This trend was built on the long history of AI in finance, and the new-generation AI, data science and machine learning are fundamentally and seamlessly transforming the vision, missions, objectives, paradigms, theories, approaches, tools and social aspects of economics and finance and driving smart FinTech. AI is empowering more personalized and advanced and better, safer and newer mainstream and alternative economic-financial mechanisms, products, models, services, systems, and applications. This review summarizes the lasting research on AI in finance and focuses on creating a comprehensive, multidimensional and economic-financial problem-driven research landscape of the roles, research directions and opportunities of AI in new-generation FinTech and finance.

Financial technology (FinTech) has been playing an increasingly critical role in driving modern economies, society, technology, and many other areas. Smart FinTech is the new-generation FinTech, largely inspired and empowered by data science and artificial intelligence (DSAI) techniques. Smart FinTech synthesizes broad DSAI and transforms finance and economies to drive intelligent, automated, whole-of-business and personalized economic and financial businesses, services and systems. The research on data science and AI in FinTech involves many latest progress made in smart FinTech for BankingTech, TradeTech, LendTech, InsurTech, WealthTech, PayTech, RiskTech, cryptocurrencies, and blockchain, and the DSAI techniques including complex system methods, quantitative methods, intelligent interactions, recognition and responses, data analytics, deep learning, federated learning, privacy-preserving processing, augmentation, optimization, and system intelligence enhancement. Here, we present a highly dense research overview of smart financial businesses and their challenges, the smart FinTech ecosystem, the DSAI techniques to enable smart FinTech, and some research directions of smart FinTech futures to the DSAI communities.

We consider the problem of anomaly detection with a small set of partially labeled anomaly examples and a large-scale unlabeled dataset. This is a common scenario in many important applications. Existing related methods either exclusively fit the limited anomaly examples that typically do not span the entire set of anomalies, or proceed with unsupervised learning from the unlabeled data. We propose here instead a deep reinforcement learning-based approach that enables an end-to-end optimization of the detection of both labeled and unlabeled anomalies. This approach learns the known abnormality by automatically interacting with an anomaly-biased simulation environment, while continuously extending the learned abnormality to novel classes of anomaly (i.e., unknown anomalies) by actively exploring possible anomalies in the unlabeled data. This is achieved by jointly optimizing the exploitation of the small labeled anomaly data and the exploration of the rare unlabeled anomalies. Extensive experiments on 48 real-world datasets show that our model significantly outperforms five state-of-the-art competing methods., Comment: Accepted to KDD 2021

Anomaly detection, a.k.a. outlier detection or novelty detection, has been a lasting yet active research area in various research communities for several decades. There are still some unique problem complexities and challenges that require advanced approaches. In recent years, deep learning enabled anomaly detection, i.e., deep anomaly detection, has emerged as a critical direction. This paper surveys the research of deep anomaly detection with a comprehensive taxonomy, covering advancements in three high-level categories and 11 fine-grained categories of the methods. We review their key intuitions, objective functions, underlying assumptions, advantages and disadvantages, and discuss how they address the aforementioned challenges. We further discuss a set of possible future opportunities and new perspectives on addressing the challenges., Comment: Survey paper, 36 pages, 180 references, 2 figures, 4 tables

© 1989-2012 IEEE. Appropriate similarity measures always play a critical role in data analytics, learning, and processing. Measuring the intrinsic similarity of categorical data for unsupervised learning has not been substantially addressed, and even less effort has been made for the similarity analysis of categorical data that is not independent and identically distributed (non-IID). In this work, a Coupled Metric Similarity (CMS) is defined for unsupervised learning which flexibly captures the value-to-attribute-to-object heterogeneous coupling relationships. CMS learns the similarities in terms of intrinsic heterogeneous intra-and inter-attribute couplings and attribute-to-object couplings in categorical data. The CMS validity is guaranteed by satisfying metric properties and conditions, and CMS can flexibly adapt to IID to non-IID data. CMS is incorporated into spectral clustering and k-modes clustering and compared with relevant state-of-the-art similarity measures that are not necessarily metrics. The experimental results and theoretical analysis show the CMS effectiveness of capturing independent and coupled data characteristics, which significantly outperforms other similarity measures on most datasets.

Traditional SVM-based multi-class classification algorithms mainly adopt the strategy of mapping the data set with all classes into a single feature space via a kernel function, in which SVM is constructed for each decomposed binary classification problem. However, it is not always possible to find an appropriate kernel function to render all the classes distinguishable in a single feature space, since each class is always derived from different data distributions. Consequently, the performance is not always as good as expected. To improve the performance of multi-class classification, this paper proposes an improved approach, called multi-state-mapping (MSM) with SVM based on hierarchical architecture, which maps the data set with all classes into different feature spaces at the different states of the decomposition of a multi-class classification problem in terms of a binary tree architecture. We prove that the computational complexity of MSM at its worst lies between that of the one-against-all scheme and one-against-one scheme. Substantial experiments have been conducted on sixteen UCI data sets to show the performance of our method. The statistical results show that MSM outperforms state-of-the-art methods in terms of accuracy and standard deviation.

Through silicon vias (TSVs) increase the interconnect bandwidth, reduce the RC delay and improve the power efficiency because the TSVs shorten vertical signal paths greatly. The process of via-reveal on the wafer backside is one of key steps for a successful implementation of 3D-IC stacking using the TSV technique, e.g. the via-first and via-middle TSV integrations. Instead of using a full chemical mechanical polish (CMP) or a dry etch process, we proposed a method of two-step full wet etching in this study. The solutions of HF/HNO3 and tetramethyl ammonium hydroxide (TMAH) are used in the two steps, respectively. The second etching step is only necessary for the wafers with a large thickness variation. Based on the full wet etching mechanism of silicon, we designed experiments to obtain the satisfied etching rate, selectivity and etching profile with optimized parameters. Using the approach, the via-reveal process was conducted for wafers with different incoming remain silicon thickness (RST) profiles. The sample characterization manifest that the present full wet etching process is well-controlled, efficient and cost effective. The process of via-reveal on the wafer backside is one of key steps for a successful implementation of 3DIC stacking using the TSV technique. Instead of using a full chemical mechanical polish (CMP) or a dry etch process, we proposed a method of full wet etching. The solutions of HF/HNO3 and tetramethyl ammonium hydroxide (TMAH) are used. Based on the full wet etching mechanism of silicon, we done some experiments to obtain the satisfied etching rate, selectivity and etching profile with optimized parameters. Using the approach, the via-reveal process was conducted for wafers with different incoming remain silicon thickness (RST) profiles. The wet etching results demonstrated that the exposed tips of TSVs were in good shape and had no damage in the liner oxide and Cu metal. The height of exposed TSV can be controlled very well and the etched silicon surface is smooth and uniform. So the present full wet etching process is well-controlled, efficient and cost effective method.Display Omitted A two-step wet etching method based on HF/HNO3 mixture and TMAH was proposed.Different etching profiles were obtained to compensate the thickness variation of incoming wafers in the etching process.The via-reveal method can be conducted for wafers with different incoming remain silicon thickness (RST) profiles.Using the approach, the exposed tips of TSVs were in good shape and had no damage in the liner oxide and Cu metal.The process is well-controlled and effective and has advantages in cost comparing to a dry etching or a full CMP method.

Short-head and long-tail distributed data are widely observed in the real world. The same is true of recommender systems (RSs), where a small number of popular items dominate the choices and feedback data while the rest only account for a small amount of feedback. As a result, most RS methods tend to learn user preferences from popular items since they account for most data. However, recent research in e-commerce and marketing has shown that future businesses will obtain greater profit from long-tail selling. Yet, although the number of long-tail items and users is much larger than that of short-head items and users, in reality, the amount of data associated with long-tail items and users is much less. As a result, user preferences tend to be popularity-biased. Furthermore, insufficient data makes long-tail items and users more vulnerable to shilling attack. To improve the quality of recommendations for items and users in the tail of distribution, we propose a coupled regularization approach that consists of two latent factor models: C-HMF, for enhancing credibility, and S-HMF, for emphasizing specialty on user choices. Specifically, the estimates learned from C-HMF and S-HMF recurrently serve as the empirical priors to regularize one another. Such coupled regularization leads to the comprehensive effects of final estimates, which produce more qualitative predictions for both tail users and tail items. To assess the effectiveness of our model, we conduct empirical evaluations on large real-world datasets with various metrics. The results prove that our approach significantly outperforms the compared methods.

Advanced analytics, data science, and new generation artificial intelligence (Al) represent probably the most promising areas and directions in today's and near -future's Information and Communications Technology and Science, Engineering and Technology sectors and disciplines, where data science has become the major driving force of the new-generation Al. Data science and new-generation Al have attracted increasing interest from major governments, vendors, and academia, with important initiatives launched by major countries such as the United States,' China, and the European Commission. However, despite the fact that the role of data scientists has been described as the sexiest job in the 21st century, the qualifications and capabilities of a data scientist are not clearly defined; it is important yet undermined to define what makes the next-generation data scientists who can transform today's and future science, technology, innovation, and economy. On one hand, as an increasing number of data science courses are being created by online course offers and traditional institutions, data science has clearly formed a new profession. 8,°,1° On the other, high -end data science employers (including major innovators, vendors, and enterprises) complain the limited availability of qualified data scientists to enable their strategic development and foster their future competitive advantages. These disclose the deviated status of existing professional and educational markets, the restricted benchmarldng and accreditation of responsibilities and capabilities of data scientists, and the urgent need of standardizing and upgrading competencies and maturity of data science qualifications and education. This article intends to address these important issues, with an aim to contributing to the standardization and formalization of next -generation data science profession and education.

A session-based recommender system (SBRS) suggests the next item by modeling the dependencies between items in a session. Most of existing SBRSs assume the items inside a session are associated with one (implicit) purpose. However, this may not always be true in reality, and a session may often consist of multiple subsets of items for different purposes (e.g., breakfast and decoration). Specifically, items (e.g., bread and milk) in a subsethave strong purpose-specific dependencies whereas items (e.g., bread and vase) from different subsets have much weaker or even no dependencies due to the difference of purposes. Therefore, we propose a mixture-channel model to accommodate the multi-purpose item subsets for more precisely representing a session. Filling gaps in existing SBRSs, this model recommends more diverse items to satisfy different purposes. Accordingly, we design effective mixture-channel purpose routing networks (MCPRN) with a purpose routing network to detect the purposes of each item and assign it into the corresponding channels. Moreover, a purpose specific recurrent network is devised to model the dependencies between items within each channel for a specific purpose. The experimental results show the superiority of MCPRN over the state-of-the-art methods in terms of both recommendation accuracy and diversity.

© 1989-2012 IEEE. The representation of categorical data with hierarchical value coupling relationships (i.e., various value-to-value cluster interactions) is very critical yet challenging for capturing complex data characteristics in learning tasks. This paper proposes a novel and flexible coupled unsupervised categorical data representation (CURE) framework, which not only captures the hierarchical couplings but is also flexible enough to be instantiated for contrastive learning tasks. CURE first learns the value clusters of different granularities based on multiple value coupling functions and then learns the value representation from the couplings between the obtained value clusters. With two complementary value coupling functions, CURE is instantiated into two models: coupled data embedding (CDE) for clustering and coupled outlier scoring of high-dimensional data (COSH) for outlier detection. These show that CURE is flexible for value clustering and coupling learning between value clusters for different learning tasks. CDE embeds categorical data into a new space in which features are independent and semantics are rich. COSH represents data w.r.t. an outlying vector to capture complex outlying behaviors of objects in high-dimensional data. Substantial experiments show that CDE significantly outperforms three popular unsupervised encoding methods and three state-of-the-art similarity measures, and COSH performs significantly better than five state-of-the-art outlier detection methods on high-dimensional data. CDE and COSH are scalable and stable, linear to data size and quadratic to the number of features, and are insensitive to their parameters.

© 1989-2012 IEEE. The structured case representation improves case-based reasoning (CBR) by exploring structures in the case base and the relevance of case structures. Recent CBR classifiers have mostly been built upon the attribute-value case representation rather than structured case representation, in which the structural relations embodied in their representation structure are accordingly overlooked in improving the similarity measure. This results in retrieval inefficiency and limitations on the performance of CBR classifiers. This paper proposes a hierarchical case-based classifier, HCBC, which introduces a concept lattice to hierarchically organize cases. By exploiting structural case relations in the concept lattice, a novel dynamic weighting model is proposed to enhance the concept similarity measure. Based on this similarity measure, HCBC retrieves the top-K concepts that are most similar to a new case by using a bottom-up pruning-based recursive retrieval (PRR) algorithm. The concepts extracted in this way are applied to suggest a class label for the case by a weighted majority voting. Experimental results show that HCBC outperforms other classifiers in terms of classification performance and robustness on categorical data, and also works confidently well on numeric datasets. In addition, PRR effectively reduces the search space and greatly improves the retrieval efficiency of HCBC.

In social networks, nodes (or users) interested in specific topics are often influenced by others. The influence is usually associated with a set of nodes rather than a single one. An interesting but challenging task for any given topic and node is to find the set of nodes that represents the source or trigger for the topic and thus identify those nodes that have the greatest influence on the given node as the topic spreads. We find that it is an NP-hard problem. This paper proposes an effective framework to deal with this problem. First, the topic propagation is represented as the Bayesian network. We then construct the propagation model by a variant of the voter model. The probability transition matrix (PTM) algorithm is presented to conduct the probability inference with the complexity $\boldsymbol {\mathcal {O}}(\boldsymbol {\theta }^{\mathbf {3}}\mathbf {log}_{\mathbf {2}}{\boldsymbol {\theta }})$ , while $\boldsymbol {\theta }$ is the number nodes in the given graph. To evaluate the PTM algorithm, we conduct extensive experiments on real datasets. The experimental results show that the PTM algorithm is both effective and efficient.

Short text is widely seen in applications including Internet of Things (IoT). The appropriate representation and classification of short text could be severely disrupted by the sparsity and shortness of short text. One important solution is to enrich short text representation by involving cognitive aspects of text, including semantic concept, knowledge, and category. In this paper, we propose a named Entity-based Concept Knowledge-Aware (ECKA) representation model which incorporates semantic information into short text representation. ECKA is a multi-level short text semantic representation model, which extracts the semantic features from the word, entity, concept and knowledge levels by CNN, respectively. Since word, entity, concept and knowledge entity in the same short text have different cognitive informativeness for short text classification, attention networks are formed to capture these category-related attentive representations from the multi-level textual features, respectively. The final multi-level semantic representations are formed by concatenating all of these individual-level representations, which are used for text classification. Experiments on three tasks demonstrate our method significantly outperforms the state-of-the-art methods.

Learning expressive low-dimensional representations of ultrahigh-dimensional data, e.g., data with thousands/millions of features, has been a major way to enable learning methods to address the curse of dimensionality. However, existing unsupervised representation learning methods mainly focus on preserving the data regularity information and learning the representations independently of subsequent outlier detection methods, which can result in suboptimal and unstable performance of detecting irregularities (i.e., outliers). This paper introduces a ranking model-based framework, called RAMODO, to address this issue. RAMODO unifies representation learning and outlier detection to learn low-dimensional representations that are tailored for a state-of-the-art outlier detection approach - the random distance-based approach. This customized learning yields more optimal and stable representations for the targeted outlier detectors. Additionally, RAMODO can leverage little labeled data as prior knowledge to learn more expressive and application-relevant representations. We instantiate RAMODO to an efficient method called REPEN to demonstrate the performance of RAMODO. Extensive empirical results on eight real-world ultrahigh dimensional data sets show that REPEN (i) enables a random distance-based detector to obtain significantly better AUC performance and two orders of magnitude speedup; (ii) performs substantially better and more stably than four state-of-the-art representation learning methods; and (iii) leverages less than 1% labeled data to achieve up to 32% AUC improvement., 10 pages, 4 figures, 3 tables. To appear in the proceedings of KDD18, Long presentation (oral)

We are living in the age of big data, advanced analytics, and data science. The trend of “big data growth” [29, 106, 266, 288, 413] (data deluge [210]) has not only triggered tremendous hype and buzz, but more importantly presents enormous challenges, which in turn have brought incredible innovation and economic opportunities.

© 2018, Springer Nature Switzerland AG. Training a binary classifier only on positive and unlabeled examples (i.e., the PU learning) is an important yet challenging issue, widely seen in many problems in which it is difficult to obtain negative examples. Existing methods for handling this challenge often perform unsatisfactorily, since they often ignore the relations between positive and unlabeled examples and are also limited to the traditional shallow learning frameworks. Therefore, this work proposes a new approach: Deep Positive and Unlabeled learning with Manifold Regularization (Deep-PUMR), which integrates the manifold regularization with deep neural networks to address the above issues with classic PU learning. Deep-PUMR holds two major advantages: (i) Our method exploits the manifold properties of data distribution to capture the relationship of positive and unlabeled examples; (ii) The adopted deep network enables Deep-PUMR with strong learning ability, especially on large-scale datasets. Extensive experiments on five diverse datasets demonstrate that Deep-PUMR achieves the state-of-the-art performance in comparison with classic PU learning algorithms and risk estimators.

In the age of analytics, data analytics and learning form a comprehensive spectrum and evolutionary map that cover.

In principle, data science can be applied to any application area or business domain. However, data science projects are intrinsically different from ordinary software development and IT projects.

What are the greatest challenges of big data and data science? This question itself is problematic as data science is at a very early stage and has been built on existing disciplines. This chapter explores this important issue.

Data science and big data analytics have led to next-generation economy innovation, competition and productivity [288], as typically shown by the rapidly updated big data landscape [26].

© 2017 Association for Computing Machinery. This paper introduces a novel framework, namely SelectVC and its instance POP, for learning selective value couplings (i.e., interactions between the full value set and a set of outlying values) to identify outliers in high-dimensional categorical data. Existing outlier detection methods work on a full data space or feature subspaces that are identified independently from subsequent outlier scoring. As a result, they are significantly challenged by overwhelming irrelevant features in high-dimensional data due to the noise brought by the irrelevant features and its huge search space. In contrast, SelectVC works on a clean and condensed data space spanned by selective value couplings by jointly optimizing outlying value selection and value outlierness scoring. Its instance POP defines a value outlierness scoring function by modeling a partial outlierness propagation process to capture the selective value couplings. POP further defines a top-k outlying value selection method to ensure its scalability to the huge search space. We show that POP (i) significantly outperforms five state-of-the-art full space or subspace-based outlier detectors and their combinations with three feature selection methods on 12 real-world high-dimensional data sets with different levels of irrelevant features; and (ii) obtains good scalability, stable performance w.r.t. k, and fast convergence rate.

In the field of traffic bottleneck analysis, it is expected to discover traffic congestion patterns from the reports of road conditions. However, data patterns mined by existing KDD algorithms may not coincide with the real application requirements. Different from academic researchers, traffic management officers do not pursue the most frequent patterns but always hold multiple views for mining task to facilitate traffic planning. They expect to study the correlation between traffic congestion and various kinds of road properties, especially the road properties easily to be improved. In this multi-view analysis, each view actually denotes a kind of user preference of road properties. Thus it is required to integrate user-defined attribute preferences into pattern mining process. To tackle this problem, we propose a multi-view attribute reduction model to discover the patterns of user interests. In this model, user views are expressed with attribute preferences and formally represented by attribute orders. Based on this, we implement a workflow of multi-view traffic bottleneck analysis, which consists of data preprocessing, preference representation and congestion pattern mining. We validate our approach based on the reports of road conditions from Shanghai. Experimental results show that the resultant multi-view mining outcomes are effective for analyzing congestion causes and traffic management.

© 1993-2012 IEEE. When an intelligent model is used for medical diagnosis, it is desirable to have a high level of interpretability and transparent model reliability for users. Compared with most of the existing intelligence models, fuzzy systems have shown a distinctive advantage in their interpretabilities. However, how to determine the model reliability of a fuzzy system trained for a recognition task is still an unsolved problem at present. In this study, a minimax probability Takagi-Sugeno-Kang (TSK) fuzzy system classifier called MP-TSK-FSC is proposed to train a fuzzy system classifier and determine the model reliability simultaneously. For the proposed MP-TSK-FSC, a lower bound of correct classification can be presented to the users to characterize the reliability of the trained fuzzy classifier. Thus, the obtained classifier has the distinctive characteristics of both a high level of interpretability and transparent model reliability inherited from the fuzzy system and minimax probability learning strategy, respectively. Our experiments on synthetic datasets and several real-world datasets for medical diagnosis have confirmed the distinctive characteristics of the proposed method.

Group behavior interactions, such as multirobot teamwork and group communications in social networks, are widely seen in both natural, social, and artificial behavior-related applications. Behavior interactions in a group are often associated with varying coupling relationships, for instance, conjunction or disjunction. Such coupling relationships challenge existing behavior representation methods, because they involve multiple behaviors from different actors, constraints on the interactions, and behavior evolution. In addition, the quality of behavior interactions are not checked through verification techniques. In this paper, we propose an ontology-based behavior modeling and checking system (OntoB for short) to explicitly represent and verify complex behavior relationships, aggregations, and constraints. The OntoB system provides both a visual behavior model and an abstract behavior tuple to capture behavioral elements, as well as building blocks. It formalizes various intra-coupled interactions (behaviors conducted by the same actor) via transition systems (TSs), and inter-coupled behavior aggregations (behaviors conducted by different actors) from temporal, inferential, and party-based perspectives. OntoB converts a behavior-oriented application into a TS and temporal logic formulas for further verification and refinement. We demonstrate and evaluate the effectiveness of the OntoB in modeling multirobot behaviors and their interactions in the Robocup soccer competition game. We show, that the OntoB system can effectively model complex behavior interactions, verify and refine the modeling of complex group behavior interactions in a sound manner.

Many real-world applications expose the nonlinear manifold structure of the lower dimension rather than its high-dimensional input space. This greatly challenges most existing clustering and representative selection algorithms which do not take the manifold characteristics into consideration. The performance of the corresponding learning algorithms can be greatly improved if the manifold structure is considered. In this paper, we propose a graph-based k-means algorithm, GKM, which bears the simplicity of classic k-means while incorporating global information of data geometric distribution. GKM fully exploits the intrinsic manifold structure for appropriate data clustering and representative selection. GKM is evaluated on both synthetic and real-life data sets and achieves very impressive results compared to the state-of-the-art approaches, including classic k-means, kernel k-means, spectral clustering, and clustering through ranking and for representative selection. Given the widespread appearance of manifold structures in real world problems, GKM shows promising potential for partitioning manifold-distributed data.

© 2017, The Author(s). A novel online ensemble strategy, ensemble BPegasos (EBPegasos), is proposed to solve the problems simultaneously caused by concept drifting and the curse of dimensionality in classifying high-dimensional evolving data streams, which has not been addressed in the literature. First, EBPegasos uses BPegasos, an online kernelized SVM-based algorithm, as the component classifier to address the scalability and sparsity of high-dimensional data. Second, EBPegasos takes full advantage of the characteristics of BPegasos to cope with various types of concept drifts. Specifically, EBPegasos constructs diverse component classifiers by controlling the budget size of BPegasos; it also equips each component with a drift detector to monitor and evaluate its performance, and modifies the ensemble structure only when large performance degradation occurs. Such conditional structural modification strategy makes EBPegasos strike a good balance between exploiting and forgetting old knowledge. Lastly, we first prove experimentally that EBPegasos is more effective and resource-efficient than the tree ensembles on high-dimensional data. Then comprehensive experiments on synthetic and real-life datasets also show that EBPegasos can cope with various types of concept drifts significantly better than the state-of-the-art ensemble frameworks when all ensembles use BPegasos as the base learner.

© 2017 ACM. Precisely recommending relevant items from massive candidates to a large number of users is an indispensable yet computationally expensive task in many online platforms (e.g., Amazon.com and Netfix.com). A promising way is to project users and items into a Hamming space and then recommend items via Hamming distance. However, previous studies didn't address the cold-start challenges and couldn't make the best use of preference data like implicit feedback. To fill this gap, we propose a Discrete Content-aware Matrix Factorization (DCMF) model, 1) to derive compact yet informative binary codes at the presence of user/item content information; 2) to support the classification task based on a local upper bound of logit loss; 3) to introduce an interaction regularization for dealing with the sparsity issue. We further develop an eficient discrete optimization algorithm for parameter learning. Based on extensive experiments on three real-world datasets, we show that DCFM outperforms the state-of-the-arts on both regression and classification tasks.

Learning the representation of categorical data with hierarchical value coupling relationships is very challenging but critical for the effective analysis and learning of such data. This paper proposes a novel coupled unsupervised categorical data representation (CURE) framework and its instantiation, i.e., a coupled data embedding (CDE) method, for representing categorical data by hierarchical value-to-value cluster coupling learning. Unlike existing embedding- and similarity-based representation methods which can capture only a part or none of these complex couplings, CDE explicitly incorporates the hierarchical couplings into its embedding representation. CDE first learns two complementary feature value couplings which are then used to cluster values with different granularities. It further models the couplings in value clusters within the same granularity and with different granularities to embed feature values into a new numerical space with independent dimensions. Substantial experiments show that CDE significantly outperforms three popular unsupervised embedding methods and three state-of-the-art similarity-based representation methods.

The 21st century has ushered in the age of big data and data economy, in which data DNA , which carries important knowledge, insights, and potential, has become an intrinsic constituent of all data-based organisms. An appropriate understanding of data DNA and its organisms relies on the new field of data science and its keystone, analytics . Although it is widely debated whether big data is only hype and buzz, and data science is still in a very early phase, significant challenges and opportunities are emerging or have been inspired by the research, innovation, business, profession, and education of data science. This article provides a comprehensive survey and tutorial of the fundamental aspects of data science: the evolution from data analysis to data science, the data science concepts, a big picture of the era of data science, the major challenges and directions in data innovation, the nature of data analytics, new industrialization and service opportunities in the data economy, the profession and competency of data education, and the future of data science. This article is the first in the field to draw a comprehensive big picture, in addition to offering rich observations, lessons, and thinking about data science and analytics.

Display Omitted A sequential contrast patterns based septic shock prediction approach is proposed.Reconstructs a patient instance as an ordered sequence of contrast patterns.Coupled HMMs are applied to multiple channels of contrast pattern sequences.Coupled HMM results are compared against single variable HMM and SVM models.Meta-information about patterns, like sequence order, improve model performance. Background and objectiveCritical care patient events like sepsis or septic shock in intensive care units (ICUs) are dangerous complications which can cause multiple organ failures and eventual death. Preventive prediction of such events will allow clinicians to stage effective interventions for averting these critical complications. MethodsIt is widely understood that physiological conditions of patients on variables such as blood pressure and heart rate are suggestive to gradual changes over a certain period of time, prior to the occurrence of a septic shock. This work investigates the performance of a novel machine learning approach for the early prediction of septic shock. The approach combines highly informative sequential patterns extracted from multiple physiological variables and captures the interactions among these patterns via coupled hidden Markov models (CHMM). In particular, the patterns are extracted from three non-invasive waveform measurements: the mean arterial pressure levels, the heart rates and respiratory rates of septic shock patients from a large clinical ICU dataset called MIMIC-II. Evaluation and resultsFor baseline estimations, SVM and HMM models on the continuous time series data for the given patients, using MAP (mean arterial pressure), HR (heart rate), and RR (respiratory rate) are employed. Single channel patterns based HMM (SCP-HMM) and multi-channel patterns based coupled HMM (MCP-HMM) are compared against baseline models using 5-fold cross validation accuracies over multiple rounds. Particularly, the results of MCP-HMM are statistically significant having a p-value of 0.0014, in comparison to baseline models. Our experiments demonstrate a strong competitive accuracy in the prediction of septic shock, especially when the interactions between the multiple variables are coupled by the learning model. ConclusionsIt can be concluded that the novelty of the approach, stems from the integration of sequence-based physiological pattern markers with the sequential CHMM model to learn dynamic physiological behavior, as well as from the coupling of such patterns to build powerful risk stratification models for septic shock patients.

© 2016 IEEE. Proper feature selection for unsupervised outlier detection can improve detection performance but is very challenging due to complex feature interactions, the mixture of relevant features with noisy/redundant features in imbalanced data, and the unavailability of class labels. Little work has been done on this challenge. This paper proposes a novel Coupled Unsupervised Feature Selection framework (CUFS for short) to filter out noisy or redundant features for subsequent outlier detection in categorical data. CUFS quantifies the outlierness (or relevance) of features by learning and integrating both the feature value couplings and feature couplings. Such value-To-feature couplings capture intrinsic data characteristics and distinguish relevant features from those noisy/redundant features. CUFS is further instantiated into a parameter-free Dense Subgraph-based Feature Selection method, called DSFS. We prove that DSFS retains a 2-Approximation feature subset to the optimal subset. Extensive evaluation results on 15 real-world data sets show that DSFS obtains an average 48% feature reduction rate, and enables three different types of pattern-based outlier detection methods to achieve substantially better AUC improvements and/or perform orders of magnitude faster than on the original feature set. Compared to its feature selection contender, on average, all three DSFS-based detectors achieve more than 20% AUC improvement.