Search

HUIM (High utility itemsets mining) is a sub-division of data mining dealing with the task to obtain promising patterns in the quantitative datasets. A variant of HUIM is to discover the HAUIM (High average-utility itemsets mining) where average-utility measure is used to obtain the utility of itemsets. HAUIM is the refined version of FIM (Frequent itemset mining) problem and has various applications in the field of market basket analysis, bio-informatics, text mining, network traffic analysis, product recommendation and e-learning among others. In this paper, we provide a comprehensive survey of the state-of-the-art methods of HAUIM to mine the HAUIs (High average-utility itemsets) from the static and dynamic datasets since the induction of the HAUIM problem. We discuss the pros and cons of each category of mining approaches in detail. The taxonomy of HAUIM is presented according to the mining approaches. Finally,various extensions, future directions and research opportunities of HAUIM algorithms are discussed.

The current study refers to the affinity analysis of a prescription and diagnosis dataset and focuses on how prescription behavior changes after the detection of a disorder. Using a subset of patients that have been diagnosed, at least once, with hypertension and hyperlipidemia, examines the prescription behavior of the doctor after the detection of gastroesophageal reflux (K21), or insulin-dependent diabetes mellitus (E10). Diagnosis and prescription data were collected during consecutive visits of 4473 patients in a 3 years period. The analysis of the prescription data before and after the diagnosis of K21 and E10 reveals the popular substances for each disorder, such as Metformin for E10, or Omeprazole for K21 and substances that are discontinued including the same popular substances as well. It also reveals that substances that treat the main disorders of the group, such as Simvastatin and Hydrochlorothiazide are discontinued after the diagnosis of K21 and E10. Apart from the medical findings, which must be subject of further research, the proposed methodology can be employed for studying the prescription behavior of physicians and how it changes after specific diagnoses.

The problem of frequent itemset mining is one of the more important problems in data mining which has been extensively employed across a wide range of other relevant tasks such as market basket analysis in marketing, or text analysis in text mining applications. The majority of the deterministic frequent itemset mining algorithms which have been proposed in recent years use some sort or another of an optimal data structures to reduce the overall execution time of the algorithm. In this paper, however, we have tried instead to introduce an approximation algorithm which works by converting the problem into a clustering problem where similar transactions are grouped together. Each cluster centroid represents an itemset which may be assumed to be a candidate frequent itemsets. The validity of this assumption is simply verified by calculating the support count of these itemsets. Those who meet the min-support condition are considered to be an actual frequent itemset. As for the remaining itemsets, they are then passed to MAFIA which extract all maximal frequent itemsets therefrom. Experimentations made on several well-known and diverse datasets show that the proposed algorithm performs almost always faster, and in some cases up to 10 times faster, than the existing deterministic algorithms, and all this by retaining up to 95% of its accuracy.

Data analysis plays a vital role in the present era as it helps us to understand the patterns by exploring it in meaningful ways. Market—basket is one of the main methods used to find frequently occurring items in a transactional database and many researchers use the Apriori algorithm for this purpose. This paper presents the application of Market Basket Analysis to the healthcare section. The present work tries to find frequent diseases that occur together in an area by using the Apriori algorithm. This could help the residents of an area to be more cautious about the frequently occurring diseases and take all possible precautionary measures to safeguard their health. In addition, it could also help the doctors so that, they are ready with required medications to treat the patients.

The popularity of online shopping is growing rapidly in modern virtual market. Generally, customers take decision to purchase goods based on their basic need and relative need. Shopkeepers play an important role to influence the customers in real market. Recommendation engine is nothing but a good automated shopkeeper. In this paper, we propose a model of dynamic recommendation system (DRS) for online market. Our proposed technique provides an intelligent solution model to overcome the problems of customers’ rating and their feedback by integrating market basket analysis, frequent item mining, bestselling items and customer personalization.

We compare several unsupervised probabilistic machine learning methods for market basket analysis, namely binary factor analysis, two topic models (latent Dirichlet allocation and the correlated topic model), the restricted Boltzmann machine and the deep belief net. After an overview of previous applications of unsupervised probabilistic machine learning methods to market basket analysis we shortly present the methods which we investigate and outline their estimation. Performance is measured by tenfold cross-validated log likelihood values. Binary factor analysis vastly outperforms topic models. The restricted Boltzmann machine attains a similar performance advantage over binary factor analysis. Overall, a deep belief net with 45 variables in the first and 15 variables in the second hidden layers turns out to be the best model. We also compare the investigated machine learning methods with respect to ease of interpretation and runtimes. In addition, we show how to interpret the relationships between hidden variables and observed category purchases. To demonstrate managerial implications we estimate the effect of promoting each category both on purchase probability increases of other product categories and the relative increase of basket size. Finally, we indicate several possibilities to extend restricted Boltzmann machines and deep belief nets for market basket analysis.

Companion vehicle discovery received much attention from the research community. It has been widely adopted by traffic management departments in many aspects such as the involved vehicle tracking. Since there are a massive amount of traffic data that have complex and inaccurate accompanying vehicle relationships, companion vehicle discovery has become a challenge yet hot research topic. Several algorithms have been proposed to solve this issue on transactional datasets some of which are based on the frequent item mining algorithms that are used to extract knowledge from data in several real-world applications, such as market basket analysis, crime detection/prevention, and crowd mining. However, most of those algorithms mostly fail on large-scale datasets since it needs to scan the datasets iteratively for several times, which makes them unfeasible and time-consuming while dealing with big data. To this end, we proposed a novel HD-FIM algorithm to extract the companion vehicles from a massive amount of traffic data with the best execution efficiency on spark platform. It works in a hybrid approach between depth first and breadth first to handle the big data in distributed clusters. Experiment results show that the proposed algorithm, HD-FIM, outperforms the existing typical frequent itemset mining algorithms through practical vehicle set extraction calculations and it can be applied in any applicable traffic big data.

A periodic high-utility sequential pattern (PHUSP) is a pattern that not only yields a high-utility (e.g. high profit) but also appears regularly in a sequence database. Finding PHUSPs is useful for several applications such as market basket analysis, where it can reveal recurring and profitable customer behavior. Although discovering PHUSPs is desirable, it is computationally difficult. To discover PHUSPs efficiently, this paper proposes a structure for periodic high-utility sequential pattern mining (PHUSPM) named PUSP. Furthermore, to reduce the search space and speed up PHUSPM, a pruning strategy is developed. This results in an efficient algorithm called periodic high-utility sequential pattern optimal miner (PUSOM). An experimental evaluation was performed on both synthetic and real-life datasets to compare the performance of PUSOM with state-of-the-art PHUSPM algorithms in terms of execution time, memory usage and scalability. Experimental results show that the PUSOM algorithm can efficiently discover the complete set of PHUSPs. Moreover, it outperforms the other four algorithms as the former can prune many unpromising patterns using its designed structure and pruning strategy.

Tanimoto, or extended Jaccard, is an important similarity measure which has seen prominent use in fields such as data mining and chemoinformatics. Many of the existing state-of-the-art methods for market basket analysis, plagiarism and anomaly detection, compound database search, and ligand-based virtual screening rely heavily on identifying Tanimoto nearest neighbors. Given the rapidly increasing size of data that must be analyzed, new algorithms are needed that can speed up nearest neighbor search, while at the same time providing reliable results. While many search algorithms address the complexity of the task by retrieving only some of the nearest neighbors, we propose a method that finds all of the exact nearest neighbors efficiently by leveraging recent advances in similarity search filtering. We provide tighter filtering bounds for the Tanimoto coefficient and show that our method, TAPNN, greatly outperforms existing baselines across a variety of real-world datasets and similarity thresholds.

In this paper we use the information theoretic Infomap algorithm (Rosvall and Bergstrom in Proc Natl Acad Sci 105(4):1118---1123, 2008) iteratively in order to cluster the direct citation network of the Astro Data Set (publications in 59 astrophysical journals between 2003 and 2010.) We obtain 22 clusters of documents from the giant component of the network that we interpret as constituting `topics' in the field of astrophysics. Upon investigation of the content of the topics we find a grouping of topics by shared features of their `journal signature', that is the journals that are most characteristic for a topic due to their popularity and distinctiveness. These groups of topics match sub disciplines within the field. We generate a cognitive map of the field using a topic affinity network that shows what topics are disproportionally well connected (by citations) to other topics. The topology of the topic affinity network highlights a high-level organization of the field by sub-discipline and observational distance of the research object from Earth.

Retailers routinely use association mining to investigate trends in the use of their products. In the medical world, association mining is mostly used to identify associations between symptoms and diseases, or between drugs and adverse events. In comparison, there is a relative paucity of work that focuses on relationships between drugs exclusively. In this work, we use the Medical expenditure panel survey to examine relationships between drugs in the United States. In addition to examining the rules generated by association mining, we introduce the notion of a target drug network and demonstrate via different drugs that it can offer additional medical insight. For example, we were able to find drugs that are commonly taken together despite containing the same active compound. Future work can expand on the concept of target drug network, for example, by annotating the networks with the compounds and intended uses of each drug, to yield additional insight for pharmacosurveillance as well as pharmaceutical companies.

With the advent of online marketplaces, the buying practices have changed. The Amazon co-purchase network provides us with dynamic snapshots of co-purchases. We analyse the various properties of the graph like clustering co-efficient, degree distributions, etc. and try to reason out the underlying relation between such distributions. In an attempt to understand the motifs in buying pattern, we propose algorithms to mine patterns of interest. In the end, we take into account all these various parameters and develop recommendation algorithms to suit the needs of the customers.

Data mining is the process of determining new, unanticipated, valuable patterns from existing databases by considering historical and recent developments in statistics, artificial intelligence, and machine learning. It can help companies focus on the most important information in their data warehouses. Association rule mining is one of the most highly researched and popular data mining techniques for finding associations between items in a set. It is frequently used in marketing, advertising, and inventory control. Typically, association rules only consider items in transactions (positive association rules). They do not consider items that do not occur together, which can be used to create rules that are also useful for market basket analysis. Also, existing algorithms often generate too many candidate itemsets when mining the data and scan the database multiple times. To resolve these issues in association rule mining algorithms, we propose SARIC (set particle swarm optimization for association rules using the itemset range and correlation coefficient). Our method uses set particle swarm optimization to generate association rules from a database and considers both positive and negative occurrences of attributes. SARIC applies the itemset range and correlation coefficient so that we do not need to specify the minimum support and confidence, because it automatically determines them quickly and objectively. We verified the efficiency of SARIC using two differently sized databases. Our simulation results demonstrate that SARIC generates more promising results than Apriori, Eclat, HMINE, and a genetic algorithm.

We introduce a model which differs from the well-known multivariate logit model (MVL) used to analyze the cross-category dependence in market baskets by the addition of binary hidden variables. This model is called restricted Boltzmann machine (RBM) and new to the marketing literature. Extant applications of the MVL model for higher numbers of categories typically follow a two-step approach as simultaneous maximum likelihood estimation is computationally infeasible. In contrast to the MVL, the RBM can be simultaneously estimated by maximum likelihood even for a higher number of categories as long as the number of hidden variables is moderate. We measure the cross-category dependence by pairwise marginal cross effects which are obtained using estimated coefficients and sampling of baskets. In the empirical study, we analyze market baskets consisting of the 60 most frequently purchased categories of the assortment of a supermarket. For a validation data set, the RBM performs better than the MVL model estimated by maximum pseudo-likelihood. For our data, about 75 % of the baskets are reproduced by the model without cross-category dependence, but 25 % of the baskets cannot be adequately modeled if cross effects are ignored. Moreover, it turns out that both the number of significant cross effects and their relationships can be grasped rather easily.

Market basket analysis (MBA) is a powerful and common practice in modern retailing that has some limitations stemming from the fact that it infers purchase sequence from joint-purchasing data. However, internet retailers automatically collect purchase-sequence data from their shoppers, and new technology is available for traditional (bricks and mortar) retailers to do the same, making it possible to analyze purchase sequences, rather than inferring them from joint purchases. This study first compares and contrasts traditional market basket analysis with a sequential extension, and then proposes a framework for purchase-sequence analysis, which is illustrated utilizing shopping trip data from one grocery store.

The field of market basket analysis, the search for meaningful associations in customer purchase data, is one of the oldest areas of data mining. The typical solution involves the mining and analysis of association rules, which take the form of statements such as “people who buy diapers are likely to buy beer”. It is well-known, however, that typical transaction datasets can support hundreds or thousands of obvious association rules for each interesting rule, and filtering through the rules is a non-trivial task (Klemettinen et al. In: Proceedings of CIKM, pp 401–407, 1994). One may use an interestingness measure to quantify the usefulness of various rules, but there is no single agreed-upon measure and different measures can result in very different rankings of association rules. In this work, we take a different approach to mining transaction data. By modeling the data as a product network, we discover expressive communities (clusters) in the data, which can then be targeted for further analysis. We demonstrate that our network based approach can concisely isolate influence among products, mitigating the need to search through massive lists of association rules. We develop an interestingness measure for communities of products and show that it isolates useful, actionable communities. Finally, we build upon our experience with product networks to propose a comprehensive analysis strategy by combining both traditional and network-based techniques. This framework is capable of generating insights that are difficult to achieve with traditional analysis methods.

Increasingly, taxonomies are being developed and used by industry practitioners to facilitate information interoperability and retrieval. Within a single industrial domain, there exist many taxonomies that are intended for different applications. Industry specific taxonomies often represent the vocabularies that are commonly used by the practitioners. Their jobs are multi-faceted, which include checking for code and regulatory compliance. As such, it will be very desirable if industry practitioners are able to easily locate and browse regulations of interest. In practice, multiple sources of government regulations exist and they are often organized and classified by the needs of the issuing agencies that enforce them rather than the needs of the communities that use them. One way to bridge these two distinct needs is to develop methods and tools that enable practitioners to browse and retrieve government regulations using their own terms and vocabularies, for example, via existing industry taxonomies. The mapping from a single taxonomy to a single regulation is a trivial keyword matching task. We examine a relatedness analysis approach for mapping a single taxonomy to multiple regulations. We then present an approach for mapping multiple taxonomies to a single regulation by measuring the relatedness of concepts. Cosine similarity, Jaccard coefficient and market basket analysis are used to measure the semantic relatedness between concepts from two different taxonomies. Preliminary evaluations of the three relatedness analysis measures are performed using examples from the civil engineering and building industry. These examples illustrate the potential benefits of regulatory usage from the mapping between various taxonomies and regulations.

It has been claimed that the discovery of association rules is well suited for applications of market basket analysis to reveal regularities in the purchase behaviour of customers. However today, one disadvantage of associations discovery is that there is no provision for taking into account the business value of an association. Therefore, recent work indicates that the discovery of interesting rules can in fact best be addressed within a microeconomic framework. This study integrates the discovery of frequent itemsets with a (microeconomic) model for product selection (PROFSET). The model enables the integration of both quantitative and qualitative (domain knowledge) criteria. Sales transaction data from a fully automated convenience store are used to demonstrate the effectiveness of the model against a heuristic for product selection based on product-specific profitability. We show that with the use of frequent itemsets we are able to identify the cross-sales potential of product items and use this information for better product selection. Furthermore, we demonstrate that the impact of product assortment decisions on overall assortment profitability can easily be evaluated by means of sensitivity analysis.

Recently, constraint-based mining of itemsets for questions like “find all frequent itemsets whose total price is at least d50” has attracted much attention. Two classes of constraints, monotone and antimonotone, have been very useful in this area. There exist algorithms that efficiently take advantage of either one of these two classes, but no previous algorithms can efficiently handle both types of constraints simultaneously. In this paper, we present DualMiner, the first algorithm that efficiently prunes its search space using both monotone and antimonotone constraints. We complement a theoretical analysis and proof of correctness of DualMiner with an experimental study that shows the efficacy of DualMiner compared to previous work.

Collaborative recommender systems allow personalization for e-commerce by exploiting similarities and dissimilarities among customers' preferences. We investigate the use of association rule mining as an underlying technology for collaborative recommender systems. Association rules have been used with success in other domains. However, most currently existing association rule mining algorithms were designed with market basket analysis in mind. Such algorithms are inefficient for collaborative recommendation because they mine many rules that are not relevant to a given user. Also, it is necessary to specify the minimum support of the mined rules in advance, often leading to either too many or too few ruless this negatively impacts the performance of the overall system. We describe a collaborative recommendation technique based on a new algorithm specifically designed to mine association rules for this purpose. Our algorithm does not require the minimum support to be specified in advance. Rather, a target range is given for the number of rules, and the algorithm adjusts the minimum support for each user in order to obtain a ruleset whose size is in the desired range. Rules are mined for a specific target user, reducing the time required for the mining process. We employ associations between users as well as associations between items in making recommendations. Experimental evaluation of a system based on our algorithm reveals performance that is significantly better than that of traditional correlation-based approaches.

Rule induction methods are widely applied tools for mining large data bases. They are often used as a starting point in undirected data mining, i.e. when you do not know what specific patterns to look for. One form of rule induction methods are association rules which have their origin in market basket analysis. Since an evaluation of their results is often hard due to the mass of output, pruning methods are needed to turn the output of association rules into a manageable number of patterns. We will present some statistical measures and their depictions that are useful to assess the quality of an association rule. We will show plots portraying confidence and support for individual rules as well as for sets of rules as alternatives to displays currently used in commercial software. A new quality measure for association rules, the doc, will be introduced which overcomes some of the problems of support and confidence of association rules and can be visualised even for hundreds of rules in one diagram.

Summary Concepts and methods for use in the study of plant community variation across a single landscape, or for the comparative analysis of two or more landscapes, are presented. The method is called affinity analysis because it produces a sorting and scaling of all the communities in a data set according to degree of differentiation in composition away from an objectively identified subset of modal communities. Modal communities possess maximum affinity with the whole landscape because they contain a large number of the species common throughout the landscape. The modal communities provide a kind of central tendency for the landscape and an anchor for depicting the dispersion of all other communities in it. To accomplish affinity analysis, the Jaccard similarity coefficient and the Wilcoxon Tstatistic are used in a two-step transformation of the primary species-by-site data matrix. From these calculations we obtain both a visual image, the S-T graph, and summary statistics for the landscape-wide diversity described by the data. One summary statistic is a high-order diversity measure for the total patchwork of communities. We refer to this measure as mosaic diversity. The analytical results are referred to collectively as the "metastructure' because they provide a general, abstract, characterization of any set of community data. This abstract rendering of the data allows comparison of pattern and variation between taxonomically, geographically, and temporally different landscapes. Either presence/absence or abundance data may be used. Examples using artificial and field data are presented. All but one of the field data sets showed a significantly higher mosaic diversity than would be expected from a randomly constructed landscape. We also show how the new methods may be used with ordination to explore intralandscape patterns in more detail than was previously possible. In addition to statistical matters, ecological and evoutionary interpretations of affinity analysis are discussed. Topics included in this discussion involve reasoning about the influences on diversity arising from micro- and macroevolution, species packing and association, environmental gradients, differential fitness expression among species-populations, continuity in community variation, and the uses of both presence/ absence and abundance data in community studies. Among the examples provided, mosaic diversity is independent of 13 diversity (Whittaker, 1972).

Affinity analysis (AA) is a group of methods for the study of the variation of degrees of compositional relatedness among all of the communities in a landscape. AA cab be used with statistical inference to compare mosaic diversities (89-1) among different landscapes, identify unusual sites within a landscape, and determine when a pair of sites are significantly different from each other. These procedures were done with a set of samples from meadows in the Danube River Valley near Ulm by using random simulations to provide expectations of the summary statistics. The sampling and statistical limitations of AA were discussed.