Search

Automated scoring of free drawings or images as responses has yet to be used in large-scale assessments of student achievement. In this study, we propose artificial neural networks to classify these types of graphical responses from a TIMSS 2019 item. We are comparing classification accuracy of convolutional and feed-forward approaches. Our results show that convolutional neural networks (CNNs) outperform feed-forward neural networks in both loss and accuracy. The CNN models classified up to 97.53% of the image responses into the appropriate scoring category, which is comparable to, if not more accurate, than typical human raters. These findings were further strengthened by the observation that the most accurate CNN models correctly classified some image responses that had been incorrectly scored by the human raters. As an additional innovation, we outline a method to select human-rated responses for the training sample based on an application of the expected response function derived from item response theory. This paper argues that CNN-based automated scoring of image responses is a highly accurate procedure that could potentially replace the workload and cost of second human raters for international large-scale assessments (ILSAs), while improving the validity and comparability of scoring complex constructed-response items. [ABSTRACT FROM AUTHOR]

Background: Automatic and accurate recognition of various biomedical named entities from literature is an important task of biomedical text mining, which is the foundation of extracting biomedical knowledge from unstructured texts into structured formats. Using the sequence labeling framework and deep neural networks to implement biomedical named entity recognition (BioNER) is a common method at present. However, the above method often underutilizes syntactic features such as dependencies and topology of sentences. Therefore, it is an urgent problem to be solved to integrate semantic and syntactic features into the BioNER model. Results: In this paper, we propose a novel biomedical named entity recognition model, named BioByGANS (BioBERT/SpaCy-Graph Attention Network-Softmax), which uses a graph to model the dependencies and topology of a sentence and formulate the BioNER task as a node classification problem. This formulation can introduce more topological features of language and no longer be only concerned about the distance between words in the sequence. First, we use periods to segment sentences and spaces and symbols to segment words. Second, contextual features are encoded by BioBERT, and syntactic features such as part of speeches, dependencies and topology are preprocessed by SpaCy respectively. A graph attention network is then used to generate a fusing representation considering both the contextual features and syntactic features. Last, a softmax function is used to calculate the probabilities and get the results. We conduct experiments on 8 benchmark datasets, and our proposed model outperforms existing BioNER state-of-the-art methods on the BC2GM, JNLPBA, BC4CHEMD, BC5CDR-chem, BC5CDR-disease, NCBI-disease, Species-800, and LINNAEUS datasets, and achieves F1-scores of 85.15%, 78.16%, 92.97%, 94.74%, 87.74%, 91.57%, 75.01%, 90.99%, respectively. Conclusion: The experimental results on 8 biomedical benchmark datasets demonstrate the effectiveness of our model, and indicate that formulating the BioNER task into a node classification problem and combining syntactic features into the graph attention networks can significantly improve model performance. [ABSTRACT FROM AUTHOR]

In order to learn the mappings from words to referents, children must integrate co-occurrence information across individually ambiguous pairs of scenes and utterances, a challenge known as crosssituational word learning. In machine learning, recent multimodal neural networks have been shown to learn meaningful visual-linguistic mappings from cross-situational data, as needed to solve problems such as image captioning and visual question answering. These networks are potentially appealing as cognitive models because they can learn from raw visual and linguistic stimuli, something previous cognitive models have not addressed. In this paper, we examine whether recent machine learning approaches can help explain various behavioral phenomena from the psychological literature on crosssituational word learning. We consider two variants of a multimodal neural network architecture and look at seven different phenomena associated with cross-situational word learning and word learning more generally. Our results show that these networks can learn word-referent mappings from a single epoch of training, mimicking the amount of training commonly found in cross-situational word learning experiments. Additionally, these networks capture some, but not all of the phenomena we studied, with all of the failures related to reasoning via mutual exclusivity. These results provide insight into the kinds of phenomena that arise naturally from relatively generic neural network learning algorithms, and which word learning phenomena require additional inductive biases. [ABSTRACT FROM AUTHOR]

This paper investigates how institutional pressures affect the development of Circular Economy (CE) in firms. Using Institutional Entrepreneurship as a theoretical framework, this paper considers three different levels of institutional pressures (coercive, normative, and mimetic) to examine the effect of each pressure and their interactions on the development of CE. Seeking to clarify the debate on the effect of institutional pressures, this paper considers that the main limitation arises from the fact that previous research has analysed the relationship between institutional pressures without considering the interaction between them and the non-linearity of the processes. Deviating from previous papers, our analysis combines regression methods with Machine learning (i.e. Artificial Neural Networks), and employs data from the EU survey on Public Consultation on the Circular Economy. This research finds that while coercive pressures have a compulsory effect on the development of CE, mimetic and normative pressures do not have an effect by themselves, but only in interaction with coercive pressures. Moreover, this paper shows that the application of machine learning tools has an important contribution in solving interaction problems. From the perspective of environmental policy, this means that a comprehensive policy is required, which implies the coexistence or interaction of the three types of pressures. [Display omitted] • Discrepancies on the effect of institutional pressures are due to methodology. • Coercive pressure has a compulsory effect on the development of Circular Economy. • Mimetic & Normative pressures only have effect in interaction with coercive pressure. • Artificial Neural Networks contribute to solving interaction problems. • Comprehensive environmental policy that considers these interactions is required. [ABSTRACT FROM AUTHOR]

Analyzing source code to solve software engineering problems such as fault prediction, cost, and effort estimation always receives attention of researchers as well as companies. The traditional approaches are based on machine learning, and software metrics obtained by computing standard measures of software projects. However, these methods have faced many challenges due to limitations of using software metrics which were not enough to capture the complexity of programs. To overcome the limitations, this paper aims to solve software engineering problems by exploring information of programs' abstract syntax trees (ASTs) instead of software metrics. We propose two combination models between a tree-based convolutional neural network (TBCNN) and k-Nearest Neighbors (kNN), support vector machines (SVMs) to exploit both structural and semantic ASTs' information. In addition, to deal with high-dimensional data of ASTs, we present several pruning tree techniques which not only reduce the complexity of data but also enhance the performance of classifiers in terms of computational time and accuracy. We survey many machine learning algorithms on different types of program representations including software metrics, sequences, and tree structures. The approaches are evaluated based on classifying 52000 programs written in C language into 104 target labels. The experiments show that the tree-based classifiers dramatically achieve high performance in comparison with those of metrics-based or sequences-based; and two proposed models TBCNN + SVM and TBCNN + kNN rank as the top and the second classifiers. Pruning redundant AST branches leads to not only a substantial reduction in execution time but also an increase in accuracy. [ABSTRACT FROM AUTHOR]

It is well known that the feedforward neural networks meet numbers of difficulties in the applications because of its slow learning speed. The extreme learning machine (ELM) is a new single hidden layer feedforward neural network method aiming at improving the training speed. Nowadays ELM algorithm has received wide application with its good generalization performance under fast learning speed. However, there are still several problems needed to be solved in ELM. In this paper, a new improved ELM algorithm named R-ELM is proposed to handle the multicollinear problem appearing in calculation of the ELM algorithm. The proposed algorithm is employed in bearing fault detection using stator current monitoring. Simulative results show that R-ELM algorithm has better stability and generalization performance compared with the original ELM and the other neural network methods. [ABSTRACT FROM AUTHOR]

The techniques of fuzzy measure and fuzzy integral have been successfully applied in various real-world applications. The determination of fuzzy measures is the most difficult part in problem solving. Signed efficiency measure, which is a special kind of fuzzy measure with the best representation ability but the highest complexity, is even harder to determine. Some methodologies have been developed for solving this problem such as artificial neural networks (ANNs) and genetic algorithms (GAs). However, none of the existing methods can outperform the others with unique advantages. Thus, there is a strong need to develop a new technique for learning distinct signed efficiency measures from data. Extreme learning machine (ELM) is a new learning paradigm for training single hidden layer feed-forward networks (SLFNs) with randomly chosen input weights and analytically determined output weights. In this paper, we propose an ELM based algorithm for signed efficiency measure determination. Experimental comparisons demonstrate the effectiveness of the proposed method in both time and accuracy. [ABSTRACT FROM AUTHOR]

Abstract: Widespread digitization of information in today’s internet age has intensified the need for effective textual document classification algorithms. Most real life classification problems, including text classification, genetic classification, medical classification, and others, are complex in nature and are characterized by high dimensionality. Current solution strategies include Naïve Bayes (NB), Neural Network (NN), Linear Least Squares Fit (LLSF), k-Nearest-Neighbor (kNN), and Support Vector Machines (SVM); with SVMs showing better results in most cases. In this paper we introduce a new approach called dynamic architecture for artificial neural networks (DAN2) as an alternative for solving textual document classification problems. DAN2 is a scalable algorithm that does not require parameter settings or network architecture configuration. To show DAN2 as an effective and scalable alternative for text classification, we present comparative results for the Reuters-21578 benchmark dataset. Our results show DAN2 to perform very well against the current leading solutions (kNN and SVM) using established classification metrics. [Copyright &y& Elsevier]

The particulate matter PM 10 concentrations have been impacting hospital admissions due to respiratory diseases. The air pollution studies seek to understand how this pollutant affects the health system. Since prediction involves several variables, any disparity causes a disturbance in the overall system, increasing the difficulty of the models' development. Due to the complex nonlinear behavior of the problem and their influencing factors, Artificial Neural Networks are attractive approaches for solving estimations problems. This paper explores two neural network architectures denoted unorganized machines: the echo state networks and the extreme learning machines. Beyond the standard forms, models variations are also proposed: the regularization parameter (RP) to increase the generalization capability, and the Volterra filter to explore nonlinear patterns of the hidden layers. To evaluate the proposed models' performance for the hospital admissions estimation by respiratory diseases, three cities of São Paulo state, Brazil: Cubatão, Campinas and São Paulo, are investigated. Numerical results show the standard models' superior performance for most scenarios. Nevertheless, considering divergent intensity in hospital admissions, the RP models present the best results in terms of data dispersion. Finally, an overall analysis highlights the models' efficiency to assist the hospital admissions management during high air pollution episodes. [ABSTRACT FROM AUTHOR]

In this paper, we propose two cooperative ensemble learning algorithms, i.e., NegBagg and NegBoost, for designing neural network (NN) ensembles. The proposed algorithms incrementally train different individual NNs in an ensemble using the negative correlation learning algorithm. Bagging and boosting algorithms are used in NegBagg and NegBoost, respectively, to create different training sets for different NNs in the ensemble. The idea behind using negative correlation learning in conjunction with the bagging/boosting algorithm is to facilitate interaction and cooperation among NNs during their training. Both NegBagg and NegBoost use a constructive approach to automatically determine the number of hidden neurons for NNs. NegBoost also uses the constructive approach to automatically determine the number of NNs for the ensemble. The two algorithms have been tested on a number of benchmark problems in machine learning and NNs, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, satellite, soybean, and waveform problems. The experimental results show that NegBagg and NegBoost require a small number of training epochs to produce compact NN ensembles with good generalization. [ABSTRACT FROM AUTHOR]

Support vector machines (SVMs) have been extensively used. However, it is known that SVMs face difficulty in solving large complex problems due to the intensive computation involved in their training algorithms, which are at least quadratic with respect to the number of training examples. This paper proposes a new, simple, and efficient network architecture which consists of several SVMs each trained on a small subregion of the whole data sampling space and the same number of simple neural quantizer modules which inhibit the outputs of all the remote SVMs and only allow a single local SVM to fire (produce actual output) at any time. In principle, this region-computing based modular network method can significantly reduce the learning time of SYM algorithms without sacrificing much generalization performance. The experiments on a few real large complex bench- mark problems demonstrate that our method can be significantly faster than single SVMs without losing much generalization performance. [ABSTRACT FROM AUTHOR]

Non-orthogonal multiple access (NOMA) emerges as a promising candidate for 5G, which radically alters the way users share the spectrum. In the NOMA system, user clustering (UC) becomes another research issue as grouping the users on different subcarriers with different power levels has a significant impact on spectral utilization. In previous literature, plenty of works have been devoted to solving the UC problem. Recently, the artificial neural network (ANN) has gained considerable attention due to the availability of UC datasets, obtained from the Brute-Force search (BF-S) method. In this paper, deep neural network-based UC (DNN-UC) is employed to effectively characterize the nonlinearity between the cluster formation with channel diversity and transmission powers. Compared to the ANN-UC, the DNN-UC is more competent as UC is a non-convex NP-complete problem, which cannot be entirely captured by the ANN model. In this work, the DNN-UC is first trained with the training samples and then validated with the testing samples to examine its mean square error (MSE) and throughput performance in an asymmetrical fading NOMA channel. Unlike the ANN-UC, the DNN-UC model offers greater room for hyper-parameter optimizations to maximize its learning capability. With the optimized hyper-parameters, the DNN-UC can achieve near-optimal throughput performance, approximately 97% of the throughput of the BF-S method. [ABSTRACT FROM AUTHOR]

In this paper, a proposed algorithm that dynamically changes the neural network structure is presented. The structure is changed based on some features in the cascade correlation algorithm. Cascade correlation is an important algorithm that is used to solve the actual problem by artificial neural networks as a new architecture and supervised learning algorithm. This process optimizes the architectures of the network which intends to accelerate the learning process and produce better performance in generalization. Many researchers have to date proposed several growing algorithms to optimize the feedforward neural network architectures. The proposed algorithm has been tested on various medical data sets. The results prove that the proposed algorithm is a better method to evaluate the accuracy and flexibility resulting from it. [ABSTRACT FROM AUTHOR]

Optimization seeks to find inputs for an objective function that result in a maximum or minimum. Optimization methods are divided into exact and approximate (algorithms). Several optimization algorithms imitate natural phenomena, laws of physics, and behavior of living organisms. Optimization based on algorithms is the challenge that underlies machine learning, from logistic regression to training neural networks for artificial intelligence. In this paper, a new algorithm called two-stage optimization (TSO) is proposed. The TSO algorithm updates population members in two steps at each iteration. For this purpose, a group of good population members is selected and then two members of this group are randomly used to update the position of each of them. This update is based on the first selected good member at the first stage, and on the second selected good member at the second stage. We describe the stages of the TSO algorithm and model them mathematically. Performance of the TSO algorithm is evaluated for twenty-three standard objective functions. In order to compare the optimization results of the TSO algorithm, eight other competing algorithms are considered, including genetic, gravitational search, grey wolf, marine predators, particle swarm, teaching-learning-based, tunicate swarm, and whale approaches. The numerical results show that the new algorithm is superior and more competitive in solving optimization problems when compared with other algorithms. [ABSTRACT FROM AUTHOR]

Machine-learning technologies have exhibited robust competences in solving many petroleum engineering problems. The accurate predictivity and fast computational speed enable a large volume of time-consuming engineering processes such as history-matching and field development optimization. The Southwest Regional Partnership on Carbon Sequestration (SWP) project desires rigorous history-matching and multi-objective optimization processes, which fits the superiorities of the machine-learning approaches. Although the machine-learning proxy models are trained and validated before imposing to solve practical problems, the error margin would essentially introduce uncertainties to the results. In this paper, a hybrid numerical machine-learning workflow solving various optimization problems is presented. By coupling the expert machine-learning proxies with a global optimizer, the workflow successfully solves the history-matching and CO2 water alternative gas (WAG) design problem with low computational overheads. The history-matching work considers the heterogeneities of multiphase relative characteristics, and the CO2-WAG injection design takes multiple techno-economic objective functions into accounts. This work trained an expert response surface, a support vector machine, and a multi-layer neural network as proxy models to effectively learn the high-dimensional nonlinear data structure. The proposed workflow suggests revisiting the high-fidelity numerical simulator for validation purposes. The experience gained from this work would provide valuable guiding insights to similar CO2 enhanced oil recovery (EOR) projects. [ABSTRACT FROM AUTHOR]