Search

This paper describes a novel end‐to‐end deep generative model‐based speaker recognition system using prosodic features. The usefulness of variational autoencoders (VAE) in learning the speaker‐specific prosody representations for the speaker recognition task is examined herein for the first time. The speech signal is first automatically segmented into syllable‐like units using vowel onset points (VOP) and energy valleys. Prosodic features, such as the dynamics of duration, energy, and fundamental frequency (F0), are then extracted at the syllable level and used to train/adapt a speaker‐dependent VAE from a universal VAE. The initial comparative studies on VAEs and traditional autoencoders (AE) suggest that the former can efficiently learn speaker representations. Investigations on the impact of gender information in speaker recognition also point out that gender‐dependent impostor banks lead to higher accuracies. Finally, the evaluation on the NIST SRE 2010 dataset demonstrates the usefulness of the proposed approach for speaker recognition. [ABSTRACT FROM AUTHOR]

A semitransparent photovoltaic‐thermal (PV/T) air collector can produce electricity and heat simultaneously. To maximize the thermal and overall efficiency of the semitransparent PV/T air collector, its availability should be maximum; this can be determined through a Markov analysis. In this paper, a Markov model is developed to select an optimized number of semitransparent PV modules in service with five states and two states by considering two parameters, namely failure rate (λ) and repair rate (µ). Three artificial neural network (ANN) models are developed to obtain the minimum cost, minimum temperature, and maximum thermal efficiency of the semitransparent PV/T air collector by setting its type appropriately and optimizing the number of photovoltaic modules and cost. An attempt is also made to achieve maximum thermal and overall efficiency for the semitransparent PV/T air collector by using ANN after obtaining its minimum temperature and available solar radiation. [ABSTRACT FROM AUTHOR]

Named entity recognition (NER) continues to be an important task in natural language processing because it is featured as a subtask and/or subproblem in information extraction and machine translation. In Urdu language processing, it is a very difficult task. This paper proposes various deep recurrent neural network (DRNN) learning models with word embedding. Experimental results demonstrate that they improve upon current state‐of‐the‐art NER approaches for Urdu. The DRRN models evaluated include forward and bidirectional extensions of the long short‐term memory and back propagation through time approaches. The proposed models consider both language‐dependent features, such as part‐of‐speech tags, and language‐independent features, such as the "context windows" of words. The effectiveness of the DRNN models with word embedding for NER in Urdu is demonstrated using three datasets. The results reveal that the proposed approach significantly outperforms previous conditional random field and artificial neural network approaches. The best f‐measure values achieved on the three benchmark datasets using the proposed deep learning approaches are 81.1%, 79.94%, and 63.21%, respectively. [ABSTRACT FROM AUTHOR]

In this paper, we propose a rank‐weighted reconstruction feature to improve the robustness of a feed‐forward deep neural network (FFDNN)‐based acoustic model. In the FFDNN‐based acoustic model, an input feature is constructed by vectorizing a submatrix that is created by slicing the feature vectors of frames within a context window. In this type of feature construction, the appropriate context window size is important because it determines the amount of trivial or discriminative information, such as redundancy, or temporal context of the input features. However, we ascertained whether a single parameter is sufficiently able to control the quantity of information. Therefore, we investigated the input feature construction from the perspectives of rank and nullity, and proposed a rank‐weighted reconstruction feature herein, that allows for the retention of speech information components and the reduction in trivial components. The proposed method was evaluated in the TIMIT phone recognition and Wall Street Journal (WSJ) domains. The proposed method reduced the phone error rate of the TIMIT domain from 18.4% to 18.0%, and the word error rate of the WSJ domain from 4.70% to 4.43%. [ABSTRACT FROM AUTHOR]

This paper explores new techniques that are based on a hidden‐layer linear transformation for fast speaker adaptation used in deep neural networks (DNNs). Conventional methods using affine transformations are ineffective because they require a relatively large number of parameters to perform. Meanwhile, methods that employ singular‐value decomposition (SVD) are utilized because they are effective at reducing adaptive parameters. However, a matrix decomposition is computationally expensive when using online services. We propose the use of an extended diagonal linear transformation method to minimize adaptation parameters without SVD to increase the performance level for tasks that require smaller degrees of adaptation. In Korean large vocabulary continuous speech recognition (LVCSR) tasks, the proposed method shows significant improvements with error‐reduction rates of 8.4% and 17.1% in five and 50 conversational sentence adaptations, respectively. Compared with the adaptation methods using SVD, there is an increased recognition performance with fewer parameters. [ABSTRACT FROM AUTHOR]

Projection spectral analysis is investigated and refined in this paper, in order to unify principal component analysis and independent component analysis. Singular value decomposition and spectral theorems are applied to nonsymmetric correlation or covariance matrices with multiplicities or singularities, where projections and nilpotents are obtained. Therefore, the suggested approach not only utilizes a sum‐product of orthogonal projection operators and real distinct eigenvalues for squared singular values, but also reduces the dimension of correlation or covariance if there are multiple zero eigenvalues. Moreover, incremental learning strategies of projection spectral analysis are also suggested to improve the performance. [ABSTRACT FROM AUTHOR]

This paper proposes a human‐like sign‐language learning method that uses a deep‐learning technique. Inspired by the fact that humans can learn sign language from just a set of pictures in a book, in the proposed method, the input data are pre‐processed into an image. In addition, the network is partially pre‐trained to imitate the preliminarily obtained knowledge of humans. The learning process is implemented with a well‐known network, that is, a convolutional neural network. Twelve sign actions are learned in 10 situations, and can be recognized with an accuracy of 99% in scenarios with low‐cost equipment and limited data. The results show that the system is highly practical, as well as accurate and robust. [ABSTRACT FROM AUTHOR]

The application of deep neural networks (DNNs) to connect the world with cyber physical systems (CPSs) has attracted much attention. However, DNNs require a large amount of memory and computational cost, which hinders their use in the relatively low‐end smart devices that are widely used in CPSs. In this paper, we aim to determine whether DNNs can be efficiently deployed and operated in low‐end smart devices. To do this, we develop a method to reduce the memory requirement of DNNs and increase the inference speed, while maintaining the performance (for example, accuracy) close to the original level. The parameters of DNNs are decomposed using a hybrid of canonical polyadic–singular value decomposition, approximated using a tensor power method, and fine‐tuned by performing iterative one‐shot hybrid fine‐tuning to recover from a decreased accuracy. In this study, we evaluate our method on frequently used networks. We also present results from extensive experiments on the effects of several fine‐tuning methods, the importance of iterative fine‐tuning, and decomposition techniques. We demonstrate the effectiveness of the proposed method by deploying compressed networks in smartphones. [ABSTRACT FROM AUTHOR]

Recently, deep recurrent neural networks have achieved great success in various machine learning tasks, and have also been applied for sound event detection. The detection of temporally overlapping sound events in realistic environments is much more challenging than in monophonic detection problems. In this paper, we present an approach to improve the accuracy of polyphonic sound event detection in multichannel audio based on gated recurrent neural networks in combination with auditory spectral features. In the proposed method, human hearing perception-based spatial and spectral-domain noise-reduced harmonic features are extracted from multichannel audio and used as high-resolution spectral inputs to train gated recurrent neural networks. This provides a fast and stable convergence rate compared to long short-term memory recurrent neural networks. Our evaluation reveals that the proposed method outperforms the conventional approaches. [ABSTRACT FROM AUTHOR]

Parkinson's disease is a growing and chronic movement disorder, and its diagnosis is difficult especially at the initial stages. In this paper, movement characteristics extracted by a computer using multilayer back propagation neural network mapping are converted to the symptoms of this disease. Then, modulation of three classifiers of C4.5, k-nearest neighbors, and support vector machine with majority voting are applied to support experts in diagnosing the disease. The purpose of this study is to choose appropriate characteristics and increase the accuracy of the diagnosis. Experiments were performed to demonstrate the improvement of Parkinson's disease diagnosis using this method. [ABSTRACT FROM AUTHOR]

This paper presents a versatile image recognition method, combining conventional pattern matching and a learning system. The proposed system first quickly determines candidate regions containing the target object in the image, and then examines the result by using the learning system. In the pattern matching, “active search” is used, which quickly extracts the target object using multiple two-dimensional reference images. As a learning system, an adaptive fuzzy inference neural network (AFINN) is used, which can perform knowledge processing. The AFINN automatically constructs the inference model, such as the selection of the necessary input information, and automatically acquires the inference rules through learning. Consequently, by applying the AFINN to the image recognition problem, highly versatile object recognition is realized without requiring explicit knowledge for recognition. The proposed system can be applied to a wide range of problems, since diversified objects can be recognized by simply modifying the registered reference images. In this paper, the method is applied to a sample problem, consisting of an image of the laboratory containing various objects, such as computers, monitors, and chairs, and it is verified that high recognition accuracy can be realized. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(3): 69–79, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20269 [ABSTRACT FROM AUTHOR]

In this paper, the environmental factors surrounding fruit trees are used as the explanatory variables, and the factors concerning the quality of the agricultural produce are used as the explained variables. The observed data are analyzed. The object of analysis is the Onshu orange (mandarin orange). The purposes of this paper are analysis of the effect of environmental factors on product quality, the target value settings for the controllable environmental factors in order to produce high-quality products, and evaluation of the cultivation environment for each item. A system for data analysis is proposed which extracts the useful information for the decision-making such as cultivation management, as well as harvest/shipping planning, from the observed data. The system uses a probabilistic neural network and learns the mapping from the describing variables consisting of observed data to the explained variables. By superposing the acquired knowledge, inverse modeling can be performed. It is shown that by using the proposed system, multivariate nonlinear inverse mapping can be realized instead of the linear inverse regression estimation used in conventional single regression analysis. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(11): 102–110, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10577 [ABSTRACT FROM AUTHOR]

In a radio communications network, all of the nodes cannot communicate with each other directly. Then packets are transferred from a source node to a destination node through several nodes. Therefore, we need to schedule transfer timing at each node, and communications routes must be assigned to minimize the total transfer time when many packet transfers are requested. This problem is divided into two problems: the communications route assignment problem and the scheduling problem. The former problem is subdivided into the communication route candidate extraction problem and the communication route selection problem. This paper first proposes an evaluation function (Cost) which gives the lowest limit of the total transfer time for the communication route assignment problem. Next this paper proposes a k-shortest route extraction procedure for the communication route extraction problem. This procedure is based on the k-shortest route algorithm. It prevents extraction of a route whose number of hops is more than the upper limit, sets an appropriate number of extraction routes, prevents loops, and prevents redundant route extraction. We also propose a greedy neural network procedure for the communication route selection problem. This procedure introduces the ω function into the operation equation of the neuron initial value setting based on the number of hops and a cost minimization term for the evaluation function. The procedure uses an appropriate termination condition for iterative computation. It also uses the first-order maximum neuron. Through simulations of 500-vertex communication network examples, the proposed procedure has the merits of high precision, short computation, and smaller number of computations in a region. © 1999 Scripta Technica, Syst Comp Jpn, 30(13): 52–60, 1999 [ABSTRACT FROM AUTHOR]

It is known that mutually coupled neural networks are suited for solving optimization problems. So far, we have shown that a neural network which is tolerant to unidirectional faults can be realized by selecting automatically one of the two complementary representations of the solution. It is also shown that a fault-tolerant neural network can be realized by a triplication structure with an appropriate merging function, even if both stuck-at-1 and 0 faults may occur simultaneously. In these realizations, the conventional property of a neuron, a component of the neural network, is utilized, where the output of the neuron is determined by the sigmoid function of the sum of its inputs multiplied by the respective weights (connection coefficients). This paper shows that if a neuron can have a higher function than the weighted sum of its inputs, a neural network which is tolerant to stuck-at-1 and 0 mixed faults can be realized by the duplication structure. Since the duplication structure suffices, it is expected that less hardware will be needed than in the triplication structure. Although the higher function of the neuron assumed in this paper has not actually been realized, we expect that such a function will be realized in hardware in the near future as a result of progress in integrated circuit technology. In this sense, the proposal in this paper is practically realistic, and provides a new means of implementing fault tolerance of neural networks. © 1999 Scripta Technica, Syst Comp Jpn, 30(10): 22–33, 1999 [ABSTRACT FROM AUTHOR]

This paper presents a method for designing neural networks using a genetic algorithm (GA) with deterministic mutation (DM) based on learning. The GA presented in this paper has a large framework including DM, which is performed on the basis of the results from neural network learning. It can achieve better convergence properties than traditional GAs. This framework is an evolutional system based on mutual interaction between DM and traditional genetic operators including stochastic mutation. It is also a model of transcription and reverse transcription in DNA. We show that the present method is better than conventional GAs with respect to convergence in learning. © 1998 Scripta Technica. Syst. Comp Jpn, 29(3): 10–17, 1998 [ABSTRACT FROM AUTHOR]

Recently, due to remarkable technological developments and because their logic can be modified flexible and easily, Field Programmable Gate Arrays (FPGAs) have increasingly been applied to hardware prototyping and the design of restructurable computer architctures. For this reason, it is highly accessory to establish logic design techniques for FPGAs. This paper describes methods for optimizing the circuits mapped in look-up table-type FPGAs. These methods apply the concept of permissible functions from the transduction method, which s a logic circuit optimization technique based on design improvement, and attempt to reduce the degree of redundancy. The following two methods were investigated: 1) reducing the number of blocks by using a logic block to replace another; and 2) together with replacing logic blocks, modifying the internal logic that implements a logic block as long as the outputs are not affected. Although the latter generally is more flexible, the former requires less processing time and sometimes produces better results. The effectiveness of these methods is demonstrated by showing that the degree of redundancy was reduced by about 9 percent when the methods were applied to circuits mapped in look-up table-type FPGAs. The methods proposed in this paper can also be applied to designs using standard cells and gate arrays. [ABSTRACT FROM AUTHOR]

Overlearning in the neural network can cause a failure in the approximation of a function because of the limited generalization power of the neural network. To solve this problem, it is considered effective that the neural network learns the probability density function of the learning samples, and the result "answer is impossible" is obtained when an input is given in the region containing no learning sample. This paper proposes a method in which the probability density of the learning sample is reflected on the output for the function by alternately inputting the learning sample and the random value. It is reported in this paper through a simulation that the proposed method works correctly in the multilayered perception (MLP). When this method is applied, there is a close relation among the SN ratio, the output norm and the probability density of the learning samples. [ABSTRACT FROM AUTHOR]

This paper proposes a method optically to inspect the state of each soldered lead of a semiconductor package. The surface of the soldered lead is recorded using a CCD camera from a direction perpendicular to it and illuminated from the same direction as the camera. The features of the pattern and the distribution of the brightness on soldered lead are extracted from the recorded image, so that the quality of the soldering is judged from them. It has already been proven that the method is effective to inspect bumps on a soldered part. This paper proves experimentally that the method is also effective to inspect soldered leads. The images of soldered leads have more variations than those of bumps. Therefore, the determination of condition that correctly represent the features and sections of an image is the key in this experiment. After the best conditions have been found, the experimental results using 508 samples (including 175 defective samples) show a 100 percent detection rate for defective samples and 95.7 percent of normal solderings. These results agree with those gained from human visual inspections. [ABSTRACT FROM AUTHOR]

This paper presents a neural network algorithm called SOLAR (Supervised One-shot Learning Algorithm for Real-valued inputs) that can complete learning by a single presentation of the real-valued inputs. The similarity matrix is introduced in SOLAR, which can measure the Euclidean distance between training instances represented by a real number. Based on the geometrical structure of the similarity matrix, the structure of the network, the connection weights, the learning parameters and the linear disciminant function are determined. Then, the learning is executed by a single presentation of the training set. The nonlinearly separable case by decomposing the set of supervisor signals into linearly separable sets. The main contribution of this paper is that SOLAR realizes a high-speed learning by a single presentation of the input data composed of real values and a large-scale complex set of data can be learned. This has been difficult in previous models. In addition, since the number of hidden units and the parameters are determined by the algorithm, there is no problem such as trial and error. [ABSTRACT FROM AUTHOR]

The number of interconnections between cells in a neural network, which is proportional to square of the number, causes a problem in the structure of the network when the number is large. The fast Fourier transform is a method to speed up matrix operation by using a butterfly operation. This paper proposes a method of reducing the number of connections between cells in a neural network by using a multilayer structure which is based on a butterfly operation. Since the neural network inevitably a multilayer structure, it is necessary to employ a new learning method which replaces the conventional backpropagation method. To satisfy this condition, this paper proposes an inverse function method which generates an intermediate target of learning in an inverse direction from a teacher signal at the last state, and a bidirectional learning method which is an improvement of the inverse function method. The bidirectional learning method improves the learning effect, since the learning process is more stable than the inverse function method, and intermediate targets are generated taking into account the state of the output of each step. This is demonstrated by using a two-dimensional pattern transform problem. [ABSTRACT FROM AUTHOR]

This paper discusses speaker-independent continuous speech recognition using the neural network with a fuzzy partition model (FPM) for phoneme discrimination. Higher-speed learning can be realized with FPM than with the time-delay neural network (TDNN). Using this high-speed property allows speaker-independent phoneme discrimination learning time was a serious drawback). This paper presents speaker-independent continuous speech recognition using the FPM-LR speech recognition system, wherein FPM is used for the phoneme discrimination training is executed using as speech samples 8 males and 8 females; the recognition performance is evaluated using 278 phrases. The following observations are derived as a result of experiment. A FPM requires less training time than TDNN. The performance can be improved by using the Multi-FPM-LR system, where processes for male, female and mixed case are combined. It is used to add the power and the delta-spectrum to the acoustic feature parameters. It is effective to use speeches of various utterances (word and phrase) in the training. An 80.0 percent recognition rate is achieved in the recognition of 278 phrases. Finally, the result for the sentence speech recognition is presented. [ABSTRACT FROM AUTHOR]

The neocognitron is a hierarchical neural network model with pattern recognition ability. The intermediate layers of the neocognitron contain various kinds of cells (feature-extracting cells) that extract partial features of the input patterns. The feature-extracting cells has modifiable input connections, with the connecting weights depending on the kind of feature to be extracted. The connecting weight is determined by an unsupervised learning. When a standard pattern is given as a stimulus, the connecting weights are adjusted so that the cell exhibits the maximum output. The tolerance in deformation of the feature to be extracted by the cell (that is, feature selectivity) can be adjusted by the thresholds of the cell. This paper discusses how the thresholds of the feature-extracting cell in the intermediate layers affect the recognition rate of the neocognitron. In the conventional neocognitron, the same threshold values were used in both learning and recognition phases. This paper shows also that the recognition rate can be improved greatly by using higher threshold values in the recognition phase than in the learning phase. [ABSTRACT FROM AUTHOR]

This paper proposes a multimodule neural network for complex pattern association. In the associative memory discussed in this paper, it is assumed that the memorized pattern can be divided into a certain number of attributes. An associative module is allocated to each of the divided attributes. The network is constructed by connecting those modules. Then, the relations among the attributes are memorized in the connections among the modules, as the mutual relations among the modules. By this structure, an association function is realized, in which given a partial information concerning some of the attributes, all information related to the input can be retrieved by tracing the relations among the attributes. Furthermore, by providing a module for each attribute, similar patterns that differ only in a very limited number of attributes, can be memorized and associated and separate patterns. By combining the modules of the associative memory so that multiple patterns can be produced sequentially as the output for an input pattern, a many-to-many association can be realized by the network composed of such modules, where multiple candidate patterns are recalled from various combinations of partial information. [ABSTRACT FROM AUTHOR]

Kohomen's model as the self-organizing model for the neural network can be considered as a kind of adaptive vector quantization algorithm. Numerous reports have been presented on the application of the model to practical problems. Although some results have been presented for the theoretical properties of Kohonen's model, many properties remain to be clarified. Among various properties of Kohonen's model as an adaptive vector quantization algorithm, this paper considers the problem of how the reference vectors are placed according to the probability distribution of the input signal. Considering the limit where the number of reference vectors is increased to infinity, this problem can be discussed theoretically as the distribution of the reference vectors. Due to the effect of the "learning by neighborhood," which is the feature of Kohonen's model, the property of the Kohonen model differs quantitatively from the property of the ordinary vector quantization algorithm. This paper discusses quantitatively the properties of Kohonen's model using the average learning equation. [ABSTRACT FROM AUTHOR]

Information processing by humans within themselves, e.g., such as reading letter characters, recognizing patterns, and other objects, and discriminating what they hear depends largely on the cerebral nerve. The human cerebrum learns various subjects and self-organizes the human neural network with plastic changes of synapses that arc formed through learning. Based on this viewpoint, many studies on neural network models and synaptic transfer efficiency have been reported from the engineering field. In this paper, the authors propose pulse-type hardware neuron models. These models employ pulse-type hardware cell body models which exhibit negative resistance characteristics and also chaos phenomena as reported in their earlier papers. The authors analyze the characteristics of the models and discuss an application of the models to a recognition circuit. Thai is, considering the subject of neurons, they propose pulse-type hardware models taking into account that information transmission is in a form of pulse repetition rate and that the synaptic weight which is equivalent to the transmission efficiency of synapses is subject to plastic changes. The pulse-type hardware neuron models employ pulse-type input/output circuits. The models are of a simple circuitry, consisting of a cell body section and a synaptic section. The cell body section employs V-shaped transistors. The synaptic section employs a CR time constant circuit to simulate the synaptic weight and a MOS variable capacitance device to simulate plastic changes of synapses. The authors propose two models. One type changes depending on learning signals which include functions of external signals such as teacher signal, as an example of the plastic change of synaptic weight The other model is of a type that changes depending on the input signal. The learning functions of the models are discussed by applying Hebb's rule as a typical learning rule. It is concluded that the pulse-type hardware neuron models provide methods for plastic synapse models with learning and memory functions, and also for synapse models with square characteristics. The neural network models made of these pulse-type hardware neuron models are effective for pattern recognition circuits, including vowel speech pattern recognition circuits. [ABSTRACT FROM AUTHOR]

This paper proposes a method of constructing a multilayered neural network, using the multiple logistic model (MLM). The model is a nonlinear multivariate analysis considering the output logistic function of each unit, which is used in the back- propagation method (BP). The idea can be applied directly to the determination of the multilayered neural network structure. The model can also be utilized as a systematic method to introduce such information as pattern distribution into the neural network structure. Considering the speaker-independent vowel recognition as the problem, this paper compares the results by the proposed method (MLM), the construction by the linear multiple regression analysis (MRA), the learning by BP with the weight being defined at random as the initial value, and the learning by BP with the initial weight determined by MLM or MRA. It is seen as a result that the recognition rate is the best when BP is applied after introducing the speaker distribution information by the proposed method. It is seen also that the computation time is reduced compared with the BP, with the initial weight being defined at random. [ABSTRACT FROM AUTHOR]

This paper proposes a neural network model, which performs deterministic, parallel pattern matching. The pattern is recognized by matching an input pattern containing noise and a template constructed through learning. In this paper, it is assumed that no feature detection has already been accomplished. We consider a circuit structure which merely compares a template with the noisy feature distribution pattern that appears at the output of the feature detector. This pattern matching method is based on our previous work using neural networks to determine the correspondence between stereo images. The input from the left eye, for example, corresponds to the feature distribution pattern, and the input from the right eye corresponds to the template. The difference in the positions of features in the template and the input pattern corresponds to the binocular parallax. The proposed model is composed of a pattern matching layer, a minimum distance detection layer (which selects the template close to the input feature distribution pattern, and feeds it back to the pattern matching layer for pattern matching) and the recognition layer which classifies the matched template. The results of our computer simulation indicated that perfect matching was achieved for inputs having feature distribution pat- terns corresponding to those used during learning. Since sequences of labels or character strings can be considered to be feature distribution patterns, the proposed network can be applied to a wide range of problems related to pattern matchings [ABSTRACT FROM AUTHOR]

This paper describes a stereo-matching algorithm which works effectively even for a scene with a large number of discontinuities such as in the case of multiple interlaced surfaces. Prazdny (61 pointed out that the discontinuity can be handled by dissolving the ambiguity based on the facilitation by the similarity, rather than employing the discontinuity of disparity as the penalty. However, his method has a problem in that the ability to dissolve the ambiguity is decreased in correspondence to the reduction of scene restriction. In this paper, to determine the dispartity field maximizing the facilitation of similarity, several matching rules and matching primitives are combined, and the result is integrated into an energy-minimum model of a network similar to Hopfield's neural network [7]. This resulted in a strong disambiguation power. Especially, we propose a new multiresolution processing method robust against discontinuity, in which processing of the different levels is in parallel. Only the information data concerning the image features corresponding to the same physical entity are propagated from the coarse to fine levels. By the reasonings in this paper, a natural extension is derived for the Marr-Poggio first stereo algorithm [2]. An experiment was made for a scene through a fence, and it was verified that the interactions built in the network work effectively. [ABSTRACT FROM AUTHOR]

This paper aims at a constructive description of which neural network model can represent the human problem-solving process and how the process is related to the various human memories by what context. From such a viewpoint, a neural network model is constructed for the representation of the problem-solving process. In this paper, a model for the addition process is constructed as the first step, using the associative memory model HASP (human associative processor) proposed by one of the present authors. The model is composed of the following components: the procedure memory, which stores and retrieves the procedures of the addition; semantics memory, which stores the knowledge about the addition such as 2 + 2 = 4; the working memory, which temporarily stores the external and internal information; and the goal memory, which stores the problem-solving process as a chain of subgoals, and manages the flow of procedures from one higher level in the control hierarchy. In this study, a computer simulation was performed for the addition processes for several numbers with multiple digits, and it is shown that the addition process can be represented. Then the computer simulation is made for the generation of four kinds of bugs in the addition. It is shown that the bug generation can be represented by the model. [ABSTRACT FROM AUTHOR]

Numerous researchers have recently become interested in the information processing in the brain. This is because the brain seems to process information in a mechanism which is completely different from that of the computer. The parallel information processing, learning and self-organizing ability, as well as the memory mechanism of the brain, present interesting principles of information processing which will be very helpful in the design of future computers. To arrive at a clear understanding of those principles, various attempts have been made to describe the underlying principles in a mathematical and systematic way, in parallel to the examination of the behavior of the neural network model. This paper outlines those efforts from the viewpoint of pattern dynamics of the neural field and the visual information processing, learning function in the input-output characteristics and the control of movement, self-organization of information recognition mechanism, the associative model, and its stochastic realization. However, this paper aims not only at a review of the present status, but also at pointing out the present problems such as the bottleneck, indicating the problems posed and the direction of the new possible researches. [ABSTRACT FROM AUTHOR]

In this paper, we present a segmentation-based method for offline Farsi handwritten word recognition. Although most segmentation-based systems suffer from segmentation errors within the first stages of recognition, using the inherent features of the Farsi writing script, we have segmented the words into sub-words. Instead of using a single complex classifier with many (N) output classes, we have created N simple recurrent neural network classifiers, each having only true/false outputs with the ability to recognize sub-words. Through the extraction of the number of sub-words in each word, and labeling the position of each sub-word (beginning/middle/ end), many of the sub-word classifiers can be pruned, and a few remaining sub-word classifiers can be evaluated during the sub-word recognition stage. The candidate subwords are then joined together and the closest word from the lexicon is chosen. The proposed method was evaluated using the Iranshahr database, which consists of 17,000 samples of Iranian handwritten city names. The results show the high recognition accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we propose a dual-phase approach to improve the process of heart disease prediction in a mobile environment. Firstly, only the confident frequent rules are extracted from a patient's clinical information. These are then used to foretell the possibility of the presence of heart disease. However, in some cases, subjects cannot describe exactly what has happened to them or they may have a silent disease -- in which case it won't be possible to detect any symptoms at this stage. To address these problems, data records collected over a long period of time of a patient's heart rate variability (HRV) are used to predict whether the patient is suffering from heart disease. By analyzing HRV patterns, doctors can determine whether a patient is suffering from heart disease. The task of collecting HRV patterns is done by an online artificial neural network, which as well as learning knew knowledge, is able to store and preserve all previously learned knowledge. An experiment is conducted to evaluate the performance of the proposed heart disease prediction process under different settings. The results show that the process's performance outperforms existing techniques such as that of the self-organizing map and gas neural growing in terms of classification and diagnostic accuracy, and network structure. [ABSTRACT FROM AUTHOR]

This paper presents a hybrid model solution for user motion recognition. The use of a single classifier in motion recognition models does not guarantee a high recognition rate. To enhance the motion recognition rate, a hybrid model consisting of decision trees and artificial neural networks is proposed. We define six user motions commonly performed in an indoor environment. To demonstrate the performance of the proposed model, we conduct a real field test with ten subjects (five males and five females). Experimental results show that the proposed model provides a more accurate recognition rate compared to that of other single classifiers. [ABSTRACT FROM AUTHOR]

This paper proposes a method which succinctly structures neural networks having a few thousand weights. Here structuring means weight sharing where weights in a network are divided into clusters and weights within a cluster have the same value. We newly introduce a weight sharing technique called bidirectional clustering of weights (BCW), together with second-order optimal criteria for both cluster merging and splitting. Our experiments using two artificial data sets showed that the BCW method works well to find a succinct network structure from a neural network having about 2000 weights in both regression and classification problems. © 2007 Wiley Periodicals, Inc. Syst Comp Jpn, 38(10): 46–57, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20535 [ABSTRACT FROM AUTHOR]

This paper presents a generation procedure for high-order threshold functions, considering the high-order neighbor vector and the monotonic property. Specifically, it is first shown that the high-order neighbor vector can be derived from an input vector which is noncomparable by the high-order threshold function. Then, it is shown that a monotonic function of a high-order variable can easily be generated from the given monotonic function. Based on the generation process for the monotonic function, the generation procedure for high-order threshold functions is derived. It is also shown that if a given high-order threshold function preserves monotonicity, the generated high-order threshold function also preserves monotonicity, and also the high-order terms. © 2007 Wiley Periodicals, Inc. Syst Comp Jpn, 38(9): 63–73, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20243 [ABSTRACT FROM AUTHOR]

A self-organizing map (SOM) performs a mapping of an object preserving its topological relations between input and output spaces, and also can be seen as a coordinate transformer that preserves adjacency relations. Since the standard SOM cannot deal with temporal data intrinsically, in this paper we provide new feedback pathways around the competitive layer to refer to context information of the past history. An extra output layer is added next to the competitive layer to represent secondary candidates and a quantitative measure of confidence. From the viewpoint of its structural similarities, we refer to this architecture as an Elman-type feedback SOM. In order to clarify the effectiveness of the proposed model, we then adopt a temporal signal processing task of Braille recognition. Braille is a character set for visually impaired people made up of 6 dots on a 3 by 2 grid. When Braille is read by running the fingertip over the characters, the vertical axis can be seen as providing spatial information and the horizontal axis as providing temporal information. As a result of computer simulations with partially common four city names, we confirm that the proposed model can recognize them appropriately. In addition, we find that this approach is robust to both the temporal elasticity and spatial displacement. Moreover, when we conduct an analysis with the neuro-bar model, we find that the Braille recognition task is executed based on a series of state transition along a type of pathway that has been developed during training. © 2007 Wiley Periodicals, Inc. Syst Comp Jpn, 38(3): 62– 71, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20260 [ABSTRACT FROM AUTHOR]

Freehand curve recognition method FSCI is already proposed and consists of the following procedure. The freehand drawing is represented as a fuzzy curve model by using a fuzziness generator, and fuzzy inference is used to find as simple a geometrical curve as possible, so that the drawing intention of the user is identified. It is recognized that the adequacy of the parameter setting in the fuzziness generator and the inference rules has a serious effect on the identification performance of FSCI. Thus, training optimization for the fuzziness generator using the genetic algorithm, and training optimization of inference rules using a neural network have been separately proposed. This paper proposes a new method which achieves simultaneous training optimization of the fuzziness generator and the inference rules using a neural network. The effectiveness of the method is demonstrated experimentally. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 38(1): 51– 61, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10387. [ABSTRACT FROM AUTHOR]

We have proposed the forward-propagation rule (FP) as an inverse model learning scheme from the viewpoint of biological motor control. This learning scheme is based on a Newton-like method, by which multilayered neural network can acquire an inverse model of the controlled object by a small number of iterative learning trials. There is a problem, however, that the learning procedure, which is characterized by estimation of the supervisor's signal for the input–output signal of the neuron, and also the solution of a linear multiple regression problem for updating the connection weights, is complicated, making it difficult to analyze the learning process. This paper introduces the correlation of the propagated error signal from the viewpoint of the maximum-likelihood method in order to realize a goal-directed learning, which has not hitherto been considered in FP, and extends the learning rule to the generalized least-square method. As a result, it is clearly shown that the learning rule in FP is an approximate gradient method. The learning ability of the method is demonstrated by computer simulation. The proposed procedure contains a regularization term derived from the logarithmic likelihood, and the behavior after the convergence of learning exhibits a more stable tendency than in the conventional method. It is also shown that learning can be performed by a simplified method in which the error is simply propagated in the forward direction. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(13): 54–66, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20484 [ABSTRACT FROM AUTHOR]

This research paper proposes an omnidirectional pointing device that uses EMG signals. Since this device does not keep the user in front of the mouse or keyboard, there are expectations for its use as an input device for a wearable computer, and other equipment. A Recurrent Log-Linearized Gaussian Mixture Network is introduced in order to make use of the time series characteristics of EMG signals. This neural net is configured based on the hidden Markov model, which is a dynamic probability model, and infinitely many movement directions of a pointer can be expressed as combinations of probabilities of movement in preset reference directions. As a result of a verification experiment, it was confirmed that the pointer could be controlled with good precision. The effectiveness of the proposed method was confirmed by comparative experimentation with conventional methods. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(4): 55–63, 2006; Published online in Wiley InterScience (www.interscience.wiley. com). DOI 10.1002/scj.20401 [ABSTRACT FROM AUTHOR]

This paper proposes a neural network model which receives visual inputs from a robot moving freely in a room, and extracts its position and direction information separately. The model has three-dimensional structure in which two-dimensional neural fields (unit planes) are arranged in a toroidal form. The learning algorithm takes a form of neural gas in each unit plane and self-organizing map (SOM) in the toroidal direction. It is shown by computer simulation that position and view direction of the robot can be extracted from visual inputs, and the effectiveness of the learning algorithm is verified. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(11): 1–11, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20349 [ABSTRACT FROM AUTHOR]

Self-Generating Neural Networks (SGNN) have the feature of fast processing by automatically constructing a self-generating neural tree (SGNT) from a given training data set. The prediction accuracy of SGNNs for chaotic time series prediction is improved by ensemble averaging of various SGNNs. However, the computation time increases with the number of SGNNs on a single processor. In this paper, we investigate improvement of the prediction accuracy and parallel efficiency of ensemble SGNNs for chaotic time series prediction problems on an MIMD parallel computer. We allocate each SGNN to one processor. Our results show that as the number of node processors increases, a continuing improvement of prediction accuracy is obtained for all problems, while maintaining the high-speed processing performance of the single SGNN. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(10): 82–92, 2005; Published online in Wiley InterScience (www.interscience. wiley.com). DOI 10.1002/scj.10297 [ABSTRACT FROM AUTHOR]

In a layered neural network, the error backpropagation method is generally used as the supervised learning procedure for the hidden layer, which cannot be observed from the outside. In order to apply the method, however, the output function of the neuron model must be differentiable. This paper proposes a supervised learning method for the hidden layer neurons in which the teaching signal is calculated for the hidden layer neurons, by utilizing the learning rule for the connection weight and the duality in the output layer neuron. The method is applicable so long as the neuron model contains duality, and it does not require that the output layer neurons or the hidden layer neurons be differentiable. As an example of a case in which the error backpropagation cannot be applied, a perceptron composed of neurons with a step output function is considered. The proposed method is applied, and the learning rule for the whole network is constructed. The XOR problem was actually learned by the network, and the same learning success rate was obtained as in the error backpropagation method for a perceptron composed of neurons with a sigmoid output function. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(9): 34–42, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20206 [ABSTRACT FROM AUTHOR]

In order to achieve fast parallel processing operation by a neural network, it is desirable to implement all neurons in parallel in hardware rather than to realize operation by serial operation in software. The authors have proposed a pulsed neural network based on ΔΣ modulation which is suited to the hardware implementation of digital circuits. The proposed neural network can be realized on a small circuit scale, since the variable can be represented by a ΔΣ-modulated 1-bit pulse signal. Highly accurate operation can be realized even though only 1 bit is used. This paper proposes a hardware implementation of the above neural network. A neural network with built-in GHA learning rule is hardware implemented on an FPGA, and its operation is investigated. The proposed neural network and CPU are implemented on an FPGA and are compared in terms of the number of bits that can be processed in unit time per unit of circuit scale. The proposed scheme achieved an evaluation score 200 times as great as a software realization on a CPU. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(9): 14–24, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20235 [ABSTRACT FROM AUTHOR]

In this paper the authors propose a method for high-accuracy recognition of large character sets using a new combination of a statistical method and neural networks. In their method, a hierarchical structure that has several neural networks arranged in a line after the statistical method is used. First, recognition using a statistical method is performed, and this represents the final result if the top candidate does not belong to a predefined set of similar characters. If it does, then the input character is discriminated in a neural network which designates the top candidate by determining the similar characters. The results are output as final results. The basic idea of this method is the functional division of a statistical method and neural networks, and the use of a neural network as determined by a statistical method. The results of recognizing 3201 character types including JIS-1 Kanji showed an improvement in the correct recognition rate due to the combined use of a statistical method and neural networks, thereby demonstrating the validity of the authors' approach. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(9): 97–107, 2005; Published online in Wiley InterScience (www.interscience. wiley.com). DOI 10.1002/scj.20330 [ABSTRACT FROM AUTHOR]

A forward-propagation learning scheme has been proposed to acquire inverse models of controlled objects in multilayered neural networks. This scheme is quite different from back-propagation learning rule. The algorithm of the forward-propagation rule consists of two stages. One is the estimation of the instruction signal at each layer by the Newton-like method, and the other is the updating of the connection weights by linear multiple regression. In this scheme, convergence of learning has been faster than other schemes based on backpropagation rule. However, the problems arise that complex parameters must be set for learning and the learning process is too complex and sometimes stops. This paper proposes to use a method of recursive least squares in the forward-propagation rule. The effectiveness of the proposed method is confirmed by computer simulation for the learning of the inverse dynamics model for a two-link arm. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(8): 71–80, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20237 [ABSTRACT FROM AUTHOR]

In neural networks based on a distributed information representation, when different patterns are to be recalled from the same input by association depending on the context, the usual method is to concatenate the pattern representing the context with the input pattern. However, there is a serious problem in this approach, and strong constraints are imposed on the number of inputs and the number of context patterns. This paper tries to apply a different method of contextual modification to nonmonotonic neural networks in order to construct a context-dependent associative model that can solve the longstanding problem. In the proposed model, the number of associations that can be learned increases almost in proportion to the number of elements, regardless of the numbers of the input and context patterns. In addition, the state transitions among attractors can be controlled flexibly by switching the context, which enables the model to simulate the behavior of any finite automaton without inducing an explosive increase in the number of elements or the training time. The model also has a high generalization power ability based on fully distributed representations and has potential for overcoming the limitations of conventional symbol processing. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(7): 73–83, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10477 [ABSTRACT FROM AUTHOR]

With the development of our highly information-oriented society, there is an increasing demand for large-scale and high-level information processing. Toward this goal, studies have sought to create a new computation principle having an information processing ability exceeding the existing Neumann-type computer, such as the creation of a new computation theory or the integration of the frameworks of existing computation theories. As one such approach, quantum neural computing is considered to be interesting, which integrates neural computing and quantum computation. This paper constructs the feed-forward neural network, which is widely used in practice, based on the qubit neuron model. The 4-bit parity-check problem and the general function identification problem are considered. The performance is compared to the feed-forward network based on the conventional neuron model, and it is shown that the proposed model has a higher performance than the conventional model, using the learning diagram composed of convergence rate and the number of learning iterations. The reason for the better performance is also discussed. © 2004 Wiley Periodicals, Inc. Syst Comp Jpn, 35(13): 43–51, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10342 [ABSTRACT FROM AUTHOR]

It is known that the Hopfield-type neural network can solve the combinatorial optimization problem with a high speed. In order to realize that advantage, however, the energy function must be constructed for each problem, and there is no general framework to be applied to various problems. This paper notes that the logic programming language Prolog can be represented as the logical operation of facts and the rules, and proposes an algorithm, which converts the logical operation network to the Hopfield-type neural network. Using this conversion algorithm, the power of the Hopfield-type network as an optimization machine can be utilized in the logic inference described by Prolog. Then, it is expected that the range of its application will be enlarged greatly. © 2004 Wiley Periodicals, Inc. Syst Comp Jpn, 35(13): 63–70, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10102 [ABSTRACT FROM AUTHOR]

There are still some statistical properties which have not been clarified in the regression model based on the three-layered neural network. This paper presents an analysis of these problems, in terms of the probability distribution of the parameter estimators. It is first shown numerically that the least-square estimator for the condition in which the probability distribution for the parameter estimator has not been clearly described follows a distribution which is different from the probability distribution derived in the past for various conditions. Based on the result, a theoretical analysis is presented for the simplified regression model, and it is shown that the least-square parameter estimator follows the double-exponential distribution. © 2004 Wiley Periodicals, Inc. Syst Comp Jpn, 35(12): 1–9, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10580 [ABSTRACT FROM AUTHOR]