Search

This study investigates the design of a novel real-time system to detect walking behavior changes using an accelerometer on a rollator. No sensor is required on the user. We propose a new non-invasive approach to detect walking behavior based on the motion transfer by the user on the walker. Our method has two main steps; the first is to extract a gait feature vector by analyzing the three-axis accelerometer data in terms of magnitude, gait cycle and frequency. The second is to classify gait with the use of a decision tree of multilayer perceptrons. To assess the performance of our technique, we evaluated different sampling window lengths of 1, 3 an 5 seconds and four different Neural Network architectures. The results revealed that the algorithm can distinguish walking behavior such as normal, slow and fast with an accuracy of about 86%. This research study is part of a project aiming at providing a simple and non-invasive walking behavior detector for elderly who use rollators.

Nowadays, there are many different types of mobility aids for elderly people. Nevertheless, these devices may lead to accidents, depending on the terrain where they are being used. In this context, the goal of the EyeWalker project is to develop a ultralight computer vision device for users with mobility problems. One of the main objective of this work is to develop a ground change detection module that will warn the user before entering dangerous terrains or hostile situations. This software component integrated on a ”smart” walker will be able to react in real time, to operate both indoor and outdoor, as well as in familiar or unfamiliar environments. Specifically, we propose a classification algorithm using colour and texture as a descriptor to detect ground changes. In our classifier, the distributions of HSV colours and Local Edge Patterns are used to compare the similarity between the current frame and the average of the k previous frames. To compare similarities, we used four different techniques (Histogram Intersection, Kolmogorov-Smirnov, Cumulative Integral and Artificial Neural Networks) with outdoor training images. Preliminary results reveal that artificial neural networks achieved the best performances.

See ColOr is a mobility aid for visually impaired people that uses the auditory channel to represent portions of captured images in real time. A distinctive feature of the See ColOr interface is the simultaneous coding of colour and depth. Four main modules were developed in order to replicate a number of mechanisms present in the human visual systems. In this work we first present the main experiments carried out in the first years of the project; among them : the avoidance of obstacles the recognition and localization of objects the detection of edges and the identification of coloured targets. Finally we introduce new undergoing experiments in Colombia with blind persons whose purpose is (1) to determine and to touch a target; (2) to navigate and to find a person; and (3) to find particular objects. Preliminary results illustrate encouraging results.

In this work we describe a computer aid system designed to interact with a user via auditory feedback so as to assist the exploration of the nearby environment. We made use of a Microsoft‘s Kinect sensor as input peripheral and of an iPad that hosts the multitouch end user interface which deliver coded data into sound. A computer vision based strategy has been implemented to recognize objects present in a real time scene. Afterwards these objects are associated to sounds of musical instruments and sonified through virtual space sources that create the illusion of sounds originating from their location. Users can hear explore and understand the scenery composition while finger triggering the sounds within the iPad surface on which a top view image of the scene has been mapped. Preliminary experiments using geometrical objects randomly placed on a table reveal that our system can build very accurately a scene in someone‘s mind in a satisfactory time despite the absence of vision.

The See Color interface transforms a small portion of a colored video image into sound sources represented by spatialized musical instruments. Basically the conversion of colors into sounds is achieved by quantization of the HSL color system. Our purpose is to provide visually impaired individuals with a capability of perception of the environment in real time. In this work the novelty is the simultaneous sonification of color and depth depth being coded by sound rhythm. Our sonification model is illustrated by several experiments such as: (1) detecting an open door in order to go out from the office; (2) walking in a hallway and looking for a blue cabinet; (3) walking in a hallway and looking for a red tee shirt; (4) moving outside and avoiding a parked car. Videos with sounds of experiments are available on http://www.youtube.com/guidobologna.

The See ColOr interface transforms a small portion of a colored video image into sound sources represented by spatialized musical instruments. This interface aims at providing visually impaired people with a capability of perception of the environment. As a first step of this on-going project, the purpose is to verify the hypothesis that it is possible to use sounds from musical instruments to replace color. Compared to state of the art devices, a quality of the See ColOr interface is that it allows the user to receive a feed-back auditory signal from the environment and its colors, promptly. Two experiments based on a head mounted camera have beenperformed. The first experiment pertaining to object manipulation is based on the pairing of colored socks, while the second experiment is related to outdoor navigation with the goal of following a colored serpentine. The “socks” experiment demonstrated that seven blindfolded individuals were able to accurately match pairs of colored socks. The same participants successfully followed a red serpentine for more than 80 meters.

The context of this work is the development of a mobility aid for visually impaired persons. We present here an original approach for a real time alerting system, based on the detection of visual salient parts in videos. The particularity of our approach lies in the use of a new feature map constructed from the depth gradient. A distance function is described, which takes into account both stereoscopic camera limitations and user's choices. We also report how we automatically estimate the contribution of conspicuity maps, which enables the unsupervised determination of the final saliency map. We demonstrate here that this additional depth-based feature map allows the system to detect salient regions with good accuracy in most situations, even in the presence of noisy disparity maps.

The context of this work is the development of a mobility aid for visually impaired persons. We present here an original approach for a real time alerting system, based on the detection of visual salient parts in videos. The particularity of our approach lies in the use of a new feature map constructed from the depth gradient. A distance function is described, which takes into account both stereoscopic camera limitations and user's choices. We also report how we automatically estimate the contribution of conspicuity maps, which enables the unsupervised determination of the final saliency map. We demonstrate here that this additional depth-based feature map allows the system to detect salient regions with good accuracy in most situations, even in the presence of noisy disparity maps.

This last decade multi-layer perceptrons (MLPs) have been widely used in classification tasks. Nevertheless, the difficulty of explaining their results represents a main drawback for their acceptance in critical domain applications such as medical diagnosis. In this context how can we trust a black box without any form of explanation capability ? To redress this situation, the internal representation of a multi-layer perceptron should be transformed into symbolic rules. Such a network is a neural expert system. In the field of symbolic rule extraction from neural networks Andrews et al. proposed a taxonomy to explain and compare the characteristics of the existing techniques. After having studied what we consider the main contribution of the domain we propose the new approach of extracting symbolic rules by precisely locating the discriminant frontiers between two classes. Basically, in our mathematical analysis we point out that a frontier is built according to an equation with one linear term and one logarithmic term. When the logarithmic term is constant the frontier is a hyper-plane. However, as the combination of hyper-planes gives polyhedrons, the symbolic rule representation corresponding to hyper-rectangles is not matched. So, the idea is to introduce an MLP architecture which builds axis-parallel hyper-planes. The Interpretable Multi Layer Perceptron is a special multi layer perceptron architecture which splits the input space into hyper-rectangles. In this model the key idea is to use threshold activation functions in the first hidden layer. Rule extraction is carried out by solving a Boolean minimization problem. In practice rules with 100% fidelity are extracted in polynomial time. To our knowledge no other rule extraction technique reaches such a performance in every classification problem. In addition, input variables do not need to be quantized and rules can also be inserted to perform rule refinement. Finally, in spite of the reduced "power of expression" with respect to the standard multi layer perceptron, IMLP is also an universal approximator. The key ideas introduced in IMLP have been applied to other architectures denoted as OMLP, HOOMLP, DIMLP, and MTB. Briefly, OMLP (Oblique Multi Layer Perceptron) is a model from which we extract rules having linear combinations of antecedents. At the level of its internal representation it is the most similar to the standard multi-layer perceptron. By creating the HOOMLP (High Order OMLP) model we introduce the notion of paraboloidal ellipsoidal, spherical rule extraction. That is the creation of rules splitting the input space into hyperparaboloids, hyper-ellipsis, and hyper-spheres. DIMLP (Discretized IMLP) is a generalization of IMLP with a more compact internal representation than IMLP. Finally, MTB (Modular Transparent Boxes) is a model in which several interpretable sub-models are combined together. The remarkable characteristic of modular transparent boxes resides in the fact that symbolic rules are not uniquely extracted at the level of each single model, but also at the level of the global combination. IMLP has been tested on 1L applications of the public domain and two real world special applications. The conclusion is that from a predictive accuracy point of view, in half of the classification problems IMLP performs better than MLP and C4.5 (one of the main references in rule extraction from datasets). Finally, concerning symbolic rules, IMLP has the tendency to generate less comprehensible rules than C4.5. However, applying an approximate covering in the boolean minimization step of the IMLP rule extraction algorithm has given more understandable rules at the price of slightly worse accuracy.

Making diagnosis by learning from examples is a typical field of artificial neural networks. However, justifications of network responses are difficult to obtain, especially when input examples have analog variables. We propose a particular multi-layer Perceptron model in which explanations of responses are obtained through symbolic rules. The originality of this model consists in its architecture. Experiments using three datasets related to breast cancer diagnosis, coronary heart disease and thyroid dysfunctions have shown high mean predictive accuracy (respectively: 96.3%, 90.0%, 99.3%). Comparisons with the C4.5 algorithm, which builds inductive decision trees, have shown that the predictive accuracy of both approaches is roughly the same, with neural networks slightly more accurate.

The See ColOr project aims at developing a mobility system for blind persons based on image color sonification. Within this project the present work addresses the optimal use of auditory multi touch interaction and in particular the matter of the number of fingers needed for efficient exploration. To determine the actual significance of mono and multi touch interaction onto the auditory feedback a color matching memory game was implemented. Sounds of this game were generated by touching a tablet with one or two fingers. A group of 20 blindfolded users was tasked to find color matches into an image grid represented on the tablet by listening to their associated color sound representation. Our results show that for an easy task aiming at matching few objects the use of two fingers is moderately more efficient than the use of one finger. Whereas against our intuition this cannot be statistically confirmed in the case of similar tasks of increasing difficulty.