Search

Context: Several text books and papers published between 2000 and 2002 have attempted to introduce experimental design and statistical methods to software engineers undertaking empirical studies. Objective: This paper investigates whether there has been an increase in the quality of human-centric experimental and quasi-experimental journal papers over the time period 1993 to 2010. Method: Seventy experimental and quasi-experimental papers published in four general software engineering journals in the years 1992-2002 and 2006-2010 were each assessed for quality by three empirical software engineering researchers using two quality assessment methods (a questionnaire-based method and a subjective overall assessment). Regression analysis was used to assess the relationship between paper quality and the year of publication, publication date group (before 2003 and after 2005), source journal, average coauthor experience, citation of statistical text books and papers, and paper length. The results were validated both by removing papers for which the quality score appeared unreliable and using an alternative quality measure. Results: Paper quality was significantly associated with year, citing general statistical texts, and paper length (p < 0.05). Paper length did not reach significance when quality was measured using an overall subjective assessment. Conclusions: The quality of experimental and quasi-experimental software engineering papers appears to have improved gradually since 1993. [ABSTRACT FROM PUBLISHER]

This paper presents a systematic literature review of learning analytics dashboards (LADs) research that reports empirical findings to assess the impact on learning and teaching. Several previous literature reviews identified self-regulated learning as a primary focus of LADs. However, there has been much less understanding how learning analytics are grounded in the literature on self-regulated learning and how self-regulated learning is supported. To address this limitation, this review analyzed the existing empirical studies on LADs based on the well-known model of self-regulated learning proposed by Winne and Hadwin. The results show that existing LADs are rarely grounded in learning theory, cannot be suggested to support metacognition, do not offer any information about effective learning tactics and strategies, and have significant limitations in how their evaluation is conducted and reported. Based on the findings of the study and through the synthesis of the literature, the paper proposes that future research and development should not make any a priori design decisions about representation of data and analytic results in learning analytics systems such as LADs. To formalize this proposal, the paper defines the model for user-centered learning analytics systems (MULAS). The MULAS consists of the four dimensions that are cyclically and recursively interconnected including: theory, design, feedback, and evaluation. [ABSTRACT FROM AUTHOR]

Genetic improvement (GI) uses automated search to find improved versions of existing software. We present a comprehensive survey of this nascent field of research with a focus on the core papers in the area published between 1995 and 2015. We identified core publications including empirical studies, 96% of which use evolutionary algorithms (genetic programming in particular). Although we can trace the foundations of GI back to the origins of computer science itself, our analysis reveals a significant upsurge in activity since 2012. GI has resulted in dramatic performance improvements for a diverse set of properties such as execution time, energy and memory consumption, as well as results for fixing and extending existing system functionality. Moreover, we present examples of research work that lies on the boundary between GI and other areas, such as program transformation, approximate computing, and software repair, with the intention of encouraging further exchange of ideas between researchers in these fields. [ABSTRACT FROM AUTHOR]

So-called spatial web queries retrieve web content representing points of interest, such that the points of interest have descriptions that are relevant to query keywords and are located close to a query location. Two broad categories of such queries exist. The first encompasses queries that retrieve single spatial web objects that each satisfy the query arguments. Most proposals belong to this category. The second category, to which this paper's proposal belongs, encompasses queries that support exploratory user behavior and retrieve sets of objects that represent regions of space that may be of interest to the user. Specifically, the paper proposes a new type of query, the top- $k$ k spatial textual cluster retrieval ($k$ k -STC) query that returns the top- $k$ k clusters that (i) are located close to a query location, (ii) contain objects that are relevant with regard to given query keywords, and (iii) have an object density that exceeds a given threshold. To compute this query, we propose a DBSCAN-based approach and an OPTICS-based approach that rely on on-line density-based clustering and that exploit early stop conditions. Empirical studies on real data sets offer evidence that the paper's proposals can find good quality clusters and are capable of excellent performance. [ABSTRACT FROM AUTHOR]

Production Lead Time \mbi (LT) is the average time a part spends in the system, being processed or waiting for processing. In systems with unlimited buffers, \mbi LT may be orders of magnitude larger than the total processing time, leading to serious economic and quality problems. At present, no systematic analytical methods for evaluation, analysis, and control of \mbi LT in systems with machines having up- and downtime characterized by continuous random variables are available. This paper is intended to develop such methods. Specifically, we address synchronous serial lines with exponential machines and derive formulas for \mbi LT as a function of machine parameters and raw material release rate. Using these formulas, we develop methods for open- and closed-loop raw material release, which result in the desired \mbi LT. For asynchronous exponential lines, we provide an upper bound on LT. For non-exponential lines (e.g., Weibull, gamma, and log-normal), we offer an empirical formula for \mbi LT as an affine function of the coefficient of variation. The results reported in this paper enable a new paradigm for production systems management, namely: manage a production system so that the desired \mbi LT is ensured, while the throughput is maximized. [ABSTRACT FROM AUTHOR]

We consider the fundamental problem of matching a template to a signal. We do so by M-estimation, which encompasses procedures that are robust to gross errors (i.e., outliers). Using standard results from empirical process theory, we derive the convergence rate and the asymptotic distribution of the M-estimator under relatively mild assumptions. We also discuss the optimality of the estimator, both in finite samples in the minimax sense and in the large-sample limit in terms of local minimaxity and relative efficiency. Although most of the paper is dedicated to the study of the basic shift model in the context of a random design, we consider many extensions towards the end of the paper, including more flexible templates, fixed designs, the agnostic setting, and more. [ABSTRACT FROM AUTHOR]

This paper considers the problem of sequential empirical coordination, where the objective is to achieve a given value of the expected uniform deviation between the state-action empirical averages and the statistical expectations under a given strategic probability measure with respect to a given universal Glivenko–Cantelli class of test functions. A communication constraint is imposed on the Shannon entropy of the resulting action sequence. It is shown that the fundamental limit on the output entropy is given by the minimum of the mutual information between the state and the action processes under all strategic measures that have the same marginal state process as the target measure and approximate the target measure to desired accuracy with respect to the underlying Glivenko–Cantelli seminorm. The fundamental limit is shown to be asymptotically achievable by the tree-structured codes. [ABSTRACT FROM AUTHOR]

Human action recognition in video is often approached by means of sequential probabilistic models as they offer a natural match to the temporal dimension of the actions. However, effective estimation of the models' parameters is critical if one wants to achieve significant recognition accuracy. Parameter estimation is typically performed over a set of training data by maximizing objective functions such as the data likelihood or the conditional likelihood. However, such functions are nonconvex in nature and subject to local maxima. This problem is major since any solution algorithm (expectation-maximization, gradient ascent, variational methods and others) requires an arbitrary initialization and can only find a corresponding local maximum. Exhaustive search is otherwise impossible since the number of local maxima is unknown. While no theoretical solutions are available for this problem, the only practicable mollification is to repeat training with different initializations until satisfactory cross-validation accuracy is attained. Such a process is overall empirical and highly time-consuming. In this paper, we propose two methods for one-off initialization of hidden Markov models achieving interesting tradeoffs between accuracy and training time. Experiments over three challenging human action video datasets (Weizmann, MuHAVi and Hollywood Human Actions) and with various feature sets measured from the frames (STIP descriptors, projection histograms, notable contour points) prove that the proposed one-off initializations are capable of achieving accuracy above the average of repeated random initializations and comparable to the best. In addition, the methods proposed are not restricted solely to human action recognition as they suit time series classification as a general problem. [ABSTRACT FROM AUTHOR]

The emergence of Bluetooth low energy (BLE) beacons has promoted the development of proximity-based service (PBS), which is a context-aware application delivered subject to the Proximity of Interest (PoI). Most commercial applications use the sequential proximity detection with a fixed scanning mechanism to identify the target PoI. Such sequential execution, though is able to produce reliable detection, suffers severe performance degradation especially when the number of deployed beacons in the vicinity increases. To understand the effects of dense deployment, we conduct an empirical investigation and derive the statistical properties of both received signal strength (RSS) and signal inter-arrival time. In light of the statistical insights, this paper proposes a high resolution proximity detection using an adaptive scanning mechanism fusion with a spontaneous Differential Evolution (AS+sDE ). This novel approach enables the receiver to adapt its scanning duration conditioned on the deployment density and make an almost spontaneous detection in parallel with the scanning. The feasibility of the proposed approach is verified by both simulations and real-world implementations. For a density of $\leq 5\ beacons/m^2$ , AS+sDE achieves a superior performance with a high accuracy rate, i.e., on average $<1s$ is spent to guarantee at least 90 percent accuracy. [ABSTRACT FROM AUTHOR]

In a decentralized and self-configuring network, the communication devices are considered as autonomous decision makers that sense their environment and that implement optimal transmission schemes. It is essential that these autonomous devices cooperate and coordinate their actions, to ensure the reliability of the transmissions and the stability of the network. We study a point-to-point scenario in which the encoder and the decoder implement decentralized policies that are coordinated. The coordination is measured in terms of empirical frequency of symbols of source and channel. The encoder and the decoder perform a coding scheme, such that the empirical distribution of the symbols is close to a target joint probability distribution. We characterize the set of achievable target probability distributions for a point-to-point source-channel model, in which the encoder is non-causal and the decoder is strictly causal, i.e., it returns an action based on the observation of the past channel outputs. The objectives of the encoder and of the decoder are captured by some utility function, evaluated with respect to the set of achievable target probability distributions. In this paper, we investigate the maximization problem of a utility function that is common to both encoder and decoder. We show that the compression and the transmission of information are particular cases of the empirical coordination. [ABSTRACT FROM AUTHOR]

Remote laboratories give students the opportunity of experimenting in STEM by using the Internet to control and measure an experimental setting. Remote laboratories are increasingly used in the classroom to complement, or substitute for, hands-on laboratories, so it is important to know its learning value. While many authors approach this question through qualitative analyses, this paper reports a replicated quantitative study that evaluates the teaching performance of one of these resources, the virtual instrument systems in reality (VISIR) remote laboratory. VISIR, described here, is the most popular remote laboratory for basic analog electronics. This paper hypothesizes that use of a remote laboratory has a positive effect on students’ learning process. This report analyzes the effect of the use of VISIR in five different groups of students from two different academic years (2013–2014 and 2014–2015), with three teachers and at two educational levels. The empirical experience focuses on Ohm’s Law. The results obtained are reported using a pretest and post-test design. The tests were carefully designed and analyzed, and their reliability and validity were assessed. The analysis of knowledge test question results shows that the post-test scores are higher that the pretest. The difference is significant according to Wilcoxon test ( p <0.001 ), and produces a Cohen effect size of 1.0. The VISIR remote laboratory’s positive effect on students’ learning processes indicates that remote laboratories can produce a positive effect in students’ learning if an appropriate activity is used. [ABSTRACT FROM PUBLISHER]

Large-scale multiple-input–multiple-output (MIMO) networks, together with relay technology, have received considerable attention due to their remarkable capacity potential. Nevertheless, the majority of recent research studies on the asymptotic capacity of large-scale MIMO amplify-and-forward (AF) relay networks are limited to a fundamental assumption of small-scale fading, where the effect of large-scale fading, including shadow fading and path loss, is ignored. Motivated by this observation, in this paper, we present an extended model for the asymptotic capacity analysis in a composite fading environment, where Rayleigh fading, shadow fading, and path loss effect are incorporated altogether. Moreover, a realistic relay distribution is considered for real-world application of our capacity analysis. The proposed model consists of one source and one destination that are both equipped with $M$ antennas, which communicate through $K$ relays, each having $N$ antennas. We first employ random matrix theory to derive the empirical cumulative distribution of the channel covariance matrix and analyze the asymptotic capacity per source–destination antenna pair as both $M$ and $NK$ tend to infinity, but their ratio remains as a constant. We then consider the asymptotic capacity under two asymptotic cases for a finite and an infinite number of relays, respectively. It is shown through both theoretical analysis and Monte Carlo simulations that increasing the shadowing standard deviation can improve asymptotic capacity, whereas increasing the path loss exponent would lead to a capacity reduction. Our simulation results also reveal the effects of the relay distribution, as well as the source and relay transmission powers, on the asymptotic capacity. [ABSTRACT FROM PUBLISHER]

A very important function of an issue tracking system is to assign labels to issue reports, such as bug, feature, enhancement, etc., in order to categorize issues to facilitate various development activities. In practice, it is very common that an issue has multiple labels. However, current works are mainly based on single-label prediction, which are not suitable for just-in-time multi-labeling services, due to the low efficiency. Therefore, in this paper, we propose MULA, a just-in-time MUlti-LAbeling system, which learns and automatically assigns multiple labels to issue reports. We have built a dataset with 81,601 entries and 11 labels, as the first benchmark for this task, and implemented a GitHub app. To the best of our knowledge, this is the first work and tool for online multi-labeling GitHub issues based on their categories. We conduct a comprehensive empirical study, including comparisons with five commonly adopted labeling models that show the superiority of MULA, as well as an evaluation that shows high consistency between MULA’s suggestions and developers’ opinions. [ABSTRACT FROM AUTHOR]

Representation-based classification (RC) methods such as sparse RC (SRC) have attracted great interest in pattern recognition recently. Despite their empirical success, few theoretical results are reported to justify their effectiveness. In this paper, we establish the theoretical guarantees for a general unified framework termed as atomic representation-based classification (ARC), which includes most RC methods as special cases. We introduce a new condition called atomic classification condition (ACC), which reveals important geometric insights for the theory of ARC. We show that under such condition ARC is provably effective in correctly recognizing any new test sample, even corrupted with noise. Our theoretical analysis significantly broadens the range of conditions under which RC methods succeed for classification in the following two aspects: (1) prior theoretical advances of RC are mainly concerned with the single SRC method while our theory can apply to the general unified ARC framework, including SRC and many other RC methods; and (2) previous works are confined to the analysis of noiseless test data while we provide theoretical guarantees for ARC using both noiseless and noisy test data. Numerical results are provided to validate and complement our theoretical analysis of ARC and its important special cases for both noiseless and noisy test data. [ABSTRACT FROM AUTHOR]

In this paper, we present the experimental results of a new spray paint algorithm presented in previous publications. Both theory and simulations indicate that the proposed method allows a robotic manipulator to paint a given surface using substantially lower joint torques than with conventional approaches. In this paper, we confirm this by implementing the algorithm on an ABB robot and we find that the joint torques needed to follow the trajectory are substantially lower than for the conventional approach. [ABSTRACT FROM AUTHOR]

The article analyzes previous studies on the results of Capability Maturity Model (CMM) program investments. One of these works is from Brad Clark, who investigated the relationship between the developer's CMM level and software development efficiency for 112 projects. Another by Donald Harter, Mayuram Krishnan and Sandra Slaughter analyzed 30 large-scale projects at a major information technology firm. The seven common performance metrics are also discussed including error density, software development productivity, percentage of rework, and schedule fidelity.

Goals: This paper discusses the need for a predictable method to evaluate gains and gaps of collaborative technology-mediated workflows and introduces an evaluation framework to address this need. Methods: The Collaborative Space – Analysis Framework (CS-AF), introduced in this research, is a cross-disciplinary evaluation method designed to evaluate technology-mediated collaborative workflows. The 5-step CS-AF meta-process includes: (1) current-state workflow definition, (2) current-state (baseline) workflow assessment, (3) technology-mediated workflow development and deployment, (4) technology-mediated workflow assessment, (5) analysis and conclusions. For this research, a comprehensive, empirical study of hypertension exam workflow for telehealth was conducted using the CS-AF approach. Results: The CS-AF systemized approach reveals critical cross-disciplinary evaluation data concerning gains and gaps of collaborative workflows when technology-mediated enhancements are characterized and compared with a baseline workflow for the goal of continuous workflow improvement. Conclusion: The CS-AF is an effective meta-analysis process that can be adapted for use in multiple domains. [ABSTRACT FROM AUTHOR]

Most existing metrics that we call white-box metrics, such as coverage metrics, require white-box information, like program structure information, and historical runtime information, to evaluate the fault detection capability of a test set. In practice, such white-box information is usually unavailable or difficult to obtain, which means they often cannot be used. In this paper, we propose a black-box metric, distance entropy, based on the diversification idea behind many published diversity-based techniques. Distance entropy provides a possible solution for test set evaluation when white-box information is not available. The empirical study illustrates that distance entropy can effectively evaluate test sets if the distance metric between tests is well defined. Meanwhile, distance entropy outperforms simple diversity metrics without increasing time complexity. [ABSTRACT FROM PUBLISHER]

Metaheuristic search techniques have been extensively used to automate the process of generating test cases, and thus providing solutions for a more cost-effective testing process. This approach to test automation, often coined “Search-based Software Testing” (SBST), has been used for a wide variety of test case generation purposes. Since SBST techniques are heuristic by nature, they must be empirically investigated in terms of how costly and effective they are at reaching their test objectives and whether they scale up to realistic development artifacts. However, approaches to empirically study SBST techniques have shown wide variation in the literature. This paper presents the results of a systematic, comprehensive review that aims at characterizing how empirical studies have been designed to investigate SBST cost-effectiveness and what empirical evidence is available in the literature regarding SBST cost-effectiveness and scalability. We also provide a framework that drives the data collection process of this systematic review and can be the starting point of guidelines on how SBST techniques can be empirically assessed. The intent is to aid future researchers doing empirical studies in SBST by providing an unbiased view of the body of empirical evidence and by guiding them in performing well-designed and executed empirical studies. [ABSTRACT FROM PUBLISHER]

The autonomous vehicle, as an emerging and rapidly growing field, has received extensive attention for its futuristic driving experiences. Although the fast developing depth sensors and machine learning methods have given a huge boost to self-driving research, existing autonomous driving vehicles do meet with several avoidable accidents during their road testings. The major cause is the misunderstanding between self-driving systems and human drivers. To solve this problem, we propose a humanlike driving system in this paper to give autonomous vehicles the ability to make decisions like a human. In our method, a convolutional neural network model is used to detect, recognize, and abstract the information in the input road scene, which is captured by the on-board sensors. And then a decision-making system calculates the specific commands to control the vehicles based on the abstractions. The biggest advantage of our work is that we implement a decision-making system which can well adapt to real-life road conditions, in which a massive number of human drivers exist. In addition, we build our perception system with only the depth information, rather than the unstable RGB data. The experimental results give a good demonstration of the efficiency and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

We developed a new empirical model for corotating plasma in the Jovian magnetosphere. The model, named the corotation plasma environment model version 2 (CPEMv2), considers the charge density, velocity vector, and ion temperature based on Galileo/plasma system (PLS) ion data. In addition, we develop hot electron temperature and density models based on Galileo/PLS electron data. All of the models provide respective quantities in the magnetic equator plane of 9–30 $\mathrm {R}_{J}$ , while the charge density model can be extended to 3-D space. A characteristic feature of the CPEM is its support of the percentile as a user input. This feature enables us to model extreme conditions in addition to normal states. In this paper, we review the foundations of the new empirical model, present a general derivation algorithm, and offer a detailed formulation of each parameter of the CPEMv2. As all CPEM parameters are of the analytical form, their implementation is straightforward, and execution involves the use of a small number of computational resources. The CPEM is flexible; for example, it can be extended, as new data (from observations or simulation results) become available. The CPEM can be used for the mission operation of the European Space Agency’s mission to Jupiter, JUpiter ICy moons Explorer (JUICE), and for future data analyses. [ABSTRACT FROM AUTHOR]

IEEE Transactions on Robotics (T-RO) publishes fundamental scientific papers covering all aspects of robotics. Papers reporting on results of major research significance that draw upon any combination of theoretical, algorithmic, experimental, and empirical analysis and validation, including design and fabrication, are welcome. Paper categories include regular papers, short papers, communications, and survey papers. Multimedia aLachments are also welcome. IEEE and T-RO support the evolutionary publication paradigm, in which early versions of a work may have appeared in conferences. Sources must be indicated during the submission procedure, explicitly cited in the manuscript, and any differences, changes, or additions discussed in the text of the paper. Submission information is at URL: http://ieee-ras.org/publications/t-ro . [ABSTRACT FROM AUTHOR]

In this paper, we use cross wavelet transform (XWT) for the analysis and classification of electrocardiogram (ECG) signals. The cross-correlation between two time-domain signals gives a measure of similarity between two waveforms. The application of the continuous wavelet transform to two time series and the cross examination of the two decompositions reveal localized similarities in time and frequency. Application of the XWT to a pair of data yields wavelet cross spectrum (WCS) and wavelet coherence (WCOH). The proposed algorithm analyzes ECG data utilizing XWT and explores the resulting spectral differences. A pathologically varying pattern from the normal pattern in the QT zone of the inferior leads shows the presence of inferior myocardial infarction. A normal beat ensemble is selected as the absolute normal ECG pattern template, and the coherence between various other normal and abnormal subjects is computed. The WCS and WCOH of various ECG patterns show distinguishing characteristics over two specific regions R1 and R2, where R1 is the QRS complex area and R2 is the T-wave region. The Physikalisch-Technische Bundesanstalt diagnostic ECG database is used for evaluation of the methods. A heuristically determined mathematical formula extracts the parameter(s) from the WCS and WCOH. Empirical tests establish that the parameter(s) are relevant for classification of normal and abnormal cardiac patterns. The overall accuracy, sensitivity, and specificity after combining the three leads are obtained as 97.6%, 97.3%, and 98.8%, respectively. [ABSTRACT FROM AUTHOR]

A growing number of efforts aim to understand what people see when using a visualization. These efforts provide scientific grounding to complement design intuitions, leading to more effective visualization practice. However, published visualization research currently reflects a limited set of available methods for understanding how people process visualized data. Alternative methods from vision science offer a rich suite of tools for understanding visualizations, but no curated collection of these methods exists in either perception or visualization research. We introduce a design space of experimental methods for empirically investigating the perceptual processes involved with viewing data visualizations to ultimately inform visualization design guidelines. This paper provides a shared lexicon for facilitating experimental visualization research. We discuss popular experimental paradigms, adjustment types, response types, and dependent measures used in vision science research, rooting each in visualization examples. We then discuss the advantages and limitations of each technique. Researchers can use this design space to create innovative studies and progress scientific understanding of design choices and evaluations in visualization. We highlight a history of collaborative success between visualization and vision science research and advocate for a deeper relationship between the two fields that can elaborate on and extend the methodological design space for understanding visualization and vision. [ABSTRACT FROM AUTHOR]

Coherence time plays an important role in ocean surface observation by means of synthetic aperture radar (SAR). For X-band, the autocorrelation function (ACF) of the backscatter from the sea surface has been predicted using models. Up to now there exist only few experimental data and an empirical analysis of the sea surface coherence time based on a larger amount of data is missing. In this paper, we show how space-based bistatic SAR along-track interferometry is used for the first time to measure the ACF. We achieve this by statistical analysis of the interferometric scene coherence as a function of the along-track interferometric time lag, instead of indirect estimation from the Doppler bandwidth. We use many data acquisitions of opportunity for that purpose, thereby exploiting the diversity of along-track baselines between the two SAR sensors of the TanDEM-X satellite formation. The ACF is related to the temporal decorrelation, which is not directly observable. We extract this component by quantifying and compensating for all other sources of decorrelation that contribute to the observed coherence. [ABSTRACT FROM PUBLISHER]

In this paper, we investigate computationally efficient strategies for energy-efficiency (EE) maximization in the uplink of a single-cell orthogonal frequency-division multiple-access system. We realize EE maximization by allocating power and subcarriers for each mobile station, and it has exponential complexity to achieve optimality. The Dinkelbach algorithm and Lagrange dual decomposition are well-established frameworks to solve the EE-maximization problem, but the definition of initial values and parameters remains an unresolved issue that is generally solved by empirical methods. To do this, we develop suitable procedures to efficiently calculate such values. We show that these procedures provide a very high EE, such that they can be systematized in two sub-optimal algorithms to efficiently allocate power and subcarriers without reliance on parameter-optimization problems. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a method for simulating the response of SRAM cells to particles through basic electrical models of the composing transistors without empirical parameters to be set. This simplified approach is motivated by the fact that the SPICE parameters are often not known by the end-user. In this approach, the transient currents induced by primary or secondary ions are calculated in the framework of the diffusion/collection model. Then, they are used to calculate the voltage variation at each node of the cell in which the current of each of the four latch transistors is accounted for and is given by a standard first-order model in which three major regions of operation are modeled (subthreshold, triode and saturation). Presented results are in good agreement with ion/neutron/proton experiment for 150 nm, 90 nm and 65 nm SRAMs. Even if the proposed approach would need further validation, this paper is a first step to give guideline on how calculating SEU cross section based on not detailed technological parameters instead of empirical ones. The strong point of our proposal is that it dynamically accounts for the behavior of the four transistors of refreshing loop in the cell. [ABSTRACT FROM AUTHOR]