Search

Big data is growing fast following Big data has the characteristics of high volume, speed, and variety (3v) and continues to grow exponentially following the development of the world's use of information and communication technology. The main problem with big data is the data deluge. The need for technology and large data storage and processing methods to keep pace with the exponential data growth rate is potentially limitless, giving rise to the problem of exponentially increasing technology requirements as well. The weakness of previous big data analysis approaches (batch and online real time processing) is that it requires high technology (large storage, memory and processing). This paper proposes a new approach in big-data analysis by separating the basic knowledge (BK) construction process from original data with a much smaller volume (volume reduction), then analyzing it into final knowledge, thus requiring smaller/simpler analysis technology. The proposals include formulating the definition and representation of BK, developing methods for constructing BK from source data, and analyzing BK into final knowledge. We propose a BK construction method based on a knowledge extraction technique using BIRCH clustering algorithm for instance reduction, and handling high-dimensional problems by parallelizing the dimension calculation process to calculate the distance between instances. We use the Adjusted Rand Index (ARI) to measure the similarity of the final knowledge of the baseline and proposed methods. First, the BIRCH baseline was modified by parallelizing the calculations has succeeded in increasing speed from 17% to 25%. Next, the parallel BIRCH (PBIRCH) baseline was broken into BK construction and BK analysis, has succeeded in reducing volume by 96% or more and increasing speed by 43.50%, with similar final knowledge results (ARI=1). Based on these results, we conclude that the BK construction method and analysis from BK into final knowledge for highdimensional big data have significantly reduced volume and speed up the analytical process without reducing the quality of the final knowledge. [ABSTRACT FROM AUTHOR]

Building energy problems have various kinds of aspects, one of which is the difficulty of measuring energy efficiency. With current data development, energy efficiency measurements can be made by developing predictive models to estimate future building needs. However, with the massive amount of data, several problems arise regarding data quality and the lack of scalability in terms of computation memory and time in modeling. In this study, we used data reduction and ensemble learning techniques to overcome these problems. We used numerosity reduction, dimension reduction, and a LightGBM model based on boosting added with a bagging technique, which we compared with incremental learning. Our experimental results showed that the numerosity reduction and dimension reduction techniques could speed up the training process and model prediction without reducing the accuracy. Testing the ensemble learning model also revealed that bagging had the best performance in terms of RMSE and speed, with an RMSE of 262.304 and 1.67 times faster than the model with incremental learning. [ABSTRACT FROM AUTHOR]