Search

In the future, the Internet of things (IoT) may provide huge volumes of data. Smart grids are a class of IoT electricity distribution systems that can control bidirectional energy flows between consumers and service providers (Barman et al. "IOT Based Smart Energy Meter for Efficient Energy Utilization in Smart Grid", 978-1-5386-4769-1 1831.00, 2018 IEEE). A typical smart grid features an advanced metering infrastructure (AMI), which automatically collects meter data from widely distributed sensors. A utility company that intends to use AMI must deploy smart meters, data concentrator units, and a meter data management system (MDMS). Concentrators collect ubiquitous messages from smart meters and transmit aggregated data to their MDMS. Although ubiquitous messages may enhance the efficiency of some electricity grids, any excessive volume of messages causes data congestion. This research considers the Reference Energy Disaggregation Data Set. Our proposed algorithm can compress meter data efficiently. Our contributions are as follows: First, thus far, numerous researchers have attempted to address smart grid problems with AMI systems, and here, we provide a relatively complete review of these attempts. Second, this paper presents a strategy to analyze this problem and propose a compression algorithm to compress meter data. This proposed algorithm combines several existing compression algorithms and operates from 2 to 10% more efficiently than previously published algorithms. [ABSTRACT FROM AUTHOR]

The volume of telemetry data is gradually increasing, both because of the increasingly larger number of parameters involved, and the use of higher sampling frequencies. Efficient data compression schemes are therefore needed in space telemetry systems to improve transmission efficiency and reduce the burden of required spacecraft resources, in particular regarding their transmitter power. In our primary study, a D-CLU algorithm was proposed to perform lossless compression for telemetry data, and achieve better performance. However, a limitation of this algorithm is that the compression time may become longer when the clustering head (CH) and outlier (which are compressed by LZW algorithm) numbers increase. To reduce compression delay, this paper proposed a modified character string (MCS) parallel search strategy for LZW algorithm (denoted by MCS-based LZW). The proposed MCS-based LZW algorithm designs coding principle, dictionary update rule and search strategy according to the character string matching results. Example verification and simulation results show that the proposed algorithm can effectively decrease the dictionary search times, and thus reduce the compression time. [ABSTRACT FROM AUTHOR]

The work is devoted to the description of the development of compression algorithms for hyperspectral aerospace images based on discrete orthogonal transformations for the purpose of subsequent compression in Earth remote sensing systems. As compression algorithms necessary to reduce the amount of transmitted information, it is proposed to use the developed compression methods based on Walsh-Hadamard transformations and discrete-cosine transformation. The paper considers a methodology for developing lossy and high-quality compression algorithms during recovery of 85 % or more, taking into account which an adaptive algorithm for compressing hyperspectral AI and the generated quantization table have been developed. The existing solutions to the lossless compression problem for hyperspectral aerospace images are analyzed. Based on them, a compression algorithm is proposed taking into account inter-channel correlation and the Walsh-Hadamard transformation, characterized by data transformation with a decrease in the range of the initial values by forming a set of channel groups [10–15] with high intra-group correlation [0.9–1] of the corresponding pairs with the selection of optimal parameters. The results obtained in the course of the research allow us to determine the optimal parameters for compression: the results of the compression ratio indicators were improved by more than 30 % with an increase in the size of the parameter channels. This is due to the fact that the more values to be converted, the fewer bits are required to store them. The best values of the compression ratio [8–12] are achieved by choosing the number of channels in an ordered group with high correlation. [ABSTRACT FROM AUTHOR]

Mud pulse transmission technology is widely used in logging while drilling (LWD), while its low data transmission efficiency has restricted the development of LWD technology for a long time. In order to improve the equivalent data transmission efficiency of LWD system, a data compression method, namely, Frame Prediction Huffman Coding (FPHC) with a group of frames, is proposed in this paper. Several factors including the data accuracy, the need for real-time transmission, and the existence of transmission errors are comprehensively considered in the proposed method. Through the probability statistical analysis of LWD data, it is proved that the data have a relatively stable probability distribution after predictive coding. In addition, the Huffman code table generated from predictive coding is universally applicable to other LWD data. Through data compression experiments on logging data, it is verified that the proposed method has higher compression efficiency compared with Huffman coding, Lempel–Ziv–Welch algorithm, adaptive Huffman coding and adaptive arithmetic coding. This lossless compression method will have a good application prospect because of its ability to prevent error diffusion, simple implementation and easy to be embedded in down-hole equipment. [ABSTRACT FROM AUTHOR]

A standard interchangeable data format was recently developed through the Transportation Pooled-Fund Study TPF-5(299) to meet different requirements and needs for two- and three-dimensional (2D/3D) pavement image data storage and exchange. Considering pavement images occupy large amounts of storage space, efficient compression algorithms for both 8-bit 2D and higher-bit 3D images become one of the key components for a successful implementation of the standard. Highway agencies and industry vendors exclusively use the Joint Photographic Experts Group (JPEG) or its successor JPEG 2000 standard for 2D image compression, while the compression of 3D 16-bit images remains to be proprietary. The goal of this paper is to evaluate suitable algorithms for compressing 3D pavement images. An overview of the state-of-the-art pavement image data collection technologies was performed, and the relevant image data formats were identified. Subsequently, five compression algorithms suitable for 16-bit image data were selected based on results from a comprehensive survey of the TPF-5(299) technical community. The compressions of several representative field images were implemented and evaluated in terms of the computation efficiency and retaining of image quality. The expected benefits of this study include facilitating standard and efficient pavement image data collection and accelerating the development of consistent pavement condition evaluation. [ABSTRACT FROM AUTHOR]

Lossless data compression algorithms can use statistical redundancy to represent data using a fewer number of bits in comparison to the original uncompressed data. Run-Length Encoding is one of the simplest lossless compression algorithms in terms of understanding its principles and software implementation, as well as in terms of temporal and spatial complexity. If this principle is applied to individual bits of original uncompressed data without respecting the byte boundaries, this approach is referred to as bit-level Run-Length Encoding. Algorithm for lossless data compression, proposed in this paper, optimizes bit-level Run-Length Encoding data compression, uses special encoding of repeating data blocks, and, if necessary, combines it with delta data transformation or representation of data in its original form intending to increase compression efficiency compared to a conventional bit-level Run-Length Encoding approach. The advantage of the algorithm proposed in this paper is in the increase of the compression ratio in comparison to the bit-level Run-Length Encoding, with the higher time and space consumption as the trade-off. Test results, that were obtained by compression of the segmentation metadata of different volume datasets take place in the last part of the paper. [ABSTRACT FROM AUTHOR]

Graphical data is comprised of a graph with marks on its edges and vertices. The mark indicates the value of some attribute associated to the respective edge or vertex. Examples of such data arise in social networks, molecular and systems biology, and web graphs, as well as in several other application areas. Our goal is to design schemes that can efficiently compress such graphical data without making assumptions about its stochastic properties. Namely, we wish to develop a universal compression algorithm for graphical data sources. To formalize this goal, we employ the framework of local weak convergence, also called the objective method, which provides a technique to think of a marked graph as a kind of stationary stochastic processes, stationary with respect to movement between vertices of the graph. In recent work, we have generalized a notion of entropy for unmarked graphs in this framework, due to Bordenave and Caputo, to the case of marked graphs. We use this notion to evaluate the efficiency of a compression scheme. The lossless compression scheme we propose in this paper is then proved to be universally optimal in a precise technical sense. It is also capable of performing local data queries in the compressed form. [ABSTRACT FROM AUTHOR]

For the problem of data accumulation caused by massive sensor data in transmission line condition monitoring system, this paper analyzes the type and amount of data in the transmission line sensor network, compares the compression algorithms of wireless sensor network data at home and abroad, and proposes an efficient lossless compression algorithm suitable for sensor data in transmission line linear heterogeneous networks. The algorithm combines the wavelet compression algorithm and the neighborhood index sequence algorithm. It displays a fast operation speed and requires a small amount of calculation. It is suitable for battery powered wireless sensor network nodes. By combining wavelet correlation analysis and neighborhood index sequence coding, the compression algorithm proposed in this paper can achieve a high compression rate, has strong robustness to packet loss, has high compression performance, and can help to reduce network load and the packet loss rate. Simulation results show that the proposed method achieves a high compression rate in the compression of the transmission line parameter dataset, is superior to the existing data compression algorithms, and is suitable for the compression and transmission of transmission line condition monitoring data. [ABSTRACT FROM AUTHOR]

Today, there is a huge demand for data compression due to the need to reduce the transmission time and increase the capacity of data storage. Data compression is a technique which represents an information, images, video files in a compressed or in a compact format. There are various data compression techniques which keep information as accurately as possible with the fewest number of bits and send it through communication channel. Arithmetic algorithm, Lempel–Ziv 77 (LZ77) and run length encoding with a K-precision (K-RLE) algorithms are lossless data compression algorithms which have lower performance rate because of their processing complexity as well as execution time. This paper presents an efficient parallel approach to reduce execution time for compression algorithms. The proposed OpenMP is an efficient tool for programming within parallel shared-memory environments. Finally, it shows that performance parallel model experimented using Open Multi-Processing (OpenMP) Application Programming Interface through Intel Parallel studio on multicore architecture platform with spec of Core 2 duo—2.4 GHz, 1 Gb RAM machine of parallel approach for compression algorithms has been improved remarkably against sequential approach. The improvement in compression ratio through an efficient parallel approach leads to reduction on transmission cost, reduction in storage space and bandwidth without additional hardware infrastructure. An overall performance evaluation shows arithmetic data compression algorithm with 46% which is better than LZ77 of 44% as well as K-RLE of 37% data compression algorithms. [ABSTRACT FROM AUTHOR]

To augment situational awareness capabilities for high-frequency disturbances, the modern power system necessitates the implementation of high-sampling rates for real-time Synchro-Waveform (SW) measurements. Nevertheless, the nonlinear nature of data collected from SW measurement devices poses significant challenges to the communication and storage capacities of data centers. To effectively tackle this issue, this paper proposes a Multi-stage Hybrid Coding (MHC) method for the lossless compression of SW data. The MHC method utilizes a well-structured two-stage approach to achieve efficient compression. In the first stage, a variable frame-based method is designed to reduce redundant information and achieve initial compression using multiple periodic recursive compression. Subsequently, the dictionary-based method, specifically the Lempel–Ziv–Markov chain algorithm, is seamlessly integrated to further compress the SW data. Experimental and numerical analyses are conducted using both simulated and real-world source data, with real-time performance validation in SW measurement units. The obtained results convincingly demonstrate that the proposed method enables online compression of both event and non-event SW data, achieving an impressive reduction of over 73.4% in data space, while maintaining a data loss ratio lower than 0.4% for high-density SW data. • Multiple periodic recursive method is proposed to eliminate repetitive information. • A multi-stage hybrid coding method is implemented for the lossless compression. • A series of synchro-waveform events are collected to evaluate the performance. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new architecture for the two-level lossless data compression and decompression algorithm proposed in [8] that combines the PDLZW algorithm and an approximated adaptive Huffman algorithm with dynamic-block exchange (AHDB). In the new architecture, we replace the CAM dictionary set used in the PDLZW algorithm with a CAM-tag-based dictionary set to reduce hardware cost and the CAM-based ordered list used in the AHDB algorithm with a memory inter-reference (MIR) stage realized by using two SRAMs. The resulting architecture is then implemented based on cell-based libraries with both 0.35-μm 2P4M and 0.18-μm 1P6M process technologies, respectively. With the same process technology, the prototyped chip demonstrates the new architecture not only has better performance, at least 33% improvement, but also occupies less area, only about 44%, and consumes less power, about 50%, in comparison with the architecture proposed in [8]. In addition, the maximum data rate can achieve 2 Gbps when realizing in 0.35 μm 2P4M process technology and 4 Gbps when realizing in 0.18-μm 1P6M process technology. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new two-stage hardware architecture that combines the features of both parallel dictionary LZW (PDLZW) and an approximated adaptive Huffman (AH) algorithms. In this architecture, an ordered list instead of the tree- based structure is used in the All algorithm for speeding up the compression data rate. The resulting architecture shows that it not only outperforms the AH algorithm at the cost of only one-fourth the hardware resource but it is also competitive to the performance of LZW algorithm (compress). In addition, both compression and decompression rates of the proposed architecture are greater than those of the AH algorithm even in the case realized by software. [ABSTRACT FROM AUTHOR]

Recent advancements in sequencing methods have led to significant increase in sequencing data. Increase in sequencing data leads to research challenges such as storage, transfer, processing, etc. data compression techniques have been opted to cope with the storage of these data. There have been good achievements in compression ratio and execution time. This fast-paced advancement has raised major concerns about the security of data. Confidentiality, integrity, authenticity of data needs to be ensured. This paper presents a novel lossless reference-free algorithm that focuses on data compression along with encryption to achieve security in addition to other parameters. The proposed algorithm uses preprocessing of data before applying general-purpose compression library. Genetic algorithm is used to encrypt the data. The technique is validated with experimental results on benchmark datasets. Comparative analysis with state-of-the-art techniques is presented. The results show that the proposed method achieves better results in comparison to existing methods. [ABSTRACT FROM AUTHOR]

Vector data compression can significantly improve efficiency of geospatial data management, visualization and data transmission over internet. Existing compression methods are either based on information theory for lossless compression mainly or based on map generalization methods for lossy compression. Coordinate values of vector spatial data are mostly represented using floating-point type in which data redundancy is small and compression ratio using lossy algorithms is generally better than that of lossless compression algorithms. The purpose of paper is to implement a new algorithm for efficient compression of vector data. The algorithm, named space division based compression (SDC), employs the basic idea of linear Morton and Geohash encoding to convert floating-point type values to strings of binary chain with flexible accuracy level. Morton encoding performs multiresolution regular spatial division to geographic space. Each level of regular grid splits space horizontally and vertically. Row and column numbers in binary forms are bit interleaved to generate one integer representing the location of each grid cell. The integer values of adjacent grid cells are proximal to each other on one dimension. The algorithm can set the number of divisions according to accuracy requirements. Higher accuracy can be achieved with more levels of divisions. In this way, multiresolution vector data compression can be achieved accordingly. The compression efficiency is further improved by grid filtering and binary offset for linear and point geometries. The vector spatial data compression takes visual lossless distance on screen display as accuracy requirement. Experiments and comparisons with available algorithms show that this algorithm produces a higher data rate saving and is more adaptable to different application scenarios. [ABSTRACT FROM AUTHOR]

Trajectory compression is an efficient way of removing noise and preserving key features in location-based applications. This paper focuses on the dynamic compression of trajectory in memory, where the compression accuracy of trajectory changes dynamically with the different application scenarios. Existing methods can achieve this by adjusting the compression parameters. However, the relationship between the parameters and compression accuracy of most of these algorithms is considerably complex and varies with different trajectories, which makes it difficult to provide reasonable accuracy. We propose a novel trajectory compression algorithm that is based on the finite element method, in which the trajectory is taken as an elastomer to compress as a whole by elasticity theory, and trajectory compression can be thought of as deformation under stress. The compression accuracy can be determined by the stress size that is applied to the elastomer. When compared with the existing methods, the experimental results show that our method can provide more stable, data-independent compression accuracy under the given stress parameters, and with reasonable performance. [ABSTRACT FROM AUTHOR]

Limitations of I/O bandwidth and latency are a serious burden for many data intensive algorithms. This is even more noticeable in distributed heterogeneous clusters that heavily utilize GPU processors, where communication costs and limitation of memory play crucial role. Lightweight lossless compression methods are often applied to cope with those limitations. In this paper we investigate new approaches to parallel lossless lightweight compression methods based on fixed-length minimum bit encoding for GPU processors. By developing new memory organization and by utilizing various inter thread and inter warp communication abilities, we have proposed algorithms which suit GPU architecture better. As a result we have significantly improved compression ratio and bandwidths. This allows for many new applications in computational clusters as well as in computational algorithms. Our claims are supported by tests conducted using simulated data and TPC-H database benchmarking tools. [ABSTRACT FROM AUTHOR]

Classical lossless compression algorithm highly relies on artificially designed encoding and quantification strategies for general purposes. With the rapid development of deep learning, data-driven methods based on the neural network can learn features and show better performance on specific data domains. We propose an efficient deep lossless compression algorithm, which uses arithmetic coding to quantify the network output. This scheme compares the training effects of Bi-directional Long Short-Term Memory (Bi-LSTM) and Transformers on minute-level power data that are not sparse in the time-frequency domain. The model can automatically extract features and adapt to the quantification of the probability distribution. The results of minute-level power data show that the average compression ratio (CR) is 4.06, which has a higher compression ratio than the classical entropy coding method. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of the lossless and near-lossless compression of hyperspectral images and presents two efficient algorithms based on distributed source coding, which perform the lossless compression by means of multilevel scalar codes. The proposed algorithms are implemented on the co-located blocks in the spectral orientation. A novel multiband spectral predictor is proposed to construct the side information of each block. The back-up side information is introduced for the second algorithm to recover the images when the original side information is corrupted by errors. The encoder only requires the transmission of the least significant bit (LSB) bit-planes to the decoder, and the number of bits is computed by the maximum error between the block and its side information. The proposed algorithms are also extended to near-lossless compression. The experimental results show that the proposed algorithms have a competitive compression performance with the existing distributed compression algorithms. Moreover, the proposed algorithms can provide low complexity and different degrees of error resilience, which is suitable for onboard compression. [ABSTRACT FROM AUTHOR]

Telemetric information is great in size, requiring extra room and transmission time. There is a significant obstruction of storing or sending telemetric information. Lossless data compression (LDC) algorithms have evolved to process telemetric data effectively and efficiently with a high compression ratio and a short processing time. Telemetric information can be packed to control the extra room and association data transmission. In spite of the fact that different examinations on the pressure of telemetric information have been conducted, the idea of telemetric information makes pressure incredibly troublesome. The purpose of this study is to offer a subsampled and balanced recurrent neural lossless data compression (SB-RNLDC) approach for increasing the compression rate while decreasing the compression time. This is accomplished through the development of two models: one for subsampled averaged telemetry data preprocessing and another for BRN-LDC. Subsampling and averaging are conducted at the preprocessing stage using an adjustable sampling factor. A balanced compression interval (BCI) is used to encode the data depending on the probability measurement during the LDC stage. The aim of this research work is to compare differential compression techniques directly. The final output demonstrates that the balancing-based LDC can reduce compression time and finally improve dependability. The final experimental results show that the model proposed can enhance the computing capabilities in data compression compared to the existing methodologies. [ABSTRACT FROM AUTHOR]

This paper presents a study of various lossless compression algorithms; to test the performance and the ability of compression of each algorithm based on ten different parameters. For evaluation the compression ratios of each algorithm on different parameters are processed. To classify the algorithms based on the compression ratio, rule base is constructed to mine with frequent bit pattern to analyze the variations in various compression algorithms. Also, enhanced K- Medoid clustering is used to cluster the various data compression algorithms based on various parameters. The cluster falls dissentingly high to low after the enhancement. The framed rule base consists of 1,048,576 rules, which is used to evaluate the compression algorithm. Two hundred and eleven Compression algorithms are used for this study. The experimental result shows only few algorithm satisfies the range "High" for more number of parameters. [ABSTRACT FROM AUTHOR]

In this paper, a new lossless data compression method that is based on digram coding is introduced. This data compression method uses semi-static dictionaries: All of the used characters and most frequently used two character blocks (digrams) in the source are found and inserted into a dictionary in the first pass, compression is performed in the second pass. This two-pass structure is repeated several times and in every iteration particular number of elements is inserted in the dictionary until the dictionary is filled. This algorithm (ISSDC: Iterative Semi-Static Digram Coding) also includes some mechanisms that can decide about total number of iterations and dictionary size whenever these values are not given by the user. Our experiments show that ISSDC is better than LZW/GIF and BPE in compression ratio. It is worse than DEFLATE in compression of text and binary data, but better than PNG (which uses DEFLATE compression) in lossless compression of simple images. [ABSTRACT FROM AUTHOR]

The computer is ending up increasingly more dominant step by step. Information pressure is a well-known way to deal with diminishing information volumes and subsequently bringing down circle I/O and system information exchange times. While a few lossy information pressure systems have shown fantastic pressure proportions, lossless information pressure systems are still among the most well- known ones. Sensor systems speak to a nontraditional wellspring of data, as readings produced by sensors stream consistently, prompting a vast stream of information. Sensors are non-receptive components which are utilized to screen genuine marvels, for example, live climate conditions, organize traffic, and so on. They are typically sorted out into systems where their readings are transmitted utilizing low dimension conventions. [ABSTRACT FROM AUTHOR]

The capacity of the downlink channel is a major bottleneck for applications based on remote sensing hyperspectral imagery (HSI). Data compression is an essential tool to maximize the amount of HSI scenes that can be retrieved on the ground. At the same time, energy and hardware constraints of spaceborne devices impose limitations on the complexity of practical compression algorithms. To avoid any distortion in the analysis of the HSI data, only lossless compression is considered in this study. This work aims at finding the most advantageous compression–complexity trade-off within the state of the art in HSI compression. To do so, a novel comparison of the most competitive spectral decorrelation approaches combined with the best performing low-complexity compressors of the state is presented. Compression performance and execution time results are obtained for a set of 47 HSI scenes produced by 14 different sensors in real remote sensing missions. Assuming only a limited amount of energy is available, obtained data suggest that the FAPEC algorithm yields the best trade-off. When compared to the CCSDS 123.0-B-2 standard, FAPEC is 5.0 times faster and its compressed data rates are on average within 16% of the CCSDS standard. In scenarios where energy constraints can be relaxed, CCSDS 123.0-B-2 yields the best average compression results of all evaluated methods. [ABSTRACT FROM AUTHOR]