Big data compression processing and verification based on Hive for smart substation

The capacity and the scale of smart substation are expanding constantly, with the characteristics of information digitization and automation, leading to a quantitative trend of data. Aiming at the existing processing shortages in the big data processing, the query and analysis of smart substation, a data compression processing method is proposed for analyzing smart substation and Hive. Experimental results show that the compression ratio and query time of RCFile storage format are better than those of TextFile and SequenceFile. The query efficiency is improved for data compressed by Deflate, Gzip and Lzo compression formats. The results verify the correctness of adjacent speedup defined as the index of cluster efficiency. Results also prove that the method has a significant theoretical and practical value for big data processing of smart substation.