Search

Hadoop Distributed File System (HDFS) is based on Google File System (GFS), a big data distributed file management system included in Hadoop. Nowadays, many HDFS and many other similar frameworks have the need to store small files in the system. In this aspect, HDFS affects its performance and Namenode memory management when dealing with a large number of small files. Therefore, researchers have proposed various solutions to address the shortcomings of HDFS for storing small and medium-sized files. This paper presents three HDFS schemes for merging small files and analyzes the importance of correlation and prefetching after merging small files. The efficiency of reading small files can be improved by correlated file prefetching. Finally, the small file storage architecture is obtained to stand superior to the NHAR architecture. [ABSTRACT FROM AUTHOR]