Storage Mechanism of Processing Magnanimity Small Files Applying HDFS Architecture

Wang Xin; Jun Ma

doi:10.4028/www.scientific.net/AMR.765-767.1614

Paper Titles

Design and Implementation of Keywords Extraction and Management System Based on Java Platform
p.1596

Research of the Key Technology of AIS Base Station Web System
p.1600

Keywords Extraction Based on Text Classification
p.1604

Analysis Framework of Freemodbus
p.1610

Storage Mechanism of Processing Magnanimity Small Files Applying HDFS Architecture
p.1614

Analysis Method of Dynamic Extensible Service Model Based on Petri-Net
p.1618

Auto Covariance Combined with Artificial Neural Network for Predicting Protein-Protein Interactions
p.1622

Design and Realization of the Evaluation of Expert Reputation in the Online Evaluation System for the Master Dissertation
p.1625

A TPA Based Efficient Non-Repudiation Scheme for Cloud Storage
p.1630

HomeAdvanced Materials ResearchAdvanced Materials Research Vols. 765-767Storage Mechanism of Processing Magnanimity Small...

Storage Mechanism of Processing Magnanimity Small Files Applying HDFS Architecture

Abstract:

Pointing at the low efficiency problem of I/O operations of small files using HDFS application architecture, this article proposes a new method, that is, add disk buffer on DataNode to reduce addressing time of reading small files, and thus reduce reading time, and reduce the stress bringed by frequent NameNode access, so as to optimize the reading efficiency of small files. Experiment results show that the design programme is feasible, and adding disk buffer benefits for increasing efficiency of processing small files applying HDFS.