支持EB级存储的元数据服务器集群系统
摘要
With data volumes rapidly growing on a global scale, PB-scale storage systems will not meet the data-storage requirement capacity of applications in the future. It is time for the research and development of EB-scale storage systems that are general purpose, highly scalable, and easily deployed. Metadata server clustering is the core technology for constructing EB-scale storage systems. Based on pNFS and the Blue Whale block device file system EXFS, Blue Whale metadata server clustering technology for EB-scale storage is presented. To begin, the differences between EB-scale metadata server systems and PB-scale are analyzed. Then, the prototype architecture of a metadata server cluster and the distribution strategy of a global namespace are introduced. Following this, the atomic operation protocol for consistency based on a distributed log is presented. Finally, fine-grained metadata migration technology with low latency is proposed. Measurement results confirm that each metadata server can offer more than 10,000 OPS to EB-scale storage systems simultaneously.