A scalability optimization algorithm for big data storage architecture based on Kalman algorithm
Han Zhenyang, Zhang Lei, Ren Dong
(Shanxi Provincial Corps of the Chinese People′s Armed Police Force, Xi′an 710116,China)
Abstract： In order to optimize the scalability performance of big data storage architecture and improve the resource utilization of big data architecture, a Kalman algorithm was introduced to design a scalability optimization algorithm for big data storage architecture. Firstly, considering the compatibility between big data storage architecture and multi core environment memory layout, design the architecture memory layout. Secondly, design a distributed shared memory protocol to ensure that various processes can work together correctly when accessing shared memory, and improve the fault tolerance of the storage architecture. On this basis, the Kalman algorithm is used to dynamically adjust the load of storage nodes and optimize the big data storage architecture to improve its scalability. The experimental results show that the resource utilization rate of the big data storage architecture is consistently higher than that of the control group, reaching over 96%, with a maximum of 98%. The scalability optimization effect of the architecture is significant, and the utilization of server resources is more sufficient, enabling more efficient processing of large-scale data.
Key words : Kalman algorithm; big data storage architecture; scalability optimization; shared memory protocol; node load
