《電子技術(shù)應(yīng)用》
您所在的位置:首頁 > 可編程邏輯 > 設(shè)計應(yīng)用 > 智能網(wǎng)卡加速Ceph存儲的性能研究
智能網(wǎng)卡加速Ceph存儲的性能研究
電子技術(shù)應(yīng)用
劉寶琴,羅向征,林茂,王欽雅,蘭麗莎
邁普通信技術(shù)股份有限公司
摘要: 聚焦Ceph存儲系統(tǒng)對象存儲設(shè)備(Object Storage Device, OSD)架構(gòu)線程鎖競爭機制所導(dǎo)致的多核并行擴展能力受限問題,針對下一代Crimson-OSD架構(gòu)與智能網(wǎng)卡協(xié)同優(yōu)化技術(shù)開展研究,提出分層協(xié)同優(yōu)化框架。相關(guān)研究表明,采用智能網(wǎng)卡協(xié)同優(yōu)化,RDMA網(wǎng)絡(luò)卸載降低CPU占用率達到70%,異構(gòu)計算引擎實現(xiàn)糾刪碼硬件加速提升數(shù)據(jù)恢復(fù)速度達到4.84倍。研究成果為分布式存儲系統(tǒng)的硬件加速提供相關(guān)理論依據(jù)與關(guān)鍵技術(shù)參考,對高性能計算和云邊端融合等數(shù)據(jù)密集型場景的存儲系統(tǒng)優(yōu)化具有指導(dǎo)意義。
中圖分類號:TN915.05 文獻標(biāo)志碼:A DOI: 10.16157/j.issn.0258-7998.256678
中文引用格式: 劉寶琴,羅向征,林茂,等. 智能網(wǎng)卡加速Ceph存儲的性能研究[J]. 電子技術(shù)應(yīng)用,2025,51(12):14-19.
英文引用格式: Liu Baoqin,Luo Xiangzheng,Lin Mao,et al. Research on accelerating Ceph storage performance with SmartNICs[J]. Application of Electronic Technique,2025,51(12):14-19.
Research on accelerating Ceph storage performance with SmartNICs
Liu Baoqin,Luo Xiangzheng,Lin Mao,Wang Qinya,Lan Lisha
Maipu Communication Technology Co., Ltd.
Abstract: This paper focuses on the issue of limited multi-core parallel scalability caused by thread lock contention mechanisms in the architecture of the Ceph storage system's Object Storage Device (OSD). It conducts research on collaborative optimization technologies between the next-generation Crimson-OSD architecture and SmartNICs, proposing a hierarchical cooperative optimization framework. Related studies demonstrate that employing SmartNIC-based cooperative optimization achieves a 70% reduction in CPU utilization through RDMA network offloading, while heterogeneous computing engines enable hardware acceleration for erasure coding, improving data recovery speed by 4.84 times. The research outcomes provide theoretical foundations and key technical references for hardware acceleration in distributed storage systems, offering guidance for optimizing storage systems in data-intensive scenarios such as high-performance computing and cloud-edge-end integration.
Key words : SmartNIC;Ceph storage system;performance optimization;hardware acceleration;distributed storage system

引言

以AI訓(xùn)練、HPC、邊緣計算為代表的數(shù)據(jù)密集型應(yīng)用爆發(fā)式增長對存儲系統(tǒng)的性能與彈性提出前所未有的挑戰(zhàn)。Ceph憑借高可用性與可擴展性優(yōu)勢在云數(shù)據(jù)中心得到廣泛應(yīng)用,但其傳統(tǒng)OSD架構(gòu)在多核場景下因線程鎖競爭與跨核通信開銷,導(dǎo)致處理器(CPU)利用率偏低,難以適配NVMe SSD等高性能硬件。Ceph社區(qū)為此重構(gòu)了Crimson-OSD架構(gòu),通過Shared-Nothing設(shè)計與異步流水線模型,優(yōu)化多核擴展性。實際測試表明: 8線程配置下,4K隨機讀IOPS性能達到311k,隨著核數(shù)增長,性能得到進一步提升,驗證了架構(gòu)重構(gòu)的有效性。盡管Crimson-OSD架構(gòu)設(shè)計取得了長足進步,但在借助智能網(wǎng)卡可編程加速能力來開展協(xié)同優(yōu)化方面的研究仍顯不足。

針對Crimson-OSD 架構(gòu)特點與性能瓶頸分析的基礎(chǔ)上,本文提出基于智能網(wǎng)卡的分層協(xié)同優(yōu)化框架,其核心內(nèi)容包括兩個方面,首先是建立關(guān)鍵參數(shù)性能敏感性模型,對Crimson-OSD多核擴展能力進行量化分析;其次設(shè)計分層協(xié)同優(yōu)化框架,突破CPU算力對存儲系統(tǒng)性能的制約。進一步對存算一體架構(gòu)與AI賦能動態(tài)管理前沿方向進行了初步探討。


本文詳細內(nèi)容請下載:

http://www.ihrv.cn/resource/share/2000006869


作者信息:

劉寶琴,羅向征,林茂,王欽雅,蘭麗莎

(邁普通信技術(shù)股份有限公司,四川 成都 610094)


subscribe.jpg

此內(nèi)容為AET網(wǎng)站原創(chuàng),未經(jīng)授權(quán)禁止轉(zhuǎn)載。