joykk / cubefs

CubeFS is a cloud native distributed storage platform.

CubeFS

Community Meeting
The CubeFS Project holds bi-weekly community online meeting. To join or watch previous meeting notes and recordings, please see meeting schedule and meeting minutes.

Overview

CubeFS (储宝文件系统 in Chinese) is a cloud-native storage platform be hosted by the Cloud Native Computing Foundation (CNCF) as a incubating project.

CubeFS has been commonly used as the underlying storage infrastructure for online applications, database or data processing services and machine learning jobs orchestrated by Kubernetes. An advantage of doing so is to separate storage from compute - one can scale up or down based on the workload and independent of the other, providing total flexibility in matching resources to the actual storage and compute capacity required at any given time.

Some key features of CubeFS include:

• Multiple Protocols Support
  Allowing users to access the same file over multiple protocols, such as POSIX/HDFS/S3
• Highly Scalable Metadata Service
  Elasticity, scalability and strong consistency of metadata
• Excellent Storage Performance
  A lot of performance optimizations for large/small files and sequential/random writes
• Multi-tenancy
  Provides an efficient mechanism to improve resource utilization and tenant isolation
• Hybrid Cloud Acceleration
  Accelerate Hybrid Cloud IO performance through multi-level cache
• Multiple Storage Engines
  Provides high-performance replication engine and low-cost erasure coding engine

Documents

• English version: https://cubefs.readthedocs.io/en/latest/
• Chinese version: https://cubefs.readthedocs.io/zh_CN/latest/

Benchmark

Small file operation performance and scalability benchmark test by mdtest.

File Size (KB)	1	2	4	8	16	32	64	128
Creation (TPS)	70383	70383	73738	74617	69479	67435	47540	27147
Read (TPS)	108600	118193	118346	122975	116374	110795	90462	62082
Removal (TPS)	87648	84651	83532	79279	85498	86523	80946	84441
Stat (TPS)	231961	263270	264207	252309	240244	244906	273576	242930

Refer to cubefs.readthedocs.io for performance and scalability of IO and Metadata.

Setup CubeFS

• Set up a small CubeFS cluster
• Helm chart to Run a CubeFS Cluster in Kubernetes

Community

• Mailing list: [email protected]
• Slack: cubefs.slack.com
• WeChat: detail see here.

Partners and Users

For a list of users and success stories see ADOPTERS.md.

License

CubeFS is licensed under the Apache License, Version 2.0. For detail see LICENSE and NOTICE.

Note

The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order to get a stable set of binaries.