zqr10159 / greptimedb

An open-source, cloud-native, distributed time-series database with PromQL/SQL/Python supported.

The next-generation hybrid timeseries/analytics processing database in the cloud

   

 

What is GreptimeDB

GreptimeDB is an open-source time-series database with a special focus on scalability, analytical capabilities and efficiency. It's designed to work on infrastructure of the cloud era, and users benefit from its elasticity and commodity storage.

Our core developers have been building time-series data platform for years. Based on their best-practices, GreptimeDB is born to give you:

• A standalone binary that scales to highly-available distributed cluster, providing a transparent experience for cluster users
• Optimized columnar layout for handling time-series data; compacted, compressed, stored on various storage backends
• Flexible index options, tackling high cardinality issues down
• Distributed, parallel query execution, leveraging elastic computing resource
• Native SQL, and Python scripting for advanced analytical scenarios
• Widely adopted database protocols and APIs
• Extensible table engine architecture for extensive workloads

Quick Start

Build

Build from Source

To compile GreptimeDB from source, you'll need:

• C/C++ Toolchain: provides basic tools for compiling and linking. This is available as build-essential on ubuntu and similar name on other platforms.
• Rust: the easiest way to install Rust is to use rustup, which will check our rust-toolchain file and install correct Rust version for you.
• Protobuf: protoc is required for compiling .proto files. protobuf is available from major package manager on macos and linux distributions. You can find an installation instructions here. Note that protoc version needs to be >= 3.15 because we have used the optional keyword. You can check it with protoc --version.
• python3-dev or python3-devel(Optional feature, only needed if you want to run scripts in CPython, and also need to enable pyo3_backend feature when compiling(by cargo run -F pyo3_backend or add pyo3_backend to src/script/Cargo.toml 's features.default like default = ["python", "pyo3_backend])): this install a Python shared library required for running Python scripting engine(In CPython Mode). This is available as python3-dev on ubuntu, you can install it with sudo apt install python3-dev, or python3-devel on RPM based distributions (e.g. Fedora, Red Hat, SuSE). Mac's Python3 package should have this shared library by default. More detail for compiling with PyO3 can be found in PyO3's documentation.

Build with Docker

A docker image with necessary dependencies is provided:

docker build --network host -f docker/Dockerfile -t greptimedb .

Run

Start GreptimeDB from source code, in standalone mode:

cargo run -- standalone start

Or if you built from docker:

docker run -p 4002:4002 -v "$(pwd):/tmp/greptimedb" greptime/greptimedb standalone start

For more startup options, greptimedb's distributed mode and information about Kubernetes deployment, check our docs.

Connect

1. Connect to GreptimeDB via standard MySQL client:

   # The standalone instance listen on port 4002 by default.
   mysql -h 127.0.0.1 -P 4002
   

2. Create table:

   CREATE TABLE monitor (
     host STRING,
     ts TIMESTAMP,
     cpu DOUBLE DEFAULT 0,
     memory DOUBLE,
     TIME INDEX (ts),
     PRIMARY KEY(host)) ENGINE=mito WITH(regions=1);

3. Insert some data:

   INSERT INTO monitor(host, cpu, memory, ts) VALUES ('host1', 66.6, 1024, 1660897955000);
   INSERT INTO monitor(host, cpu, memory, ts) VALUES ('host2', 77.7, 2048, 1660897956000);
   INSERT INTO monitor(host, cpu, memory, ts) VALUES ('host3', 88.8, 4096, 1660897957000);

4. Query the data:

   SELECT * FROM monitor;

   +-------+---------------------+------+--------+
   | host  | ts                  | cpu  | memory |
   +-------+---------------------+------+--------+
   | host1 | 2022-08-19 08:32:35 | 66.6 |   1024 |
   | host2 | 2022-08-19 08:32:36 | 77.7 |   2048 |
   | host3 | 2022-08-19 08:32:37 | 88.8 |   4096 |
   +-------+---------------------+------+--------+
   3 rows in set (0.01 sec)
   

You can always cleanup test database by removing /tmp/greptimedb.

Resources

Installation

• Pre-built Binaries: For Linux and macOS, you can easily download pre-built binaries that are ready to use. In most cases, downloading the version without PyO3 is sufficient. However, if you plan to run scripts in CPython (and use Python packages like NumPy and Pandas), you will need to download the version with PyO3 and install a Python with the same version as the Python in the PyO3 version. We recommend using virtualenv for the installation process to manage multiple Python versions.
• Docker Images(recommended): pre-built Docker images, this is the easiest way to try GreptimeDB. By default it runs CPython script with pyo3_backend enabled.
• gtctl: the command-line tool for Kubernetes deployment

Documentation

• GreptimeDB User Guide
• GreptimeDB Developer Guide
• GreptimeDB internal code document

Dashboard

• The dashboard UI for GreptimeDB

SDK

• GreptimeDB Java Client

Project Status

This project is in its early stage and under heavy development. We move fast and break things. Benchmark on development branch may not represent its potential performance. We release pre-built binaries constantly for functional evaluation. Do not use it in production at the moment.

For future plans, check out GreptimeDB roadmap.

Community

Our core team is thrilled to see you participate in any ways you like. When you are stuck, try to ask for help by filling an issue with a detailed description of what you were trying to do and what went wrong. If you have any questions or if you would like to get involved in our community, please check out:

• GreptimeDB Community on Slack
• GreptimeDB GitHub Discussions
• Greptime official Website

In addition, you may:

• View our official Blog
• Connect us with Linkedin
• Follow us on Twitter

License

GreptimeDB uses the Apache 2.0 license to strike a balance between open contributions and allowing you to use the software however you want.

Contributing

Please refer to contribution guidelines for more information.

Acknowledgement

• GreptimeDB uses Apache Arrow as the memory model and Apache Parquet as the persistent file format.
• GreptimeDB's query engine is powered by Apache Arrow DataFusion.
• OpenDAL from Datafuse Labs gives GreptimeDB a very general and elegant data access abstraction layer.
• GreptimeDB’s meta service is based on etcd.
• GreptimeDB uses RustPython for experimental embedded python scripting.