Coder Social home page Coder Social logo

altinity / clickhouse Goto Github PK

View Code? Open in Web Editor NEW

This project forked from clickhouse/clickhouse

25.0 25.0 2.0 1.13 GB

ClickHouse® is a free analytics DBMS for big data

Home Page: https://clickhouse.com

License: Apache License 2.0

CMake 0.98% Shell 2.76% Python 5.63% C++ 82.81% C 1.88% Dockerfile 0.05% Perl 0.01% Assembly 4.97% HTML 0.11% CSS 0.01% JavaScript 0.01% PHP 0.01% Go 0.27% Java 0.02% Cap'n Proto 0.01% ANTLR 0.03% Vim Script 0.01% C# 0.01% Clojure 0.07% Jinja 0.38%

clickhouse's People

Contributors

4ertus2 avatar akuzm avatar al13n321 avatar alesapin avatar alexey-milovidov avatar algunenano avatar amosbird avatar antonio2368 avatar avogar avatar azat avatar blinkov avatar curtizj avatar danroscigno avatar devcrafter avatar evillique avatar felixoid avatar kitaisreal avatar kochetovnicolai avatar kssenii avatar mergify[bot] avatar nikitamikhaylov avatar proller avatar qoega avatar rschu1ze avatar serxa avatar taiyang-li avatar tavplubix avatar ucasfl avatar vdimir avatar yakov-olkhovskiy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

arthurpassosorg

clickhouse's Issues

[Urgent Help] Slow Performance with huge resource usage

Hi,

I am new to Clickhouse and I am raising to get an urgent help since I am implementing a big system using Clickhouse.

  • CH version - 23.3.8.22.altinitystable
  • Machine - centos79
  • CPU - 8
  • Nodes - 20
  • Memory - 32 GB
  • Shards - 5

Table Structure- ReplicatedMergeTree

CREATE TABLE table ON CLUSTER '{cluster}' ( col1INT(),col2INT(),col3INT(),col4INT(),timeDateTime,col5FLOAT(64),col6 INT() ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{cluster}/{shard}/<>/<>', '{replica}') ORDER BY (col1, toYYYYMMDD(time), col2, col3, col4) PARTITION BY (toYYYYMMDD(time)) PRIMARY KEY (col1)

Select Query -

SELECT col1, col2, col3, col4, time, col5, col6 FROM table WHERE col1 = 30 AND col2 = 11 ORDER BY time LIMIT 10 OFFSET 1

I am trying to hit a huge request with this query to CH cluster and the CPU is using around 80% of each nodes and my response time seems to be around 40 ms in each nodes.

I have connected CH to an golang API but my QPS of the API seems very less with max of 5K with big API and CH cluster resources.

I am using an external load balancer to clickhouse tcp client connection with 9000 port.
I feel the response time seems very less and I want to have a 100k qps with the system.
Am I aiming too much with this resources?

Do I need more resources for this usecase.
But I have seen people getting maximum performance with one or two servers.

Please help me to increase the performance or let me know If I am missing something!
Thanks in Advance!

debug RPM

We need Altinity rpm with debug info

#51492 Fix Block structure mismatch in Pipe::unitePipes for FINAL is not in altinity build

Describe the issue
According to the documentation at https://docs.altinity.com/releasenotes/altinity-stable-release-notes/22.8/altinity-stable-22.8.20/, PR #278 should be not related to #51492

I noticed ClickHouse#51492 is not v22.8.20.12.altinitystable or the most recent v23.3.8.22.altinitystable but it is v22.8.21.38-lts and v23.8.2.8-lts in the official CH repo.

Could you please take a look and see if we can get a fresh build out fast with this fix?

Thanks,

After update to v21.8.11.1 Altinity stable build RSS constantly increasing.

image
Almost all tables replicate between nodes. Stored data is Graphite and Prometheus metrics. Carbon-clickhouse used as a writer.
Graphite-clickhouse used as a reader.
04,05,06 - read/write nodes.
02 - just replica, not used for reads/writes.
mark_cache_size is set to default 5GB.
Dictionaries are not used.
The uncompressed cache is disabled.

Executed on 04:
image
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.