qryn Kubernetes setup initialization error about qryn

[JonnyBDev]

Thanks for the changes. Will check in the next hour.

[JonnyBDev]

@lmangani Can confirm that Qryn works with the latest Clickhouse Chart (https://artifacthub.io/packages/helm/bitnami/clickhouse/4.0.4) on Kubernetes v1.27.1. No errors in the initialization of the deployment.

[JonnyBDev]

Additionally it's important to mention that I've tried it with Chart version 1.0.4 (Clickhouse 22.9.5) and it successfully initialized Clickhouse. So it's obvious that the recommended version is working but not the "latest.

[lmangani]

@JonnyBDev the 22.9.5 success seems curious as this should fail the same on either version if multiple nodes are involved. Could you confirm if that execution also had multiple nodes? Were there any other changes between the working and non working charts?

[JonnyBDev]

Well, I think I have to revert that statement - at least half. Had a setup running that night with around 500 restart from qryn and somehow it worked. Tried to replicate it like written above and it's not working.

[lmangani]

@JonnyBDev I appreciate the update! it confirms the theory of our lead developer @akvlad . We'll push a change to support this type of deployment in the coming days to get you going. This feedback about the variety of possible setups out there is useful to keep the current qryn version healthy until the next is released. I appreciate your time!

[JonnyBDev]

You're welcome. If you need anything from me to test hit me up.

[JonnyBDev]

fyi. Had the logs still open for my deployment from 30m ago (22.9.5 CH + 2.3.12 Qryn). Somehow the logs are giving me some more information and it seems, that it advanced in the setup - somehow? At least there are more successfully looking logs than earlier.

> [email protected] start
> node qryn.js

{"level":30,"time":1695815859258,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"Initializing DB... cloki"}
{"level":30,"time":1695815859312,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"xxh ready"}
{"level":30,"time":1695815859510,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"v1 -> v2"}
{"level":20,"time":1695815859510,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"CREATE TABLE IF NOT EXISTS samples_v3\n    (\n      fingerprint UInt64,\n      timestamp_ns Int64 CODEC(DoubleDelta),\n      value Float64 CODEC(Gorilla),\n      string String\n    ) ENGINE = MergeTree\n    PARTITION BY toStartOfDay(toDateTime(timestamp_ns / 1000000000))\n    ORDER BY (timestamp_ns)"}
{"level":30,"time":1695815859581,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"xxh ready"}
{"level":30,"time":1695815859613,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"v2 -> v3"}
{"level":20,"time":1695815859613,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"CREATE TABLE IF NOT EXISTS settings\n    (fingerprint UInt64, type String, name String, value String, inserted_at DateTime64(9, 'UTC'))\n    ENGINE = ReplacingMergeTree(inserted_at) ORDER BY fingerprint"}
{"level":30,"time":1695815859672,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"v3 -> v4"}
{"level":20,"time":1695815859672,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"DROP TABLE IF EXISTS samples_read"}
{"level":30,"time":1695815859751,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"v4 -> v5"}
{"level":20,"time":1695815859751,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"CREATE TABLE IF NOT EXISTS samples_read\n   (fingerprint UInt64,timestamp_ms Int64,value Float64,string String)\n   ENGINE=Merge('cloki', '^(samples|samples_v2)$')"}
{"level":30,"time":1695815859795,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"v5 -> v6"}
{"level":20,"time":1695815859796,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"CREATE VIEW IF NOT EXISTS samples_read_v2_1 AS \n    SELECT fingerprint, timestamp_ms * 1000000 as timestamp_ns, value, string FROM samples_read"}
{"level":50,"time":1695815859816,"pid":19,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","err":"Error: Request failed with status code 404\nResponse: [404] undefined\nError: Request failed with status code 404\n    at createError (/app/node_modules/axios/lib/core/createError.js:16:15)\n    at settle (/app/node_modules/axios/lib/core/settle.js:17:12)\n    at IncomingMessage.handleStreamEnd (/app/node_modules/axios/lib/adapters/http.js:269:11)\n    at IncomingMessage.emit (node:events:525:35)\n    at endReadableNT (node:internal/streams/readable:1358:12)\n    at processTicksAndRejections (node:internal/process/task_queues:83:21)","msg":"Error starting qryn"}
npm notice 
npm notice New major version of npm available! 8.19.4 -> 10.1.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.1.0>
npm notice Run `npm install -g [email protected]` to update!
npm notice

> [email protected] start
> node qryn.js

{"level":30,"time":1695816162236,"pid":18,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"Initializing DB... cloki"}
{"level":30,"time":1695816162264,"pid":18,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"xxh ready"}
{"level":30,"time":1695816162471,"pid":18,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"v1 -> v2"}
{"level":20,"time":1695816162471,"pid":18,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"CREATE TABLE IF NOT EXISTS samples_v3\n    (\n      fingerprint UInt64,\n      timestamp_ns Int64 CODEC(DoubleDelta),\n      value Float64 CODEC(Gorilla),\n      string String\n    ) ENGINE = MergeTree\n    PARTITION BY toStartOfDay(toDateTime(timestamp_ns / 1000000000))\n    ORDER BY (timestamp_ns)"}
{"level":30,"time":1695816162547,"pid":18,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"v2 -> v3"}
{"level":20,"time":1695816162548,"pid":18,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"CREATE TABLE IF NOT EXISTS settings\n    (fingerprint UInt64, type String, name String, value String, inserted_at DateTime64(9, 'UTC'))\n    ENGINE = ReplacingMergeTree(inserted_at) ORDER BY fingerprint"}
{"level":30,"time":1695816162575,"pid":18,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","msg":"xxh ready"}
{"level":50,"time":1695816162588,"pid":18,"hostname":"qryn-654c6c5db7-78tdc","name":"qryn","err":"Error: Request failed with status code 404\nResponse: [404] undefined\nError: Request failed with status code 404\n    at createError (/app/node_modules/axios/lib/core/createError.js:16:15)\n    at settle (/app/node_modules/axios/lib/core/settle.js:17:12)\n    at IncomingMessage.handleStreamEnd (/app/node_modules/axios/lib/adapters/http.js:269:11)\n    at IncomingMessage.emit (node:events:525:35)\n    at endReadableNT (node:internal/streams/readable:1358:12)\n    at processTicksAndRejections (node:internal/process/task_queues:83:21)","msg":"Error starting qryn"}
npm notice 
npm notice New major version of npm available! 8.19.4 -> 10.1.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.1.0>
npm notice Run `npm install -g [email protected]` to update!
npm notice

[lmangani]

@JonnyBDev Changes are ready. Would you be able to validate this branch? #350

[lmangani]

Thanks @JonnyBDev for the configuration and to @akvlad for taking the time to make this fix over the weekend!

[lmangani]

@JonnyBDev changes are now merged to master and released into qryn 2.4.x thanks for your assistance!