Comments (4)
Hello, there are 3 services in cluster version
vmstorage
- persistence storage (statefull)
vmselect
- read gateway (stateless)
vminsert
- write gateway (stateless)
vminsert knows about all storage nodes and uses consistent hashes to choose one particular node and writes to it (hash from metric + label in sorted order), if node doesn't exist at the moment it writes to next one
vmselect - reads from all nodes and merge the result, if one of them are not reachable it marks result as partial
here a bit more info https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster#cluster-availability
from victoriametrics.
A few words about replication additionally to the info provided by @tenmozes :
We didn't come up with reliable yet simple replication scheme on VictoriaMetrics level, which could provide data safety and high availability in the event of storage loss. So vmstorage
nodes rely on durable replicated disks such as Google Compute disks instead of implementing the replication itself.
The most straightforward approach for the replication on VictoriaMetrics level - just put N
copies of data to different vmstorage
nodes, where N
is replication factor - requires complex and fragile data reshuffling scheme in order to restore the required replication factor for the data stored on broken disks. The automatic reshuffling may hurt cluster availability and performance due to increased usage of network, disk and CPU resources. Additionally, the reshuffling may fail on edge cases such as temporary unavailability of the network between vmstorage
nodes. So we chose the simplest approach - to shift the data safety headache to durable disks.
It is possible to implement replication on the Prometheus level by running multiple VictoriaMetrics clusters in distinct availability zones and writing data in parallel to all these clusters. Then the data may be queried via promxy sitting in front of all the VictoriaMetrics clusters.
from victoriametrics.
Related issue: #118
from victoriametrics.
FYI, release v1.36.0 contains application-level replication support for cluster version of VictoriaMetrics. See more details about the replication here.
Closing this issue.
from victoriametrics.
Related Issues (20)
- vmauth per-user metrics can cause high memory usage in the long term HOT 2
- vmbackup cannot open a snapshot that it just created HOT 7
- How to drop selected metrics received at victoriametrics (single) from api/v1/write remoteWrite vmagents? HOT 4
- Scrape CRD's support by single-node VictoriaMetrics HOT 2
- victorialogs crash HOT 4
- ERROR: 422 on query with binary operation and keep_metric_names HOT 4
- vmui requests are not automatically quoted HOT 1
- Graphite queries not calculated correctly HOT 1
- how to set custom tag in filebeat output.elasticsearch _stream_fields HOT 2
- vmagent k8s target discovery is too slow HOT 4
- Data integrity problem after the vmstorage breaks down HOT 3
- victoria logsQL sort unexpected result. HOT 3
- How to search log as quickly as possible while writing log in client side HOT 2
- Attempts to configure VM for small memory footprint don't yield expected results HOT 2
- Maximum advised storage capacity for a vmstorage instance? HOT 1
- VictoriaMetrics Datadog APM/Trace Agent URL integration HOT 3
- MetricsQL: Document stalness marker differences between `default_rollup` and `last_over_time` HOT 2
- How to properly setup remoteWrite credentials in vmagent HOT 8
- VictoriaLogs UI sometimes loses log records in Firefox HOT 5
- Teach -httpAuth.username to read content of a file HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from victoriametrics.