Comments (3)
For instance, if the backend mysql is unavailable can Orchestrator still do failovers based on some cached state? Can it failover it's own backend (hence dogfood)?
It cannot. You have three options available:
- running a circular master-master setup, with your own failover/routing mechanism. We do that. Generally master-master is undesired and to be avoided.
orchestrator
guarantees no conflict of writes on a master-master setup and is safe to use. We use haproxy checks to routeorchestrator
mysql traffic to the active/available master. - running Galera/xtradb cluster/InnoDB cluster
- wait; we're working on raft consensus for
orchestrator
, where nodes would be independent of each other yet collaborating, each with their own backend store. This would be eithermysql
orsqlite
.
from orchestrator.
I have an orchestrator backend with slaves and have tested "scheduled" failover to a slave. This is usually to achieve MySQL upgrades or change the backend due to hardware issues etc. This type of failover works fine though it's not quite what you're asking about. Do not expect orchestrator to be able recover from its own backend failure.
However, if you can arrange, on orchestrator backend failure, for the following to be true orchestrator will recover and that is probably all you care about:
- all connections to the old master are killed
- the new "master" is writeable (obviously)
- the name that orchestrator uses to talk to its backend (vip name) does not change but points to the new host
- the data that was in the old host is present in the new one
Under these conditions orchestrator will detect the backend failure but will recover from it to no ill effect.
The main reason that orchestrator is unable to recover itself is that currently a lot of state it may use for recovery is stored in the database with no "caching layer" running on the active node. That is not available at the moment the orchestrator backend fails.
I personally, for performance reasons, would like to see the state being managed by the active node but that requires a very significant amount of code change to achieve that. Given the complexity of all the situations that orchestrator can handle such a change if it were ever to happen needs to be made with a considerable amount of care.
from orchestrator.
Thanks guys
from orchestrator.
Related Issues (20)
- Project dead? HOT 3
- DeadIntermediateMaster auto Recovery HOT 1
- Failure from hooks are not reflected on UI HOT 1
- Automated recovery invalid HOT 5
- Should set old master's read_only to false under DeadCoMaster failure
- Report IntermediateMaster errors under CoMaster deployment
- Set the old primary host read_only=0 if failover/switch over failed.
- Whether ORC provides APIs that enable/disable recovery for specific clusters? HOT 1
- Using Azure AD Authentication
- How to set autocommit for orchestrtor detection thread
- How do you perform system tests? HOT 1
- Improve logging of problems during db connection
- Orchestrator able to display topology but still reports as failure
- After the Mysql domain name connection is configured。If the orchestrator topology displays both ip addresses and domain names of the same backup database, how can I fix this problem
- Discovery doesn't discover in extremely simple setup HOT 4
- Does orchestrator's website :3000 has login account?
- Does anyone know why updates stopped? HOT 2
- add windows version
- Ptrhon API Token
- How to login with username and password through API ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from orchestrator.