Comments (5)
If the gh-ost process dies, we could theoretically resume the operation, as long as we know where to resume in the table copy process, know where to resume in the binary log for incoming changes, and have the binary logs to resume from.
omg.
So, I've been thinking about this: we only need to know where to resume in the table-copy process, actually. It turns out, and I need to put this in detailed writing, that replaying the RBR is idempotent!! Which means we can just replay it from some point in the past (but of course we must never skip entries).
But, I would suggest this is still way in the future.
from gh-ost.
The quick answer is to select max(last_update) from _whatever_ghc
. An active gh-ost
operation will routinely update that table (within the second) even while throttling. Thus, if max(last_update)
is, say, a minute ago (are you checking on master? That's definite. Are you checking on replica? Make sure lag is OK) indicates the migration is dead.
from gh-ost.
I think abandoned and dead are two different states. With trigger-based migration tools, if the triggers are gone, we have to give up and start over, but that's not the case here. If the gh-ost process dies, we could theoretically resume the operation, as long as we know where to resume in the table copy process, know where to resume in the binary log for incoming changes, and have the binary logs to resume from.
from gh-ost.
The official query to see if a migration is running is:
select last_update from _tbl_ghc where hint='heartbeat';
(to get last known activity), or
select last_update > now() - interval 1 minute as is_alive from _tbl_ghc where hint='heartbeat';
for a heuristic "if it hasn't been updated in the last minute it must be dead"
cc @tomkrouper
from gh-ost.
The above sounds great. Having the ability to resume would be a great feature.
Now that binlog events are applied in a transaction, it should be possible to confidently store and read the last-processed binlog position. The same could be done with the table copy process, if the statements are also wrapped in a transaction.
from gh-ost.
Related Issues (20)
- "attempt-instant-ddl" should support lock_wait_timeout HOT 1
- max-lag-millis parameter description is misleading when running on master HOT 3
- When will the throttled state be triggered HOT 1
- Does gh-ost support distributed databases? HOT 2
- binlog syncer closes unexpectedly
- PR HOT 2
- Feature request: support setting the `wait_timeout` of `gh-ost` HOT 1
- cut-over locks not released when gh-ost pauses mid-cut-over
- Aliii HOT 1
- FATAL invalid sequence 8 != 1
- Cannot log in to mysql DB by ghost user
- collect table statistics before renaming table
- Clarification on `aurora_enable_repl_bin_log_filtering` in Aurora HOT 6
- RDS/Aurora Docs clarifications.
- skip delete events of changelog table in binlog listener
- [Suggestion] Apply simple validation to alter command
- ERROR Error 1146: Table '<db>._temp_no_ghc' doesn't exist HOT 2
- Add atomic cut over function that renames locked tables in the same session
- UCS2 to UTF8MB4 Encoding error
- Handling Slow Queries Without Indexes in gh-ost
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gh-ost.