Comments (4)
关于“创建反向复制任务”
- 正向和反向job不同时进行.
- 反向任务创建后, 暂时挂起, 等待正向任务finish
- 正向任务触发finish后, 记录正向任务源端Gtid为targetGtid, 此时正向任务目标端未必执行完毕
- 反向任务等待正向任务执行到targetGtid后, 反向任务启动
适用于切换复制方向的场景.
from dtle.
关于TwoWaySync
- 持续进行双向复制
- 代替之前双job方案
- 双job有潜在的无限循环问题
目前TwoWaySync会在反向任务使用正向任务的ColumnMap, 实为Bug.
from dtle.
方案1
对于反向任务, 交换 使用正向任务的ColumnMapFrom/To
(参数行为见文档).
考虑下列几个场景, 都能满足使用意图
选择列
正向
- src table (id, val1, val2)
- dst table (id, val2)
- ColumnMapFrom (id, val2)
- ColumnMapTo (id, val2)
反向
- from (id, val2)
- to (id, val2)
- 要求val1 可空或有默认值
选择列 + 重建自增id + 列重命名
正向
- src table (id auto_increment, val1, val2)
- dst table (id auto_increment, val1x, val2x)
- from (val1, val2)
- to (val1x, val2x)
reverse job
- from (val1x, val2x)
- to (val1, val2)
目标端多出列(有默认值)
正向
- src table (id, val1)
- dst table (id, val2 default 42, val1x)
- from (id, val1)
- to (id, val1x)
反向
- from (id, val1x)
- to (id, val1)
from dtle.
verified:
version:
9.9.9.9-master-6075dea
验证点:
- reversed job
-- 源端多出列
-- 目标端多出列
-- 列重命名
-- 列乱序 - TwoWaySync
-- 源端多出列
-- 目标端多出列
-- 列重命名
-- 列乱序
from dtle.
Related Issues (20)
- big tx stuck at `cleanGtidExecuted -> WaitForAllCommitted`
- 新增监控指标:当 dtle 与数据库的连接状态
- 提供运维接口,将指定gtid的以原本的形式从远端传输到目标端直接执行
- kafka输出中关于 datetime timestamp 和时区的问题 HOT 1
- dtle panic when src config does not match `DestType`
- MySQL-MySQL: src MySQL @@lctn=1 and dest MySQL @@lctn=0, `ALTER TABLE RENAME` did not match expectations
- retry TX on a deadlock HOT 3
- api: ListColumns bad order
- handle low wait_timeout. HOT 2
- schema.table renaming vs reversed job
- refactor ApplyEventQueries
- Stuck in MtsManager.WaitForAllCommitted
- Remaining `nomad logmon` process HOT 1
- 0 appears in gtid
- mysql to mysql最新数据未同步 HOT 7
- 同一个库下不同表进行同步的时候,如果参数SkipCreateDbTable未配置,会由于MDL锁hang住整个库 HOT 3
- 调整job配置文件后(新增表同步),如何加载,同时之前的配置不受影响 HOT 1
- handle query_event COMMIT HOT 2
- [ERROR] read dir failed: dataDir=/data/dtle/var/lib/nomad
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dtle.