Comments (5)
Hey @zehweh
Could you try this
https://gist.github.com/jawabuu/da27bb6bbb562d9caa5d1692955c892a
from vitess-operator.
Thanks @jawabuu for your quick response.
Unfortunately, I get connection errors again.
You can see the logs here: https://gist.github.com/zehweh/c55ebe1aedcee5acd498e954baa1cd27
I noticed, that vttablet complains about mysql running in read-only mode... This looks suspicious.
Some additional information I forgot to add:
- kubernetes version 1.22.4
- flannel for networking
from vitess-operator.
@zehweh Are you on #vitess slack channel?
from vitess-operator.
I'd like to see your
kubectl get po
after running my gist
from vitess-operator.
I found out what was causing the issues:
1: The resource limits were set too low in my VitessCluster manifest which resulted in occasional OOM kills of the mysql container and the logs you see above.
2: I'm working with a master-master replication on the external database (unmanaged tablet). The master I'm connecting to has multiple sets of GTIDs:
mysql> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
[...]
Executed_Gtid_Set: 01af65be-d03d-11ec-a14f-0cc47a6af5da:1-28538331:544939690-544939696,
d048674a-e588-11ec-80cd-0cc47a6c65c8:1-65287695
[...]
Because of this, the MoveTables workflow wouldn't start:
❯ vtctlclient MoveTables --source testdb -all Create vitesstest.testremote2vitess
Waiting for workflow to start:
0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... 0% ... E0606 09:56:06.404528 5131 main.go:67] E0606 09:56:06.404251 vtctl.go:2582] workflow did not start within 30s
MoveTables Error: rpc error: code = Unknown desc = workflow did not start within 30s
E0606 09:56:06.406961 5131 main.go:76] remote error: rpc error: code = Unknown desc = workflow did not start within 30s
After switching to a master-slave setup and resetting the GTIDs, I was able to successfully start the VReplication.
Thanks @jawabuu for your help!
from vitess-operator.
Related Issues (20)
- Update Documentation to include vtadmin
- Vitess Operator API Reference is stale
- test
- Support major MySQL version upgrade in the operator
- Missing file from documentation in AWS QuickStart
- AWS IAM Role for Service Account support HOT 2
- Vtadmin volume mount not effected
- Ready for production in 2023? HOT 11
- vitess operator doesn't accept integer values for vttablet extraFlags
- [Feature request] Allow for `Service` name overrides HOT 1
- support kubernetes v1.25 for recommended stable vitess versions HOT 6
- Vitess Operator spawns a VTOrc even for shards that have only external tablets HOT 1
- Feature Request: Allow more than one VitessShardTabletPool with a certain tablet type in a given cell
- Unable to install vitess on kubernetes behind proxy HOT 1
- Feature request: support controlled mysqld version upgrade with rollback HOT 1
- Feature Request: configure operator to communicate with Vitess components using TLS
- [Feature Request] Allow enabling additional mysqld_exporter collectors
- vtctldclient backup not working with defined VitessBackupStorages in cluster HOT 4
- `vtop-rollout` tooling for safer rollout of new Vitess versions in Kubernetes
- Add custom annotations to PVC
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vitess-operator.