planetscale / vitess-operator Goto Github PK

View Code? Open in Web Editor NEW

283.0 36.0 69.0 3.04 MB

Kubernetes Operator for Vitess

License: Apache License 2.0

Makefile 0.25% Dockerfile 0.04% Shell 4.36% Go 95.35%

vitess kubernetes operator

vitess-operator's People

Contributors

Stargazers

Watchers

Forkers

laik fortm henders isgasho carsonoid luisfmcalado vkruoso pinzhang wanyijie basecamp qu823110460 bnu0 inolddays yiliaofan brandt lana-banana gwonsoolee joshuaandrew dkhenry artemvovk jayseek zxshinxz jmoldow fzyzcjy laashub-soa michaellzc ericnjin streadway huangzhiyong chapsuk pinkbluersglobal obigtech infomaniak verolop shaardie stansocialcloud debrajmaity tenlifoxx vkovacik michaellee8 irbouh tokikanno joneug hbjydev dysosmus nianfei97 lee-joseph 480 de-metavision jcarrig ggalakhov farodin91 stankevich walkamongus bidutech potterhe yoheimuta cectc foreverup0110 mrfabio ivan-section-io jhampson-dbre frouioui

vitess-operator's Issues

Review gateway_implementation flag

https://github.com/planetscale/vitess-operator/blob/master/pkg/operator/vtgate/deployment.go#L46

sets gateway_implementation=discoverygateway.

We should consider changing the default to tabletgateway which will be the new default vitess value in the upcoming 7.0 release.

Set up CI/CD

We need to choose and set up infra for running tests and pushing images.

vitessOrchestrator complete support for command: and args: key words

I realize vtorc is beta software and this might be a feature request rather than a bug.

Using command: and args: keywords to copy a file in the vtorc pod before vtorc runs doesn't work. However, command: and args: work under initContainer. Below is an example that generates an initContainer that runs "ls -l /" successfully because the results are logged under the pod's "setup" container.

spec:
  vitessDashboard:
    replicas: 1
    resources:
      limits:
        memory: 512Mi
      requests:
        cpu: 75m
        memory: 256Mi

next section:

keyspaces:
- name: demoDB
    vitessOrchestrator:
      extraLabels:
        planetscale.com/component: vtorc
      configSecret:
        name: orc-config-80x
        key: orc_config.json
     initContainer:
        name: setup
        command: ["/bin/sh", "-c"]
        args: ["ls -l /"]
     extraVolumes:
      - name: "orc-configs"
         configMap:
          name: "orc-config"
      - name: "what"
        emptyDir: {}

Based on how initContainer works, the expected results of the below config should be the results of "ls -l /" in the vtorc container log. The actual results are that the results of "ls -l /" are NOT logged. Only Vtorc is logging. I assume that means command: and args: are not being honored at the vitessOrchestrator level. The config I tried is below:

keyspaces:
- name: demoDB
    vitessOrchestrator:
      extraLabels:
        planetscale.com/component: vtorc
      configSecret:
        name: orc-config-80x
        key: orc_config.json
      command: ["/bin/sh", "-c"]
      args: ["ls -l /"]
      extraVolumeMounts:
        - name: "orc-configs"
          mountPath: "/pickup"
        - name: "what"
          mountPath: /conf
      extraVolumeMounts:
       - name: "orc-configs"
          mountPath: "/pickup"
       - name: "what"
          mountPath: /conf
     extraVolumes:
      - name: "orc-configs"
         configMap:
          name: "orc-config"
      - name: "what"
        emptyDir: {}

Allow custom vtgate/vtctld Service names

Currently, the vtgate Service name has a unique, deterministic hash suffix. This was part of a strategy to avoid collisions in the multi-tenant environment for which our operator was initially designed. For those running the operator in their own, controlled environments, the hash suffix makes it unnecessarily difficult to configure apps to connect to Vitess.

We should make the hash suffix optional for at least the main vtgate/vtctld Services, and possibly also the per-cell vtgate Services. Ideally we would also make the suffix disabled by default, but unfortunately that would be a backwards-incompatible change for those of us already running the operator in production.

To avoid over-constraining the user's choice, we could support this by adding an optional field to override the default Service name to a custom value.

docs/api: Links for status map types are broken

In the generated docs, links for map types like map[string]*planetscale.dev/vitess-operator/pkg/apis/planetscale/v2.VitessClusterCellStatus are broken. We probably need to contribute a patch to the gen-crd-api-reference-docs tool we use, to teach it to handle maps of custom types, which is somewhat non-standard (though we only do it in status types).

Publish docs website

We should get some intro docs started, and publish the generated CRD API reference docs. Probably on Netlify?

Unable to delete old shards after reshard

On minikube (kubernetes v1.16.2) + vitess-9.0.0-rc1 + vitess-operator:latest (DIGEST:sha256:63f5b544f960288da5cd799b7e07894976c680029b1087636a99ba18ec215b08)

All is well but the leftover old shards. The new operator will not delete the old shards after resharding (after DropSources and apply the new YAML without the old partitioning).

Unable to add "extraLabels" to vbs-subcontroller

In opposite to other components of Vitess, it is not currently possible to set any extra labels to the pod of vbs-subcontroller.

I looked into the VitessCluster spec's documentation and did a quick look in the code, and it seems that is missing the updateLabels implementation for this specific pod.

pod has unbound immediate PersistentVolumeClaims

So how to create PV for vitess operator, the document Prerequisites not say that..

Vitess backup restore fails using vitess-operator on kubernetes

Overview of the Issue

Unable to see backup restored when recreating a deleted kube cluster

Reproduction Steps

Steps to reproduce this issue, example:

Deploy the following cluster yaml on eks:

# Version: 20200601
apiVersion: planetscale.com/v2
kind: VitessCluster
metadata:
  name: test
spec:
  backup:
    engine: xtrabackup
    locations:
    - s3:
        bucket: test-vitess
        region: us-east-1
        keyPrefix: testcluster
  cells:
  - name: useast1
    gateway:
      authentication:
        static:
          secret:
            name: test-cluster-config
            key: users.json
      replicas: 2
      resources:
        requests:
          cpu: 100m
          memory: 256Mi
        limits:
          memory: 256Mi
      secureTransport:
        tls:
          certSecret:
            name: test-cluster-config
            key: cert.pem
          keySecret:
            name: test-cluster-config
            key: key.pem
  vitessDashboard:
    cells:
    - zone1
    extraFlags:
      security_policy: read-only
      backup_storage_implementation: s3
    replicas: 1
    resources:
      limits:
        memory: 128Mi
      requests:
        cpu: 100m
        memory: 128Mi
  keyspaces:
  - name: commerce
    turndownPolicy: Immediate
    partitionings:
    - equal:
        parts: 1
        shardTemplate:
          databaseInitScriptSecret:
            name: test-cluster-config
            key: init_db.sql
          replication:
            enforceSemiSync: false
          tabletPools:
          - cell: useast1
            type: replica
            replicas: 2
            vttablet:
              extraFlags:
                db_charset: utf8mb4
              resources:
                requests:
                  cpu: 100m
                  memory: 256Mi
            mysqld:
              resources:
                requests:
                  cpu: 100m
                  memory: 256Mi
            dataVolumeClaimTemplate:
              accessModes: ["ReadWriteOnce"]
              resources:
                requests:
                  storage: 10Gi
  updateStrategy:
    type: Immediate
---
apiVersion: v1
kind: Secret
metadata:
  name: test-cluster-config
type: Opaque
stringData:
  init_db.sql: |
    # This file is executed immediately after mysql_install_db,
    # to initialize a fresh data directory.

    ###############################################################################
    # Equivalent of mysql_secure_installation
    ###############################################################################

    # Changes during the init db should not make it to the binlog.
    # They could potentially create errant transactions on replicas.
    SET sql_log_bin = 0;
    # Remove anonymous users.
    DELETE FROM mysql.user WHERE User = '';

    # Disable remote root access (only allow UNIX socket).
    DELETE FROM mysql.user WHERE User = 'root' AND Host != 'localhost';

    # Remove test database.
    DROP DATABASE IF EXISTS test;

    ###############################################################################
    # Vitess defaults
    ###############################################################################

    # Vitess-internal database.
    CREATE DATABASE IF NOT EXISTS _vt;
    # Note that definitions of local_metadata and shard_metadata should be the same
    # as in production which is defined in go/vt/mysqlctl/metadata_tables.go.
    CREATE TABLE IF NOT EXISTS _vt.local_metadata (
      name VARCHAR(255) NOT NULL,
      value VARCHAR(255) NOT NULL,
      db_name VARBINARY(255) NOT NULL,
      PRIMARY KEY (db_name, name)
      ) ENGINE=InnoDB;
    CREATE TABLE IF NOT EXISTS _vt.shard_metadata (
      name VARCHAR(255) NOT NULL,
      value MEDIUMBLOB NOT NULL,
      db_name VARBINARY(255) NOT NULL,
      PRIMARY KEY (db_name, name)
      ) ENGINE=InnoDB;

    # Admin user with all privileges.
    CREATE USER 'vt_dba'@'localhost';
    GRANT ALL ON *.* TO 'vt_dba'@'localhost';
    GRANT GRANT OPTION ON *.* TO 'vt_dba'@'localhost';

    # User for app traffic, with global read-write access.
    CREATE USER 'vt_app'@'localhost';
    GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, FILE,
      REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
      LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW,
      SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER
      ON *.* TO 'vt_app'@'localhost';

    # User for app debug traffic, with global read access.
    CREATE USER 'vt_appdebug'@'localhost';
    GRANT SELECT, SHOW DATABASES, PROCESS ON *.* TO 'vt_appdebug'@'localhost';

    # User for administrative operations that need to be executed as non-SUPER.
    # Same permissions as vt_app here.
    CREATE USER 'vt_allprivs'@'localhost';
    GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, FILE,
      REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
      LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW,
      SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER
      ON *.* TO 'vt_allprivs'@'localhost';

    # User for slave replication connections.
    # TODO: Should we set a password on this since it allows remote connections?
    CREATE USER 'vt_repl'@'%';
    GRANT REPLICATION SLAVE ON *.* TO 'vt_repl'@'%';

    # User for Vitess filtered replication (binlog player).
    # Same permissions as vt_app.
    CREATE USER 'vt_filtered'@'localhost';
    GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, FILE,
      REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
      LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW,
      SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER
      ON *.* TO 'vt_filtered'@'localhost';

    # User for Orchestrator (https://github.com/github/orchestrator).
    # TODO: Reenable when the password is randomly generated.
    #CREATE USER 'orc_client_user'@'%' IDENTIFIED BY 'orc_client_user_password';
    #GRANT SUPER, PROCESS, REPLICATION SLAVE, RELOAD
    #  ON *.* TO 'orc_client_user'@'%';
    #GRANT SELECT
    #  ON _vt.* TO 'orc_client_user'@'%';

    FLUSH PRIVILEGES;

    RESET SLAVE ALL;
    RESET MASTER;

Deploy the following schema:
https://github.com/vitessio/vitess/blob/main/examples/operator/create_commerce_schema.sql
Delete the existing cluster after checking backup files created in s3 bucket
Redeploy the cluster
Inspecting the logs show the following error:

I0825 20:18:47.743816 1 s3.go:261] ListBackups: [s3] dir: commerce/-, bucket: oxygen-vitess I0825 20:18:50.978970 1 s3.go:273] objName: 0xc000dc0000 time="2021-08-25T20:18:51Z" level=info msg="Reconciling VitessBackupStorage" namespace=default subcontroller=VitessBackupStorage vitessbackupstorage=test-e2a3e68d I0825 20:18:51.124050 1 s3.go:261] ListBackups: [s3] dir: commerce/-, bucket: oxygen-vitess I0825 20:18:54.306436 1 s3.go:273] objName: 0xc000a9c4a0 {"level":"error","ts":1629922734.3424866,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"vitessbackupstorage-subcontroller","request":"default/test-e2a3e68d","error":"Operation cannot be fulfilled on vitessbackupstorages.planetscale.com \"test-e2a3e68d\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:258\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:88"} time="2021-08-25T20:18:55Z" level=info msg="Reconciling VitessBackupStorage" namespace=default subcontroller=VitessBackupStorage vitessbackupstorage=test-e2a3e68d I0825 20:18:55.343225 1 s3.go:261] ListBackups: [s3] dir: commerce/-, bucket: oxygen-vitess I0825 20:18:58.524614 1 s3.go:273] objName: 0xc000ab3bd0

Inspcted the db and i see no tables restored

Is it production ready?

Hi thanks for the lib - I wonder whether ready for production?

Hot loop when using AlwaysPullImages admission controller

We've received a report of a hot loop that occurs when these conditions are met:

The vttablet image is set to use the :latest tag.
The updateStrategy is set to type: Immediate.
The imagePullPolicies field is left unset.

I will try to reproduce this. In the meantime, a potential workaround is to explicitly fill in imagePullPolicies in the VitessCluster spec, making sure to set a non-empty value for any Vitess component for which you are using a :latest image (vtctld, vtgate, vttablet, vtbackup, mysqld, mysqldExporter).

vttablet: enable_replication_reporter should be configurable

Right now in vttablet, enable_replication_reporter defaults to yes in externalDatastoreFlags. This causes issues when we add a rdonly tablet backed by RDS - Vitess tries to check replication on a tablet that's not running it.

Creating a Keyspace with >1 parts does not update VSchema

The deploy/example.yml has 2 parts in the keyspace it's creating but the VSchema is just created as an empty { }.

I think this is causing problems as I was seeing errors complaining that tables I created in that keyspace weren't sharded.
I think during instantiation of the keyspace it should detect parts > 1 and update the schema with the minimum:

{
  "sharded": true
}

Document rolling update process

We need to put up a doc on how rolling updates work with the operator. In the meantime, here is at least a dump of some discussion from Slack:

in general, all edits should be made only to the top-level vt. the others (vtc/vtk/vts) should be treated as read-only since they are actively managed by the top-level vt. this is similar to the pattern used in the core k8s Deployment API - one should not directly edit the underlying ReplicaSets it creates.

to create a new keyspace, you can just edit the vt spec to add it. it should get deployed immediately

however, edits to an already-deployed keyspace will be intentionally held back until the next rollout.

The rollout can be managed with annotations as follows:

whenever there are changes held back, the pending changes will be listed in an annotation on the object called rollout.planetscale.com/scheduled

so for example, when you edit an already-deployed keyspace in the vt, that annotation will appear on the corresponding vtk

the way to say "ok; go ahead and release those scheduled changes on this object" is to add an annotation called rollout.planetscale.com/released with any string value (even empty string). once this annotation is present, the scheduled changes will be applied

this needs to be done recursively: after releasing changes to the vtk, the vtk will schedule changes on its vts children which then need to be released; then each vts will schedule changes on the tablet Pods that will need to be released. the idea is that the operator handles actually doing the updates, but the process of deciding which things to update in which order and with what concurrency is left to an orthogonal system

btw, when the rollout.planetscale.com/released annotation is applied to a vttablet Pod with pending changes, the operator will first ensure that tablet is not a master for its shard, doing a PlannedReparentShard automatically if necessary, before recreating the Pod to update its spec

unless there are no other master-eligible replicas provisioned, in which case there's nothing that can be done except restarting the master. in that case, you must manually delete the master Pod once the rollout.planetscale.com/scheduled annotation appears on the Pod. the operator will not delete a master Pod.

how to add tolerations

I added tolerations to dashboard but I found that there is no option to add toleration to vtgate, etcd etc in vitessCluster CRD.
Is there any other way to add tolerations to these pods?

Add setting to disable replication repair

When running something like Orchestrator alongside Vitess, it's best to disable the replication repair feature of Vitess Operator so only one agent tries to fix the master. We would need to add a new field under replication to support this, since replication repair is currently always on.

Pending backup initialization pods deadlock clusters when using the `toplvm` dynamic provisioner

Overview

New cluster tablets do not start serving when backup initialization pods cannot start to take the initial backup.

Vttablet pods are scheduled on nodes with a local data volume template and are waiting for an initial backup.

Vtbackup init pods are scheduled on nodes and Pending in the pre-bind volume phase. The storage provisioner cannot create volumes for the scheduled node, leaving the the vtbackup pods Pending.

This affects clusters that use dynamic local storage provisioned data volumes on a cluster with volumes available only on some nodes. This is experienced with topolvm where some nodes in the cluster do not have an LVM Physical Volume for the provisioner.

Analysis

The vtbackup init pods are reconciled before their data volume PVC. There is more than one condition that could prevent the PVC from provisioning a volume in the pre-bind phase, and we have observed "Pre-bind failed. No topology key found".

This differs from vttablet, where the pod is reconciled after the data volume PVC. This gives a chance for the external provisioner to influence the scheduling based on the PVC claim and node capacity. Tablets are scheduled successfully.

Pods without PVCs are considered by topolvm but not supported.

Recommendation

Reconcile PVCs before Pods, in the same order for vtbackup and vttablet pods, when they share a data volume template.

Workarounds

Delete the pods to be recreated after the PVC exists.
Create the cluster with initializeBackup: false, then run vtctl BackupShard manually.
Ensure every node in the cluster has a provisioner and the capacity for scheduled pod volumes.
Enable Storage Capacity Tracking in Kubernetes >= 1.19 and Topolvm >= 0.8.4 so that Pods scheduled to nodes with unschedulable PVCs can get rescheduled to new nodes.

Reproduction Steps (topolvm + vitess operator)

Setup a Vitess CRD+operator in a Kubernetes cluster with at least 2 nodes.
Deploy topolvm and run external provisioners on example-node1 but not on example-node2.
Create a "local" storage class, using the WaitForFirstConsumer volume binding mode and the topolvm provisioner.
Create a cluster with:

apiVersion: planetscale.com/v2
kind: VitessCluster
metadata:
  name: example
  namespace: vt-example
spec:
  backup:
    engine: xtrabackup
    locations:
    - name: gcs
      # Type doesn't matter for this issue
      gcs:
        authSecret:
          key: key.json
          name: backup-auth
        bucket: vitess-operator-backup-init-topolvm-issue
  keyspaces:
  - name: keyspace1
    partitionings:
    - equal:
        parts: 2
        shardTemplate:
          databaseInitScriptSecret:
            key: init_db.sql
            name: init-config
          replication:
            initializeBackup: true
            initializeMaster: true
            recoverRestartedMaster: true
          tabletPools:
          - cell: cell1
            type: replica
            replicas: 2
            backupLocationName: gcs
            dataVolumeClaimTemplate:
              accessModes:
              - ReadWriteOnce
              resources:
                requests:
                  storage: 10Mi
              storageClassName: local
            mysqld:
              resources: {}
            vttablet:
              resources: {}
  cells:
  - name: cell1
    gateway:
      replicas: 1
      resources: {}
      authentication:
        static:
          secret:
            name: vtgate
            key: users.json

With the following secrets/keys:

backup-auth/key.json - GCP service account credentials - replace with your own.
init-config/init_db.sql - default database initialization script.
vtgate/users.json - default vtgate static authorization file.

Versions

kubernetes: 1.19.7
vitess: 10.0.1
vitess-operator: 2.4.0
vitess-operator operator-sdk: v0.16.0

topolvm: quay.io/topolvm/topolvm-with-sidecar:0.8.3
csi: k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.1.0

Observations

Schedules the following pods:

NAME                                                     READY   STATUS             RESTARTS   AGE
example-cell1-vtctld-30b77a3b-6cd48c9cb4-4ckgx           1/1     Running            6          34m
example-cell1-vtgate-5878cb1e-69f75b9546-9gmml           0/1     CrashLoopBackOff   5          8m4s
example-etcd-faf13de3-1                                  1/1     Running            5          34m
example-etcd-faf13de3-2                                  1/1     Running            5          34m
example-etcd-faf13de3-3                                  1/1     Running            5          34m
example-gcs-8bbc7eb3-vitessbackupstorage-subcontroller   1/1     Running            0          16m
example-keyspace1-80-x-vtbackup-init-cb2f81aa            0/1     Pending            0          33m
example-keyspace1-x-80-vtbackup-init-a9e8a6ee            0/1     Pending            0          33m
example-vttablet-cell1-0594325533-a04ab155               3/3     Running            1          34m
example-vttablet-cell1-0840121086-7f5eec15               3/3     Running            1          34m
example-vttablet-cell1-1199451902-7021fcd8               3/3     Running            1          34m
example-vttablet-cell1-4001313850-18fe025f               3/3     Running            1          34m
vitess-operator-6d9b44c6f4-zw585                         1/1     Running            0          37m

With the following PVCs:

NAME                                            STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
example-etcd-faf13de3-1                         Bound     pvc-107bdce3-64b9-49ac-a22f-4f7297a09958   1Gi        RWO            standard       36m
example-etcd-faf13de3-2                         Bound     pvc-402042a9-34b5-4762-ac0b-d771bc89b335   1Gi        RWO            standard       36m
example-etcd-faf13de3-3                         Bound     pvc-2d09f5a2-f393-4ce5-81b6-d04f5c01b881   1Gi        RWO            standard       36m
example-keyspace1-80-x-vtbackup-init-cb2f81aa   Pending                                                                        local          36m
example-keyspace1-x-80-vtbackup-init-a9e8a6ee   Pending                                                                        local          36m
example-vttablet-cell1-0594325533-a04ab155      Bound     pvc-fa502fb0-7dc9-42ee-a9fb-e853dcca164f   1Gi        RWO            local          36m
example-vttablet-cell1-0840121086-7f5eec15      Bound     pvc-e016d8fd-7475-4847-93aa-b40fc7adc509   1Gi        RWO            local          36m
example-vttablet-cell1-1199451902-7021fcd8      Bound     pvc-301e5e5d-695a-484c-ace3-1037124be9bc   1Gi        RWO            local          36m
example-vttablet-cell1-4001313850-18fe025f      Bound     pvc-f9b2fdbb-02e0-4560-b6cc-e5f2e883305c   1Gi        RWO            local          36m

Describing one of the pending backup init PVCs:

Name:          example-keyspace1-80-x-vtbackup-init-cb2f81aa
Namespace:     vt-example
StorageClass:  local
Status:        Pending
Volume:        
Labels:        backup.planetscale.com/location=gcs
               backup.planetscale.com/type=init
               planetscale.com/cluster=example
               planetscale.com/component=vtbackup
               planetscale.com/keyspace=keyspace1
               planetscale.com/shard=80-x
Annotations:   volume.beta.kubernetes.io/storage-provisioner: topolvm.cybozu.com
               volume.kubernetes.io/selected-node: example-node2
Finalizers:    [kubernetes.io/pvc-protection topolvm.cybozu.com/pvc]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Mounted By:    example-keyspace1-80-x-vtbackup-init-cb2f81aa
Events:
  Type     Reason                Age                    From                                                                                         Message
  ----     ------                ----                   ----                                                                                         -------
  Normal   WaitForPodScheduled   38m                    persistentvolume-controller                                                                  waiting for pod example-keyspace1-80-x-vtbackup-init-cb2f81aa to be scheduled
  Warning  ProvisioningFailed    16m (x14 over 38m)     topolvm.cybozu.com_topolvm-controller-5b96bbfc54-4zp58_2145e7a3-1e48-4305-a167-63da9774bbda  failed to provision volume with StorageClass "local": error generating accessibility requirements: no topology key found on CSINode example-node2
  Normal   ExternalProvisioning  3m30s (x142 over 38m)  persistentvolume-controller                                                                  waiting for a volume to be created, either by external provisioner "topolvm.cybozu.com" or manually created by system administrator
  Normal   Provisioning          95s (x18 over 38m)     topolvm.cybozu.com_topolvm-controller-5b96bbfc54-4zp58_2145e7a3-1e48-4305-a167-63da9774bbda  External provisioner is provisioning volume for claim "vt-example/example-keyspace1-80-x-vtbackup-init-cb2f81aa"

The CSINode for example-node2 does not have topolvm drivers - the topolvm provisioner is not running on this node.

Name:               example-node2
Labels:             <none>
Annotations:        <none>
CreationTimestamp:  Tue, 27 Apr 2021 10:27:17 +0200
Spec:
  Drivers:
    rook-ceph.rbd.csi.ceph.com:
      Node ID:  example-node2
Events:         <none>

Describing a successfully bound PVC for a vttablet succeeded to provision a PV:

Name:          example-vttablet-cell1-0594325533-a04ab155
Namespace:     vt-example
StorageClass:  local
Status:        Bound
Volume:        pvc-fa502fb0-7dc9-42ee-a9fb-e853dcca164f
Labels:        planetscale.com/cell=cell1
               planetscale.com/cluster=example
               planetscale.com/component=vttablet
               planetscale.com/keyspace=keyspace1
               planetscale.com/shard=80-x
               planetscale.com/tablet-index=1
               planetscale.com/tablet-type=replica
               planetscale.com/tablet-uid=594325533
Annotations:   pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: topolvm.cybozu.com
               volume.kubernetes.io/selected-node: example-node1
Finalizers:    [kubernetes.io/pvc-protection topolvm.cybozu.com/pvc]
Capacity:      1Gi
Access Modes:  RWO
VolumeMode:    Filesystem
Mounted By:    example-vttablet-cell1-0594325533-a04ab155
Events:
  Type    Reason                 Age   From                                                                                         Message
  ----    ------                 ----  ----                                                                                         -------
  Normal  WaitForFirstConsumer   45m   persistentvolume-controller                                                                  waiting for first consumer to be created before binding
  Normal  Provisioning           45m   topolvm.cybozu.com_topolvm-controller-5b96bbfc54-4zp58_2145e7a3-1e48-4305-a167-63da9774bbda  External provisioner is provisioning volume for claim "vt-example/example-vttablet-cell1-0594325533-a04ab155"
  Normal  ExternalProvisioning   45m   persistentvolume-controller                                                                  waiting for a volume to be created, either by external provisioner "topolvm.cybozu.com" or manually created by system administrator
  Normal  ProvisioningSucceeded  45m   topolvm.cybozu.com_topolvm-controller-5b96bbfc54-4zp58_2145e7a3-1e48-4305-a167-63da9774bbda  Successfully provisioned volume pvc-fa502fb0-7dc9-42ee-a9fb-e853dcca164f

The CSINode example-node1 for the vttablet PVC has the topolvm driver.

Name:               example-node1
Labels:             <none>
Annotations:        <none>
CreationTimestamp:  Thu, 19 Aug 2021 18:12:16 +0200
Spec:
  Drivers:
    topolvm.cybozu.com:
      Node ID:        example-node1
      Topology Keys:  [topology.topolvm.cybozu.com/node]
    rook-ceph.rbd.csi.ceph.com:
      Node ID:  example-node1
Events:         <none>

The vttablet logs are deadlocked for a backup before serving:

I0913 14:21:42.733230       1 restore.go:177] No backup found. Waiting 10s (from -wait_for_backup_interval flag) to check again.

Migrate unmanaged tablet to Vitess keyspace

Hi,

for the last couple of days I've been trying to migrate data from an unmanaged tablet to a vitess cluster in kubernetes using the vitess orchestrator.

Setting up a replicated, unsharded test keyspace together with an unmanaged tablet works like a charm but when it comes to migrating the data, it fails and I see a lot of connection errors:

# date
# Tue 15 Mar 2022 08:14:56 AM UTC
# vtctlclient MoveTables -source testdb -all Create vitesstest.testremote2vitess

MoveTables Error: rpc error: code = Unknown desc = TabletManager.ApplySchema on zone1-1728353218 error: EOF (errno 2013) (sqlstate HY000) during query: SELECT
			table_name AS table_name,
			ordinal_position AS ordinal_position,
			column_name AS column_name
		FROM information_schema.key_column_usage
		WHERE table_schema = 'vt_vitesstest'
			AND table_name IN ('wp_commentmeta', 'wp_comments', 'wp_links', 'wp_options', 'wp_postmeta', 'wp_posts', 'wp_term_relationships', 'wp_term_taxonomy', 'wp_termmeta', 'wp_terms', 'wp_usermeta', 'wp_users')
			AND constraint_name='PRIMARY'
		ORDER BY table_name, ordinal_position
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_links` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_posts` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_term_relationships` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_comments`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_postmeta`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_term_taxonomy`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_termmeta`
context canceled: EOF (errno 2013) (sqlstate HY000) during query: SELECT
			table_name AS table_name,
			ordinal_position AS ordinal_position,
			column_name AS column_name
		FROM information_schema.key_column_usage
		WHERE table_schema = 'vt_vitesstest'
			AND table_name IN ('wp_commentmeta', 'wp_comments', 'wp_links', 'wp_options', 'wp_postmeta', 'wp_posts', 'wp_term_relationships', 'wp_term_taxonomy', 'wp_termmeta', 'wp_terms', 'wp_usermeta', 'wp_users')
			AND constraint_name='PRIMARY'
		ORDER BY table_name, ordinal_position
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_links` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_posts` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_term_relationships` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_comments`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_postmeta`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_term_taxonomy`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_termmeta`
context canceled
E0315 08:14:57.172988 4085950 main.go:76] remote error: rpc error: code = Unknown desc = TabletManager.ApplySchema on zone1-1728353218 error: EOF (errno 2013) (sqlstate HY000) during query: SELECT
			table_name AS table_name,
			ordinal_position AS ordinal_position,
			column_name AS column_name
		FROM information_schema.key_column_usage
		WHERE table_schema = 'vt_vitesstest'
			AND table_name IN ('wp_commentmeta', 'wp_comments', 'wp_links', 'wp_options', 'wp_postmeta', 'wp_posts', 'wp_term_relationships', 'wp_term_taxonomy', 'wp_termmeta', 'wp_terms', 'wp_usermeta', 'wp_users')
			AND constraint_name='PRIMARY'
		ORDER BY table_name, ordinal_position
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_links` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_posts` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_term_relationships` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_comments`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_postmeta`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_term_taxonomy`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_termmeta`
context canceled: EOF (errno 2013) (sqlstate HY000) during query: SELECT
			table_name AS table_name,
			ordinal_position AS ordinal_position,
			column_name AS column_name
		FROM information_schema.key_column_usage
		WHERE table_schema = 'vt_vitesstest'
			AND table_name IN ('wp_commentmeta', 'wp_comments', 'wp_links', 'wp_options', 'wp_postmeta', 'wp_posts', 'wp_term_relationships', 'wp_term_taxonomy', 'wp_termmeta', 'wp_terms', 'wp_usermeta', 'wp_users')
			AND constraint_name='PRIMARY'
		ORDER BY table_name, ordinal_position
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_links` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_posts` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SELECT * FROM `vt_vitesstest`.`wp_term_relationships` WHERE 1=0
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_comments`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_postmeta`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_term_taxonomy`
EOF (errno 2013) (sqlstate HY000) during query: SHOW CREATE TABLE `vt_vitesstest`.`wp_termmeta`
context canceled

After checking the logs, I noticed that the mysql master on the test keyspace gets killed (by vttablet?) immediately after starting the migration.
I tested the migration with vitess versions 12, 13 and latest with percona as well as plain mysql (5.7) using a wordpress db dump.

Here are my configs and logs:

Hope that helps!

Thanks,
chris

Kubernetes 1.22+ support

Ran into issues running on Kubernetes 1.22 with CRD API versions.

Linking vitessio/vitess#8756 here in case it it helpful -- this issue outlines information and solutions

Inclusive naming

Work is in progress on the vitess repo to deprecate/rename/fix comments to not use oppressive terminology.
As part of this, tablet_type MASTER will change to PRIMARY.
On the next vitess upgrade of operator after that is merged, code changes will be needed for operator code to compile.
As part of that all usage of master from code and comments should be replaced with primary.
Also, there are (will be) new RPCs available from vitess that should be used in place of deprecated ones.

Refs:
vitessio/vitess#7113
vitessio/vitess#7114

initContainer defaults preventing existing YAML from working

I have a (test) operator cluster YAML that has a keyspace section looking like this:

  keyspaces:
  - name: main
    partitionings:
    - equal:
        parts: 2
        shardTemplate:
          databaseInitScriptSecret:
            name: example-cluster-config
            key: init_db.sql
          replication:
            enforceSemiSync: true
          tabletPools:
          - cell: uscentral1a
            type: replica
            replicas: 1
            vttablet:
              extraFlags:
                db_charset: utf8mb4
              resources:
                requests:
                  cpu: 10m
                  memory: 256Mi
            mysqld:
              resources:
                requests:
                  cpu: 10m
                  memory: 256Mi
            dataVolumeClaimTemplate:
              accessModes: ["ReadWriteOnce"]
              resources:
                requests:
                  storage: 5Gi
              storageClassName: uscentral1a-ssd
          - cell: uscentral1c
            type: replica
            replicas: 1
            vttablet:
              extraFlags:
                db_charset: utf8mb4
              resources:
                requests:
                  cpu: 10m
                  memory: 256Mi
            mysqld:
              resources:
                requests:
                  cpu: 10m
                  memory: 256Mi
            dataVolumeClaimTemplate:
              accessModes: ["ReadWriteOnce"]
              resources:
                requests:
                  storage: 5Gi
              storageClassName: uscentral1c-ssd
          - cell: uscentral1f
            type: replica
            replicas: 1
            vttablet:
              extraFlags:
                db_charset: utf8mb4
              resources:
                requests:
                  cpu: 10m
                  memory: 256Mi
            mysqld:
              resources:
                requests:
                  cpu: 10m
                  memory: 256Mi
            dataVolumeClaimTemplate:
              accessModes: ["ReadWriteOnce"]
              resources:
                requests:
                  storage: 5Gi
              storageClassName: uscentral1f-ssd

This used to work just fine. But since PR #216 I get the multiple errors (one for each vttablet) when applying this (can be seen with kubectl get events or in the operator logs):

2m38s       Warning   CreateFailed             vitessshard/example-main-x-80-84bc4397                                   failed to create Pod example-vttablet-uscentral1f-2191363457-7dead7ff: Pod "example-vttablet-uscentral1f-2191363457-7dead7ff" is invalid: [spec.initContainers[0].resources.requests: Invalid value: "256Mi": must be less than or equal to memory limit, spec.initContainers[1].resources.requests: Invalid value: "256Mi": must be less than or equal to memory limit]

I can resolve this in one of two ways:

Reverting #216
Reducing the memory requests for the vttablets to 128Mi. However, this isn't a real solution, since it would pack too many vttablet pods into a node where I want to run fewer/larger vttablets...

Environment is GKE (1.19.16-gke.1500).

Can we avoid restarting tablet Pods to complete pending PVC filesystem resizes?

hi,
We use EKS+ EBS CSI. When the cluster has more shards (there are more EBS attached), when we expand EBS, due to CSI efficiency issues, many PVs are in the corev1.PersistentVolumeClaimFileSystemResizePending state, but this is expected.
We don't want to restart POD. How to avoid this situation?

#83

Permit usage of "_" character in keyspace name

Following this issue: vitessio/vitess#6755

We have names of the tables containing "_", so in original database we have database_name and in vitess operated database we have keyspace databasename which reduces the simplicity of migration process.

With that issue we have to do many changes to the application itself, so originally we cannot run vitess - original db in some canary setup.

K8s API doc links broken

It seems like K8s has changed their doc links (again); so we will need to fix the references in docs/api/config.json appropriately and regenerate the docs.

Right now the (broken) links generated look something like:

https://v1-16.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/#objectmeta-v1-meta

extraVolumeMounts on vitessOrchestrator initContainer doesn't mount volumes

I realize this is beta software. It needs some attention at some point.

Using the simplest config possible, initContainer doesn't mount volumes specified under the extraVolumeMounts.
excerpts of the deployment below. Spacing might be wrong here due to copy/paste, but correct in the config. I'm using this URL as documentation: https://vitess-operator.planetscale.dev/api/#planetscale.com/v2.VitessOrchestratorSpec

spec:
  vitessDashboard:
    replicas: 1
    resources:
      limits:
        memory: 512Mi
      requests:
        cpu: 75m
        memory: 256Mi

next section:

keyspaces:
- name: demoDB
    vitessOrchestrator:
      extraLabels:
        planetscale.com/component: vtorc
      configSecret:
        name: orc-config-80x
        key: orc_config.json
     initContainer:
        name: setup
        command: ["/bin/sh", "-c"]
        args: ["ls -l /"]
        extraVolumeMounts:
        - name: "orc-configs"
          mountPath: "/pickup"
        - name: "what"
          mountPath: /conf
      extraVolumeMounts:
       - name: "orc-configs"
          mountPath: "/pickup"
       - name: "what"
          mountPath: /conf
     extraVolumes:
      - name: "orc-configs"
         configMap:
          name: "orc-config"
      - name: "what"
        emptyDir: {}

The initContainer is created, but doesn't mount /pickup or /conf. Here's the log from the init container.
drwxr-xr-x 1 root root 214 Apr 27 19:40 bin
drwxr-xr-x 2 root root 6 Mar 19 23:44 boot
drwxr-xr-x 5 root root 360 Jul 2 02:15 dev
drwxr-xr-x 1 root root 66 Jul 2 02:15 etc
drwxr-xr-x 2 root root 6 Mar 19 23:44 home
drwxr-xr-x 1 root root 56 Apr 27 19:39 lib
drwxr-xr-x 2 root root 34 Apr 8 00:00 lib64
drwxr-xr-x 2 root root 6 Apr 8 00:00 media
drwxr-xr-x 2 root root 6 Apr 8 00:00 mnt
drwxr-xr-x 2 root root 6 Apr 8 00:00 opt
dr-xr-xr-x 246 root root 0 Jul 2 02:15 proc
drwx------ 2 root root 37 Apr 8 00:00 root
drwxr-xr-x 1 root root 21 Jul 2 02:15 run
drwxr-xr-x 1 root root 20 Apr 27 19:39 sbin
drwxr-xr-x 2 root root 6 Apr 8 00:00 srv
dr-xr-xr-x 13 root root 0 Jul 2 02:14 sys
drwxrwxrwt 1 root root 6 Apr 27 19:40 tmp
drwxr-xr-x 1 root root 81 Apr 8 00:00 usr
drwxr-xr-x 1 root root 17 Apr 8 00:00 var
drwxr-xr-x 1 vitess vitess 21 Jul 2 02:15 vt

Meanwhile, the vtorc container mounts both /pickup and /conf.

Allow opening Tablet port 15002 when Prometheus stats are enabled

Currently even if you specify the below extraFlags for the Gateway or Tablets it won't function properly as the port won't be accessible by the prometheus (or Datadog) agents.

extraFlags:
        grpc_prometheus: "true"

We could add the ability to have an extraPort option, but that would have to be only applicable to the VTTablet container.

request new feature : support ceph backup

as we know vitess support ceph backup, but vitess-operator does not support it.

reference:
https://vitess.io/docs/user-guides/operating-vitess/backup-and-restore/

The customization of the storage size of the etcd lockserver doesn't work

Added the following configuration to cell:

  cells:                                                                                                                                                       
  - name: samplecell                                                                                                                                        
    lockserver:                                                                                                                                                
      etcd:                                                                                                                                                    
        createClientService: true                                                                                                                              
        createPDB: true                                                                                                                                        
        createPeerService: true                                                                                                                                
        dataVolumeClaimTemplate:                                                                                                                               
          accessModes:                                                                                                                                         
          - ReadWriteOnce                                                                                                                                      
          dataSource: null                                                                                                                                     
          resources:                                                                                                                                           
            requests:                                                                                                                                          
              storage: 20Gi                                                                                                                                    
        image: quay.io/coreos/etcd:v3.3.13                                                                                                                     
        resources:                                                                                                                                             
          limits:                                                                                                                                              
            memory: 256Mi                                                                                                                                      
          requests:                                                                                                                                            
            cpu: 100m                                                                                                                                          
            memory: 256Mi

here is the generated EtcdLockServer crd printed by command kubectl get etcdlockservers --output yaml:

apiVersion: planetscale.com/v2                                                                                                                                 
kind: EtcdLockserver                                                                                                                                           
metadata:                                                                                                                                                                                                                                                                          
  ...                                                                                                                  
spec:                                                                                                                                                          
  createClientService: true                                                                                                                                    
  createPDB: true                                                                                                                                              
  createPeerService: true                                                                                                                                      
  dataVolumeClaimTemplate:                                                                                                                                     
    accessModes:                                                                                                                                               
    - ReadWriteOnce                                                                                                                                            
    dataSource: null                                                                                                                                           
    resources:                                                                                                                                                 
      requests:                                                                                                                                                
        storage: 20Gi                                                                                                                                          
  image: quay.io/coreos/etcd:v3.3.13                                                                                                                           
  resources:                                                                                                                                                   
    limits:                                                                                                                                                    
      memory: 256Mi                                                                                                                                            
    requests:                                                                                                                                                  
      cpu: 100m                                                                                                                                                
      memory: 256Mi

however the generated PVC spec still has a default storage size 1Gi:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  ...
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Possibility to set up ACLs for the tables

Question:

Is there a possibility to set up ACLs for the tables somehow?
The original topic described in here

I did not find anything relevant in operator documentation https://docs.planetscale.com/vitess-operator/api

Unable to create the new target Tablets when resharding

On minikube running kubernetes v1.16.2, vitess 8.0, operator 2.2, I tried the demo case to reshard and reshard, for practicing. After I inserted 10 records into the customer keyspace, I resharded it to equal-2, then equal-3, then custom-4, then down to equal-2. At this point, I got two problems:

After kubectl apply -f the resharding yaml file, the target tablet pods would be created, how ever both -80 shards were replicas, we expected a master here. I have to use vtctlclient InitShardMaster -force cmd to set one of them to be master.
After finished the resharding and issued the cmd vtctlclient DropSources, I found the source tablet pods were not completely torn down, they were still at 2/3 ready state, forever. kubectl logs <tablet-pod> vttablet showed:

I0107 18:38:02.963331       1 rpc_server.go:83] TabletManager.VReplicationExec(query:"select id, source, message, cell, tablet_types from _vt.vreplication where workflow='cust2x3' and db_name='vt_customer'" )(on zone1-0120139806 from ): (*tabletmanagerdata.VReplicationExecResponse)(nil)
I0107 18:38:03.023257       1 shard_sync.go:78] Change in shard record
E0107 18:38:03.023329       1 shard_sync.go:81] Shard watch failed: node doesn't exist: /vitess/example/global/keyspaces/customer/shards/-80/Shard
I0107 18:38:03.023357       1 shard_watcher.go:56] Stopping shard watch...
I0107 18:38:03.023379       1 shard_watcher.go:62] Shard watch stopped.
E0107 18:38:03.024538       1 shard_sync.go:110] Failed to sync shard record: node doesn't exist: /vitess/example/global/keyspaces/customer/shards/-80/Shard
I0107 18:38:03.026150       1 controller.go:137] stream 3: stopped
I0107 18:38:03.026448       1 controller.go:137] stream 4: stopped
I0107 18:38:03.037908       1 rpc_server.go:83] TabletManager.VReplicationExec(query:"delete from _vt.vreplication where db_name='vt_customer' and workflow='cust2x3_reverse'" )(on zone1-0120139806 from ): (*tabletmanagerdata.VReplicationExecResponse)(nil)
I0107 18:38:33.024880       1 shard_sync.go:73] Retry sync
E0107 18:38:33.025909       1 shard_sync.go:110] Failed to sync shard record: node doesn't exist: /vitess/example/global/keyspaces/customer/shards/-80/Shard

The -80 and 80- shards really need to be down and deleted, however they are still trying to sync between master and replica, and refused to shutdown. READY at 2/3, and STATUS is Running.

In anther case, when I resharded custom-4 to custom-3, which just merge two shards into one shard for the resharding task. I got the symptom:
3. After kubectl apply -f the resharding yaml file, the target tablet pods were not created at all.

pod has unbound immediate PersistentVolumeClaims, when i create vitess cluster

then ,

and then,

Ready for production?

Hi thanks for the lib! I wonder whether this operator is ready for production?

Ability to define resources for init containers

The init containers init-vt-root and init-mysql-socket (and possibly others) do not specify any resource requests or limits. In some cluster setups with strict quota requirements this can prevent a pod from being created.

The ability to define these resources would allow the operator to function properly in such clusters.

VTGate in a cell doesn't see Keyspaces without replica in that particular cell

Scenario: Vitess deployed via operator. Cluster contains 3 cells. Keyspace contains replica just in one of the cells (zone1): cluster.yml

Problem: VTGates in other two cells (zone2, zone3) do not have the visibility of the keyspace (i.e. show databases against VTGate in zone2 and zone3 doesn't list the keyspace).

Opening this one as an issue as per a relevant discussion on Slack.

vttablet: vreplication_retry_delay should be configurable

We observed the following on RDS:

If vreplication has an issue like a duplicate key error it will create numerous 'Binlog Dump GTID' threads when reconnecting. These can eventually consume all resources on a server and hang new connections. This is a known mysql behaviour per this issue. We can lessen the likelihood of this by bumping -vreplication_retry_delay up from it's 5s default, though we'll need to add monitoring to detect the underlying issue.

Suggest we allow vreplication_retry_delay to be configurable.

Autoscaling and policy-driven automations

Hello everyone, 😀

This as a proposal, and a place to discuss about the implementation of autoscaling and policy-driven automations for Vitess.

The general idea is to be able to provide a list of policies / rules (possibly in the spec) for certain events / actions to take place automatically. This would be very useful for specifying custom autoscaling scenarios, or alerts, for example.

The high-level approach to this could be:

We create an "orchestrator" server that takes metrics from our Vitess clusters.
We create some "policies" on when / how to scale up and/or down (based on metrics and limits). Also, we specify the frequency of the check for each policy.
The server checks at the given intervals for each policy and if applicable, runs custom predefined actions to our Vitess clusters.

To be able to achieve this, we need to be able to specify the following info in the spec for any policy:

Metrics and limits (ie. shard size > 256GB, avg cpu load > 60%)
This allows us to specify when our automations will be executed. It involves deciding which metrics are useful, as well as a reliable and accurate way to obtain them.
Set of actions when the event gets triggered (ie. execute script, alert, perform backup, etc. )
This allows us to specify what our automations will do when executed. The "execute script" is really the only necessary one, since it allows for custom made workflows and automations.
Interval / frequency of checking (ie. every 1 hour)
If we don't specify any metric-limit, the automation just runs at the specified interval. (useful for backups, reports)

All this could be tremendously useful, allowing for custom autoscaling (horizontal and vertical), alerts, reports, integrations, and automated backups.

Please give your thoughts and ideas!

This is a followup for a Slack discussion. Please check it out for more info.

How does MysqldImage Work?

Documentation says:

MysqldImage
(Appears on: VitessImages, VitessKeyspaceImages)

MysqldImage specifies the container image to use for mysqld, as well as declaring which MySQL flavor setting in Vitess the image is compatible with.

You've got a commented-out example for what seems like VitessImages, which is great.

But, no idea how to get the tags associated with an image. And just uncommenting
images:
mysqld:
mysql80Compatible:

creates containers with mysql57 running.

Allow extra Labels to be specified like the Annotations

We currently support adding arbitrary Annotations to be added to all the components. In our company we require having specific labels added to our K8s pods to indicate things like the owning team names.

Can we follow the annotations pattern and allow another map[string][string] for labels as well?

vitess-operator not managing/creating clusters in other namespaces

Hey,

I have the following problem:
I want to run the vitess-operator in the namespace "operators-vitess-operator" which works.
What doesn't work is to use the 101_initial_cluster.yaml and to deploy the cluster in the namespace "operators-vitess-example".
I can only deploy clusters in the namespace of the operator.

Here is the operator.yaml:

# Version: v2.5.0
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.0
  creationTimestamp: null
  name: etcdlockservers.planetscale.com
spec:
  group: planetscale.com
  names:
    kind: EtcdLockserver
    listKind: EtcdLockserverList
    plural: etcdlockservers
    shortNames:
    - etcdls
    singular: etcdlockserver
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata:
          type: object
        spec:
          properties:
            advertisePeerURLs:
              items:
                type: string
              maxItems: 3
              minItems: 3
              type: array
            affinity:
              type: object
              x-kubernetes-preserve-unknown-fields: true
            annotations:
              additionalProperties:
                type: string
              type: object
            clientService:
              properties:
                annotations:
                  additionalProperties:
                    type: string
                  type: object
                clusterIP:
                  type: string
              type: object
            createClientService:
              type: boolean
            createPDB:
              type: boolean
            createPeerService:
              type: boolean
            dataVolumeClaimTemplate:
              properties:
                accessModes:
                  items:
                    type: string
                  type: array
                resources:
                  properties:
                    limits:
                      additionalProperties:
                        anyOf:
                        - type: integer
                        - type: string
                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                        x-kubernetes-int-or-string: true
                      type: object
                    requests:
                      additionalProperties:
                        anyOf:
                        - type: integer
                        - type: string
                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                        x-kubernetes-int-or-string: true
                      type: object
                  type: object
                selector:
                  properties:
                    matchExpressions:
                      items:
                        properties:
                          key:
                            type: string
                          operator:
                            type: string
                          values:
                            items:
                              type: string
                            type: array
                        required:
                        - key
                        - operator
                        type: object
                      type: array
                    matchLabels:
                      additionalProperties:
                        type: string
                      type: object
                  type: object
                storageClassName:
                  type: string
                volumeMode:
                  type: string
                volumeName:
                  type: string
              type: object
            extraEnv:
              items:
                properties:
                  name:
                    type: string
                  value:
                    type: string
                  valueFrom:
                    properties:
                      configMapKeyRef:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          optional:
                            type: boolean
                        required:
                        - key
                        type: object
                      fieldRef:
                        properties:
                          apiVersion:
                            type: string
                          fieldPath:
                            type: string
                        required:
                        - fieldPath
                        type: object
                      resourceFieldRef:
                        properties:
                          containerName:
                            type: string
                          divisor:
                            anyOf:
                            - type: integer
                            - type: string
                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                            x-kubernetes-int-or-string: true
                          resource:
                            type: string
                        required:
                        - resource
                        type: object
                      secretKeyRef:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          optional:
                            type: boolean
                        required:
                        - key
                        type: object
                    type: object
                required:
                - name
                type: object
              type: array
            extraFlags:
              additionalProperties:
                type: string
              type: object
            extraLabels:
              additionalProperties:
                type: string
              type: object
            extraVolumeMounts:
              items:
                properties:
                  mountPath:
                    type: string
                  mountPropagation:
                    type: string
                  name:
                    type: string
                  readOnly:
                    type: boolean
                  subPath:
                    type: string
                  subPathExpr:
                    type: string
                required:
                - mountPath
                - name
                type: object
              type: array
            extraVolumes:
              items:
                required:
                - name
                type: object
                x-kubernetes-preserve-unknown-fields: true
              type: array
            image:
              type: string
            imagePullPolicy:
              type: string
            imagePullSecrets:
              items:
                properties:
                  name:
                    type: string
                type: object
              type: array
            initContainers:
              items:
                required:
                - name
                type: object
                x-kubernetes-preserve-unknown-fields: true
              type: array
            localMemberIndex:
              format: int32
              maximum: 3
              minimum: 1
              type: integer
            peerService:
              properties:
                annotations:
                  additionalProperties:
                    type: string
                  type: object
                clusterIP:
                  type: string
              type: object
            resources:
              properties:
                limits:
                  additionalProperties:
                    anyOf:
                    - type: integer
                    - type: string
                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                    x-kubernetes-int-or-string: true
                  type: object
                requests:
                  additionalProperties:
                    anyOf:
                    - type: integer
                    - type: string
                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                    x-kubernetes-int-or-string: true
                  type: object
              type: object
            sidecarContainers:
              items:
                required:
                - name
                type: object
                x-kubernetes-preserve-unknown-fields: true
              type: array
            tolerations:
              items:
                type: object
                x-kubernetes-preserve-unknown-fields: true
              type: array
            zone:
              type: string
          type: object
        status:
          properties:
            available:
              type: string
            clientServiceName:
              type: string
            observedGeneration:
              format: int64
              type: integer
          type: object
      type: object
  version: v2
  versions:
  - name: v2
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.0
  creationTimestamp: null
  name: vitessbackups.planetscale.com
spec:
  group: planetscale.com
  names:
    kind: VitessBackup
    listKind: VitessBackupList
    plural: vitessbackups
    shortNames:
    - vtb
    singular: vitessbackup
  scope: Namespaced
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata:
          type: object
        spec:
          type: object
        status:
          properties:
            complete:
              type: boolean
            engine:
              type: string
            finishedTime:
              format: date-time
              type: string
            position:
              type: string
            startTime:
              format: date-time
              type: string
            storageDirectory:
              type: string
            storageName:
              type: string
          type: object
      type: object
  version: v2
  versions:
  - name: v2
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.0
  creationTimestamp: null
  name: vitessbackupstorages.planetscale.com
spec:
  group: planetscale.com
  names:
    kind: VitessBackupStorage
    listKind: VitessBackupStorageList
    plural: vitessbackupstorages
    shortNames:
    - vtbs
    singular: vitessbackupstorage
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata:
          type: object
        spec:
          properties:
            location:
              properties:
                annotations:
                  additionalProperties:
                    type: string
                  type: object
                azblob:
                  properties:
                    account:
                      minLength: 1
                      type: string
                    authSecret:
                      properties:
                        key:
                          type: string
                        name:
                          type: string
                        volumeName:
                          type: string
                      required:
                      - key
                      type: object
                    container:
                      minLength: 1
                      type: string
                    keyPrefix:
                      maxLength: 256
                      pattern: ^[^\r\n]*$
                      type: string
                  required:
                  - account
                  - authSecret
                  - container
                  type: object
                ceph:
                  properties:
                    authSecret:
                      properties:
                        key:
                          type: string
                        name:
                          type: string
                        volumeName:
                          type: string
                      required:
                      - key
                      type: object
                  required:
                  - authSecret
                  type: object
                gcs:
                  properties:
                    authSecret:
                      properties:
                        key:
                          type: string
                        name:
                          type: string
                        volumeName:
                          type: string
                      required:
                      - key
                      type: object
                    bucket:
                      minLength: 1
                      type: string
                    keyPrefix:
                      maxLength: 256
                      pattern: ^[^\r\n]*$
                      type: string
                  required:
                  - bucket
                  type: object
                name:
                  maxLength: 63
                  pattern: ^[A-Za-z0-9]([A-Za-z0-9-_.]*[A-Za-z0-9])?$
                  type: string
                s3:
                  properties:
                    authSecret:
                      properties:
                        key:
                          type: string
                        name:
                          type: string
                        volumeName:
                          type: string
                      required:
                      - key
                      type: object
                    bucket:
                      minLength: 1
                      type: string
                    endpoint:
                      type: string
                    keyPrefix:
                      maxLength: 256
                      pattern: ^[^\r\n]*$
                      type: string
                    region:
                      minLength: 1
                      type: string
                  required:
                  - bucket
                  - region
                  type: object
                volume:
                  type: object
                  x-kubernetes-preserve-unknown-fields: true
                volumeSubPath:
                  type: string
              type: object
            subcontroller:
              properties:
                serviceAccountName:
                  type: string
              type: object
          required:
          - location
          type: object
        status:
          properties:
            observedGeneration:
              format: int64
              type: integer
            totalBackupCount:
              format: int32
              type: integer
          type: object
      type: object
  version: v2
  versions:
  - name: v2
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.0
  creationTimestamp: null
  name: vitesscells.planetscale.com
spec:
  group: planetscale.com
  names:
    kind: VitessCell
    listKind: VitessCellList
    plural: vitesscells
    shortNames:
    - vtc
    singular: vitesscell
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata:
          type: object
        spec:
          properties:
            allCells:
              items:
                type: string
              type: array
            extraVitessFlags:
              additionalProperties:
                type: string
              type: object
            gateway:
              properties:
                affinity:
                  type: object
                  x-kubernetes-preserve-unknown-fields: true
                annotations:
                  additionalProperties:
                    type: string
                  type: object
                authentication:
                  properties:
                    static:
                      properties:
                        secret:
                          properties:
                            key:
                              type: string
                            name:
                              type: string
                            volumeName:
                              type: string
                          required:
                          - key
                          type: object
                      type: object
                  type: object
                extraEnv:
                  items:
                    properties:
                      name:
                        type: string
                      value:
                        type: string
                      valueFrom:
                        properties:
                          configMapKeyRef:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              optional:
                                type: boolean
                            required:
                            - key
                            type: object
                          fieldRef:
                            properties:
                              apiVersion:
                                type: string
                              fieldPath:
                                type: string
                            required:
                            - fieldPath
                            type: object
                          resourceFieldRef:
                            properties:
                              containerName:
                                type: string
                              divisor:
                                anyOf:
                                - type: integer
                                - type: string
                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                x-kubernetes-int-or-string: true
                              resource:
                                type: string
                            required:
                            - resource
                            type: object
                          secretKeyRef:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              optional:
                                type: boolean
                            required:
                            - key
                            type: object
                        type: object
                    required:
                    - name
                    type: object
                  type: array
                extraFlags:
                  additionalProperties:
                    type: string
                  type: object
                extraLabels:
                  additionalProperties:
                    type: string
                  type: object
                extraVolumeMounts:
                  items:
                    properties:
                      mountPath:
                        type: string
                      mountPropagation:
                        type: string
                      name:
                        type: string
                      readOnly:
                        type: boolean
                      subPath:
                        type: string
                      subPathExpr:
                        type: string
                    required:
                    - mountPath
                    - name
                    type: object
                  type: array
                extraVolumes:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                initContainers:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                replicas:
                  format: int32
                  minimum: 0
                  type: integer
                resources:
                  properties:
                    limits:
                      additionalProperties:
                        anyOf:
                        - type: integer
                        - type: string
                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                        x-kubernetes-int-or-string: true
                      type: object
                    requests:
                      additionalProperties:
                        anyOf:
                        - type: integer
                        - type: string
                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                        x-kubernetes-int-or-string: true
                      type: object
                  type: object
                secureTransport:
                  properties:
                    required:
                      type: boolean
                    tls:
                      properties:
                        certSecret:
                          properties:
                            key:
                              type: string
                            name:
                              type: string
                            volumeName:
                              type: string
                          required:
                          - key
                          type: object
                        clientCACertSecret:
                          properties:
                            key:
                              type: string
                            name:
                              type: string
                            volumeName:
                              type: string
                          required:
                          - key
                          type: object
                        keySecret:
                          properties:
                            key:
                              type: string
                            name:
                              type: string
                            volumeName:
                              type: string
                          required:
                          - key
                          type: object
                      type: object
                  type: object
                service:
                  properties:
                    annotations:
                      additionalProperties:
                        type: string
                      type: object
                    clusterIP:
                      type: string
                  type: object
                sidecarContainers:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                tolerations:
                  items:
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                topologySpreadConstraints:
                  items:
                    required:
                    - maxSkew
                    - topologyKey
                    - whenUnsatisfiable
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
              type: object
            globalLockserver:
              properties:
                address:
                  type: string
                implementation:
                  type: string
                rootPath:
                  type: string
              required:
              - address
              - implementation
              - rootPath
              type: object
            imagePullPolicies:
              properties:
                mysqld:
                  type: string
                mysqldExporter:
                  type: string
                vtbackup:
                  type: string
                vtctld:
                  type: string
                vtgate:
                  type: string
                vtorc:
                  type: string
                vttablet:
                  type: string
              type: object
            imagePullSecrets:
              items:
                properties:
                  name:
                    type: string
                type: object
              type: array
            images:
              properties:
                vtgate:
                  type: string
              type: object
            lockserver:
              properties:
                etcd:
                  properties:
                    advertisePeerURLs:
                      items:
                        type: string
                      maxItems: 3
                      minItems: 3
                      type: array
                    affinity:
                      type: object
                      x-kubernetes-preserve-unknown-fields: true
                    annotations:
                      additionalProperties:
                        type: string
                      type: object
                    clientService:
                      properties:
                        annotations:
                          additionalProperties:
                            type: string
                          type: object
                        clusterIP:
                          type: string
                      type: object
                    createClientService:
                      type: boolean
                    createPDB:
                      type: boolean
                    createPeerService:
                      type: boolean
                    dataVolumeClaimTemplate:
                      properties:
                        accessModes:
                          items:
                            type: string
                          type: array
                        resources:
                          properties:
                            limits:
                              additionalProperties:
                                anyOf:
                                - type: integer
                                - type: string
                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                x-kubernetes-int-or-string: true
                              type: object
                            requests:
                              additionalProperties:
                                anyOf:
                                - type: integer
                                - type: string
                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                x-kubernetes-int-or-string: true
                              type: object
                          type: object
                        selector:
                          properties:
                            matchExpressions:
                              items:
                                properties:
                                  key:
                                    type: string
                                  operator:
                                    type: string
                                  values:
                                    items:
                                      type: string
                                    type: array
                                required:
                                - key
                                - operator
                                type: object
                              type: array
                            matchLabels:
                              additionalProperties:
                                type: string
                              type: object
                          type: object
                        storageClassName:
                          type: string
                        volumeMode:
                          type: string
                        volumeName:
                          type: string
                      type: object
                    extraEnv:
                      items:
                        properties:
                          name:
                            type: string
                          value:
                            type: string
                          valueFrom:
                            properties:
                              configMapKeyRef:
                                properties:
                                  key:
                                    type: string
                                  name:
                                    type: string
                                  optional:
                                    type: boolean
                                required:
                                - key
                                type: object
                              fieldRef:
                                properties:
                                  apiVersion:
                                    type: string
                                  fieldPath:
                                    type: string
                                required:
                                - fieldPath
                                type: object
                              resourceFieldRef:
                                properties:
                                  containerName:
                                    type: string
                                  divisor:
                                    anyOf:
                                    - type: integer
                                    - type: string
                                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                    x-kubernetes-int-or-string: true
                                  resource:
                                    type: string
                                required:
                                - resource
                                type: object
                              secretKeyRef:
                                properties:
                                  key:
                                    type: string
                                  name:
                                    type: string
                                  optional:
                                    type: boolean
                                required:
                                - key
                                type: object
                            type: object
                        required:
                        - name
                        type: object
                      type: array
                    extraFlags:
                      additionalProperties:
                        type: string
                      type: object
                    extraLabels:
                      additionalProperties:
                        type: string
                      type: object
                    extraVolumeMounts:
                      items:
                        properties:
                          mountPath:
                            type: string
                          mountPropagation:
                            type: string
                          name:
                            type: string
                          readOnly:
                            type: boolean
                          subPath:
                            type: string
                          subPathExpr:
                            type: string
                        required:
                        - mountPath
                        - name
                        type: object
                      type: array
                    extraVolumes:
                      items:
                        required:
                        - name
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      type: array
                    image:
                      type: string
                    imagePullPolicy:
                      type: string
                    imagePullSecrets:
                      items:
                        properties:
                          name:
                            type: string
                        type: object
                      type: array
                    initContainers:
                      items:
                        required:
                        - name
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      type: array
                    localMemberIndex:
                      format: int32
                      maximum: 3
                      minimum: 1
                      type: integer
                    peerService:
                      properties:
                        annotations:
                          additionalProperties:
                            type: string
                          type: object
                        clusterIP:
                          type: string
                      type: object
                    resources:
                      properties:
                        limits:
                          additionalProperties:
                            anyOf:
                            - type: integer
                            - type: string
                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                            x-kubernetes-int-or-string: true
                          type: object
                        requests:
                          additionalProperties:
                            anyOf:
                            - type: integer
                            - type: string
                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                            x-kubernetes-int-or-string: true
                          type: object
                      type: object
                    sidecarContainers:
                      items:
                        required:
                        - name
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      type: array
                    tolerations:
                      items:
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      type: array
                  type: object
                external:
                  properties:
                    address:
                      type: string
                    implementation:
                      type: string
                    rootPath:
                      type: string
                  required:
                  - address
                  - implementation
                  - rootPath
                  type: object
              type: object
            name:
              maxLength: 63
              minLength: 1
              pattern: ^[A-Za-z0-9]([_.A-Za-z0-9]*[A-Za-z0-9])?$
              type: string
            topologyReconciliation:
              properties:
                pruneCells:
                  type: boolean
                pruneKeyspaces:
                  type: boolean
                pruneShardCells:
                  type: boolean
                pruneShards:
                  type: boolean
                pruneSrvKeyspaces:
                  type: boolean
                pruneTablets:
                  type: boolean
                registerCells:
                  type: boolean
                registerCellsAliases:
                  type: boolean
              type: object
            zone:
              type: string
          required:
          - allCells
          - globalLockserver
          - name
          type: object
        status:
          properties:
            gateway:
              properties:
                available:
                  type: string
                serviceName:
                  type: string
              type: object
            idle:
              type: string
            keyspaces:
              additionalProperties:
                type: object
              type: object
            lockserver:
              properties:
                etcd:
                  properties:
                    available:
                      type: string
                    clientServiceName:
                      type: string
                    observedGeneration:
                      format: int64
                      type: integer
                  type: object
              type: object
            observedGeneration:
              format: int64
              type: integer
          type: object
      type: object
  version: v2
  versions:
  - name: v2
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.0
  creationTimestamp: null
  name: vitessclusters.planetscale.com
spec:
  group: planetscale.com
  names:
    kind: VitessCluster
    listKind: VitessClusterList
    plural: vitessclusters
    shortNames:
    - vt
    singular: vitesscluster
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata:
          type: object
        spec:
          properties:
            backup:
              properties:
                engine:
                  enum:
                  - builtin
                  - xtrabackup
                  type: string
                locations:
                  items:
                    properties:
                      annotations:
                        additionalProperties:
                          type: string
                        type: object
                      azblob:
                        properties:
                          account:
                            minLength: 1
                            type: string
                          authSecret:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              volumeName:
                                type: string
                            required:
                            - key
                            type: object
                          container:
                            minLength: 1
                            type: string
                          keyPrefix:
                            maxLength: 256
                            pattern: ^[^\r\n]*$
                            type: string
                        required:
                        - account
                        - authSecret
                        - container
                        type: object
                      ceph:
                        properties:
                          authSecret:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              volumeName:
                                type: string
                            required:
                            - key
                            type: object
                        required:
                        - authSecret
                        type: object
                      gcs:
                        properties:
                          authSecret:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              volumeName:
                                type: string
                            required:
                            - key
                            type: object
                          bucket:
                            minLength: 1
                            type: string
                          keyPrefix:
                            maxLength: 256
                            pattern: ^[^\r\n]*$
                            type: string
                        required:
                        - bucket
                        type: object
                      name:
                        maxLength: 63
                        pattern: ^[A-Za-z0-9]([A-Za-z0-9-_.]*[A-Za-z0-9])?$
                        type: string
                      s3:
                        properties:
                          authSecret:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              volumeName:
                                type: string
                            required:
                            - key
                            type: object
                          bucket:
                            minLength: 1
                            type: string
                          endpoint:
                            type: string
                          keyPrefix:
                            maxLength: 256
                            pattern: ^[^\r\n]*$
                            type: string
                          region:
                            minLength: 1
                            type: string
                        required:
                        - bucket
                        - region
                        type: object
                      volume:
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      volumeSubPath:
                        type: string
                    type: object
                  minItems: 1
                  type: array
                subcontroller:
                  properties:
                    serviceAccountName:
                      type: string
                  type: object
              required:
              - locations
              type: object
            cells:
              items:
                properties:
                  gateway:
                    properties:
                      affinity:
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      annotations:
                        additionalProperties:
                          type: string
                        type: object
                      authentication:
                        properties:
                          static:
                            properties:
                              secret:
                                properties:
                                  key:
                                    type: string
                                  name:
                                    type: string
                                  volumeName:
                                    type: string
                                required:
                                - key
                                type: object
                            type: object
                        type: object
                      extraEnv:
                        items:
                          properties:
                            name:
                              type: string
                            value:
                              type: string
                            valueFrom:
                              properties:
                                configMapKeyRef:
                                  properties:
                                    key:
                                      type: string
                                    name:
                                      type: string
                                    optional:
                                      type: boolean
                                  required:
                                  - key
                                  type: object
                                fieldRef:
                                  properties:
                                    apiVersion:
                                      type: string
                                    fieldPath:
                                      type: string
                                  required:
                                  - fieldPath
                                  type: object
                                resourceFieldRef:
                                  properties:
                                    containerName:
                                      type: string
                                    divisor:
                                      anyOf:
                                      - type: integer
                                      - type: string
                                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                      x-kubernetes-int-or-string: true
                                    resource:
                                      type: string
                                  required:
                                  - resource
                                  type: object
                                secretKeyRef:
                                  properties:
                                    key:
                                      type: string
                                    name:
                                      type: string
                                    optional:
                                      type: boolean
                                  required:
                                  - key
                                  type: object
                              type: object
                          required:
                          - name
                          type: object
                        type: array
                      extraFlags:
                        additionalProperties:
                          type: string
                        type: object
                      extraLabels:
                        additionalProperties:
                          type: string
                        type: object
                      extraVolumeMounts:
                        items:
                          properties:
                            mountPath:
                              type: string
                            mountPropagation:
                              type: string
                            name:
                              type: string
                            readOnly:
                              type: boolean
                            subPath:
                              type: string
                            subPathExpr:
                              type: string
                          required:
                          - mountPath
                          - name
                          type: object
                        type: array
                      extraVolumes:
                        items:
                          required:
                          - name
                          type: object
                          x-kubernetes-preserve-unknown-fields: true
                        type: array
                      initContainers:
                        items:
                          required:
                          - name
                          type: object
                          x-kubernetes-preserve-unknown-fields: true
                        type: array
                      replicas:
                        format: int32
                        minimum: 0
                        type: integer
                      resources:
                        properties:
                          limits:
                            additionalProperties:
                              anyOf:
                              - type: integer
                              - type: string
                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                              x-kubernetes-int-or-string: true
                            type: object
                          requests:
                            additionalProperties:
                              anyOf:
                              - type: integer
                              - type: string
                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                              x-kubernetes-int-or-string: true
                            type: object
                        type: object
                      secureTransport:
                        properties:
                          required:
                            type: boolean
                          tls:
                            properties:
                              certSecret:
                                properties:
                                  key:
                                    type: string
                                  name:
                                    type: string
                                  volumeName:
                                    type: string
                                required:
                                - key
                                type: object
                              clientCACertSecret:
                                properties:
                                  key:
                                    type: string
                                  name:
                                    type: string
                                  volumeName:
                                    type: string
                                required:
                                - key
                                type: object
                              keySecret:
                                properties:
                                  key:
                                    type: string
                                  name:
                                    type: string
                                  volumeName:
                                    type: string
                                required:
                                - key
                                type: object
                            type: object
                        type: object
                      service:
                        properties:
                          annotations:
                            additionalProperties:
                              type: string
                            type: object
                          clusterIP:
                            type: string
                        type: object
                      sidecarContainers:
                        items:
                          required:
                          - name
                          type: object
                          x-kubernetes-preserve-unknown-fields: true
                        type: array
                      tolerations:
                        items:
                          type: object
                          x-kubernetes-preserve-unknown-fields: true
                        type: array
                      topologySpreadConstraints:
                        items:
                          required:
                          - maxSkew
                          - topologyKey
                          - whenUnsatisfiable
                          type: object
                          x-kubernetes-preserve-unknown-fields: true
                        type: array
                    type: object
                  lockserver:
                    properties:
                      etcd:
                        properties:
                          advertisePeerURLs:
                            items:
                              type: string
                            maxItems: 3
                            minItems: 3
                            type: array
                          affinity:
                            type: object
                            x-kubernetes-preserve-unknown-fields: true
                          annotations:
                            additionalProperties:
                              type: string
                            type: object
                          clientService:
                            properties:
                              annotations:
                                additionalProperties:
                                  type: string
                                type: object
                              clusterIP:
                                type: string
                            type: object
                          createClientService:
                            type: boolean
                          createPDB:
                            type: boolean
                          createPeerService:
                            type: boolean
                          dataVolumeClaimTemplate:
                            properties:
                              accessModes:
                                items:
                                  type: string
                                type: array
                              resources:
                                properties:
                                  limits:
                                    additionalProperties:
                                      anyOf:
                                      - type: integer
                                      - type: string
                                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                      x-kubernetes-int-or-string: true
                                    type: object
                                  requests:
                                    additionalProperties:
                                      anyOf:
                                      - type: integer
                                      - type: string
                                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                      x-kubernetes-int-or-string: true
                                    type: object
                                type: object
                              selector:
                                properties:
                                  matchExpressions:
                                    items:
                                      properties:
                                        key:
                                          type: string
                                        operator:
                                          type: string
                                        values:
                                          items:
                                            type: string
                                          type: array
                                      required:
                                      - key
                                      - operator
                                      type: object
                                    type: array
                                  matchLabels:
                                    additionalProperties:
                                      type: string
                                    type: object
                                type: object
                              storageClassName:
                                type: string
                              volumeMode:
                                type: string
                              volumeName:
                                type: string
                            type: object
                          extraEnv:
                            items:
                              properties:
                                name:
                                  type: string
                                value:
                                  type: string
                                valueFrom:
                                  properties:
                                    configMapKeyRef:
                                      properties:
                                        key:
                                          type: string
                                        name:
                                          type: string
                                        optional:
                                          type: boolean
                                      required:
                                      - key
                                      type: object
                                    fieldRef:
                                      properties:
                                        apiVersion:
                                          type: string
                                        fieldPath:
                                          type: string
                                      required:
                                      - fieldPath
                                      type: object
                                    resourceFieldRef:
                                      properties:
                                        containerName:
                                          type: string
                                        divisor:
                                          anyOf:
                                          - type: integer
                                          - type: string
                                          pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                          x-kubernetes-int-or-string: true
                                        resource:
                                          type: string
                                      required:
                                      - resource
                                      type: object
                                    secretKeyRef:
                                      properties:
                                        key:
                                          type: string
                                        name:
                                          type: string
                                        optional:
                                          type: boolean
                                      required:
                                      - key
                                      type: object
                                  type: object
                              required:
                              - name
                              type: object
                            type: array
                          extraFlags:
                            additionalProperties:
                              type: string
                            type: object
                          extraLabels:
                            additionalProperties:
                              type: string
                            type: object
                          extraVolumeMounts:
                            items:
                              properties:
                                mountPath:
                                  type: string
                                mountPropagation:
                                  type: string
                                name:
                                  type: string
                                readOnly:
                                  type: boolean
                                subPath:
                                  type: string
                                subPathExpr:
                                  type: string
                              required:
                              - mountPath
                              - name
                              type: object
                            type: array
                          extraVolumes:
                            items:
                              required:
                              - name
                              type: object
                              x-kubernetes-preserve-unknown-fields: true
                            type: array
                          image:
                            type: string
                          imagePullPolicy:
                            type: string
                          imagePullSecrets:
                            items:
                              properties:
                                name:
                                  type: string
                              type: object
                            type: array
                          initContainers:
                            items:
                              required:
                              - name
                              type: object
                              x-kubernetes-preserve-unknown-fields: true
                            type: array
                          localMemberIndex:
                            format: int32
                            maximum: 3
                            minimum: 1
                            type: integer
                          peerService:
                            properties:
                              annotations:
                                additionalProperties:
                                  type: string
                                type: object
                              clusterIP:
                                type: string
                            type: object
                          resources:
                            properties:
                              limits:
                                additionalProperties:
                                  anyOf:
                                  - type: integer
                                  - type: string
                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                  x-kubernetes-int-or-string: true
                                type: object
                              requests:
                                additionalProperties:
                                  anyOf:
                                  - type: integer
                                  - type: string
                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                  x-kubernetes-int-or-string: true
                                type: object
                            type: object
                          sidecarContainers:
                            items:
                              required:
                              - name
                              type: object
                              x-kubernetes-preserve-unknown-fields: true
                            type: array
                          tolerations:
                            items:
                              type: object
                              x-kubernetes-preserve-unknown-fields: true
                            type: array
                        type: object
                      external:
                        properties:
                          address:
                            type: string
                          implementation:
                            type: string
                          rootPath:
                            type: string
                        required:
                        - address
                        - implementation
                        - rootPath
                        type: object
                    type: object
                  name:
                    maxLength: 63
                    minLength: 1
                    pattern: ^[A-Za-z0-9]([_.A-Za-z0-9]*[A-Za-z0-9])?$
                    type: string
                  zone:
                    type: string
                required:
                - name
                type: object
              type: array
            extraVitessFlags:
              additionalProperties:
                type: string
              type: object
            gatewayService:
              properties:
                annotations:
                  additionalProperties:
                    type: string
                  type: object
                clusterIP:
                  type: string
              type: object
            globalLockserver:
              properties:
                etcd:
                  properties:
                    advertisePeerURLs:
                      items:
                        type: string
                      maxItems: 3
                      minItems: 3
                      type: array
                    affinity:
                      type: object
                      x-kubernetes-preserve-unknown-fields: true
                    annotations:
                      additionalProperties:
                        type: string
                      type: object
                    clientService:
                      properties:
                        annotations:
                          additionalProperties:
                            type: string
                          type: object
                        clusterIP:
                          type: string
                      type: object
                    createClientService:
                      type: boolean
                    createPDB:
                      type: boolean
                    createPeerService:
                      type: boolean
                    dataVolumeClaimTemplate:
                      properties:
                        accessModes:
                          items:
                            type: string
                          type: array
                        resources:
                          properties:
                            limits:
                              additionalProperties:
                                anyOf:
                                - type: integer
                                - type: string
                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                x-kubernetes-int-or-string: true
                              type: object
                            requests:
                              additionalProperties:
                                anyOf:
                                - type: integer
                                - type: string
                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                x-kubernetes-int-or-string: true
                              type: object
                          type: object
                        selector:
                          properties:
                            matchExpressions:
                              items:
                                properties:
                                  key:
                                    type: string
                                  operator:
                                    type: string
                                  values:
                                    items:
                                      type: string
                                    type: array
                                required:
                                - key
                                - operator
                                type: object
                              type: array
                            matchLabels:
                              additionalProperties:
                                type: string
                              type: object
                          type: object
                        storageClassName:
                          type: string
                        volumeMode:
                          type: string
                        volumeName:
                          type: string
                      type: object
                    extraEnv:
                      items:
                        properties:
                          name:
                            type: string
                          value:
                            type: string
                          valueFrom:
                            properties:
                              configMapKeyRef:
                                properties:
                                  key:
                                    type: string
                                  name:
                                    type: string
                                  optional:
                                    type: boolean
                                required:
                                - key
                                type: object
                              fieldRef:
                                properties:
                                  apiVersion:
                                    type: string
                                  fieldPath:
                                    type: string
                                required:
                                - fieldPath
                                type: object
                              resourceFieldRef:
                                properties:
                                  containerName:
                                    type: string
                                  divisor:
                                    anyOf:
                                    - type: integer
                                    - type: string
                                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                    x-kubernetes-int-or-string: true
                                  resource:
                                    type: string
                                required:
                                - resource
                                type: object
                              secretKeyRef:
                                properties:
                                  key:
                                    type: string
                                  name:
                                    type: string
                                  optional:
                                    type: boolean
                                required:
                                - key
                                type: object
                            type: object
                        required:
                        - name
                        type: object
                      type: array
                    extraFlags:
                      additionalProperties:
                        type: string
                      type: object
                    extraLabels:
                      additionalProperties:
                        type: string
                      type: object
                    extraVolumeMounts:
                      items:
                        properties:
                          mountPath:
                            type: string
                          mountPropagation:
                            type: string
                          name:
                            type: string
                          readOnly:
                            type: boolean
                          subPath:
                            type: string
                          subPathExpr:
                            type: string
                        required:
                        - mountPath
                        - name
                        type: object
                      type: array
                    extraVolumes:
                      items:
                        required:
                        - name
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      type: array
                    image:
                      type: string
                    imagePullPolicy:
                      type: string
                    imagePullSecrets:
                      items:
                        properties:
                          name:
                            type: string
                        type: object
                      type: array
                    initContainers:
                      items:
                        required:
                        - name
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      type: array
                    localMemberIndex:
                      format: int32
                      maximum: 3
                      minimum: 1
                      type: integer
                    peerService:
                      properties:
                        annotations:
                          additionalProperties:
                            type: string
                          type: object
                        clusterIP:
                          type: string
                      type: object
                    resources:
                      properties:
                        limits:
                          additionalProperties:
                            anyOf:
                            - type: integer
                            - type: string
                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                            x-kubernetes-int-or-string: true
                          type: object
                        requests:
                          additionalProperties:
                            anyOf:
                            - type: integer
                            - type: string
                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                            x-kubernetes-int-or-string: true
                          type: object
                      type: object
                    sidecarContainers:
                      items:
                        required:
                        - name
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      type: array
                    tolerations:
                      items:
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      type: array
                  type: object
                external:
                  properties:
                    address:
                      type: string
                    implementation:
                      type: string
                    rootPath:
                      type: string
                  required:
                  - address
                  - implementation
                  - rootPath
                  type: object
              type: object
            imagePullPolicies:
              properties:
                mysqld:
                  type: string
                mysqldExporter:
                  type: string
                vtbackup:
                  type: string
                vtctld:
                  type: string
                vtgate:
                  type: string
                vtorc:
                  type: string
                vttablet:
                  type: string
              type: object
            imagePullSecrets:
              items:
                properties:
                  name:
                    type: string
                type: object
              type: array
            images:
              properties:
                mysqld:
                  properties:
                    mariadb103Compatible:
                      type: string
                    mariadbCompatible:
                      type: string
                    mysql56Compatible:
                      type: string
                    mysql80Compatible:
                      type: string
                  type: object
                mysqldExporter:
                  type: string
                vtbackup:
                  type: string
                vtctld:
                  type: string
                vtgate:
                  type: string
                vtorc:
                  type: string
                vttablet:
                  type: string
              type: object
            keyspaces:
              items:
                properties:
                  annotations:
                    additionalProperties:
                      type: string
                    type: object
                  databaseName:
                    type: string
                  name:
                    maxLength: 63
                    minLength: 1
                    pattern: ^[A-Za-z0-9]([A-Za-z0-9-_.]*[A-Za-z0-9])?$
                    type: string
                  partitionings:
                    items:
                      properties:
                        custom:
                          properties:
                            shards:
                              items:
                                properties:
                                  annotations:
                                    additionalProperties:
                                      type: string
                                    type: object
                                  databaseInitScriptSecret:
                                    properties:
                                      key:
                                        type: string
                                      name:
                                        type: string
                                      volumeName:
                                        type: string
                                    required:
                                    - key
                                    type: object
                                  keyRange:
                                    properties:
                                      end:
                                        pattern: ^([0-9a-f][0-9a-f])*$
                                        type: string
                                      start:
                                        pattern: ^([0-9a-f][0-9a-f])*$
                                        type: string
                                    type: object
                                  replication:
                                    properties:
                                      enforceSemiSync:
                                        type: boolean
                                      initializeBackup:
                                        type: boolean
                                      initializeMaster:
                                        type: boolean
                                      recoverRestartedMaster:
                                        type: boolean
                                    type: object
                                  tabletPools:
                                    items:
                                      properties:
                                        affinity:
                                          type: object
                                          x-kubernetes-preserve-unknown-fields: true
                                        annotations:
                                          additionalProperties:
                                            type: string
                                          type: object
                                        backupLocationName:
                                          type: string
                                        cell:
                                          maxLength: 63
                                          minLength: 1
                                          pattern: ^[A-Za-z0-9]([_.A-Za-z0-9]*[A-Za-z0-9])?$
                                          type: string
                                        dataVolumeClaimTemplate:
                                          properties:
                                            accessModes:
                                              items:
                                                type: string
                                              type: array
                                            resources:
                                              properties:
                                                limits:
                                                  additionalProperties:
                                                    anyOf:
                                                    - type: integer
                                                    - type: string
                                                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                    x-kubernetes-int-or-string: true
                                                  type: object
                                                requests:
                                                  additionalProperties:
                                                    anyOf:
                                                    - type: integer
                                                    - type: string
                                                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                    x-kubernetes-int-or-string: true
                                                  type: object
                                              type: object
                                            selector:
                                              properties:
                                                matchExpressions:
                                                  items:
                                                    properties:
                                                      key:
                                                        type: string
                                                      operator:
                                                        type: string
                                                      values:
                                                        items:
                                                          type: string
                                                        type: array
                                                    required:
                                                    - key
                                                    - operator
                                                    type: object
                                                  type: array
                                                matchLabels:
                                                  additionalProperties:
                                                    type: string
                                                  type: object
                                              type: object
                                            storageClassName:
                                              type: string
                                            volumeMode:
                                              type: string
                                            volumeName:
                                              type: string
                                          type: object
                                        externalDatastore:
                                          properties:
                                            credentialsSecret:
                                              properties:
                                                key:
                                                  type: string
                                                name:
                                                  type: string
                                                volumeName:
                                                  type: string
                                              required:
                                              - key
                                              type: object
                                            database:
                                              type: string
                                            host:
                                              type: string
                                            port:
                                              format: int32
                                              maximum: 65535
                                              minimum: 1
                                              type: integer
                                            serverCACertSecret:
                                              properties:
                                                key:
                                                  type: string
                                                name:
                                                  type: string
                                                volumeName:
                                                  type: string
                                              required:
                                              - key
                                              type: object
                                            user:
                                              type: string
                                          required:
                                          - credentialsSecret
                                          - database
                                          - host
                                          - port
                                          - user
                                          type: object
                                        extraEnv:
                                          items:
                                            properties:
                                              name:
                                                type: string
                                              value:
                                                type: string
                                              valueFrom:
                                                properties:
                                                  configMapKeyRef:
                                                    properties:
                                                      key:
                                                        type: string
                                                      name:
                                                        type: string
                                                      optional:
                                                        type: boolean
                                                    required:
                                                    - key
                                                    type: object
                                                  fieldRef:
                                                    properties:
                                                      apiVersion:
                                                        type: string
                                                      fieldPath:
                                                        type: string
                                                    required:
                                                    - fieldPath
                                                    type: object
                                                  resourceFieldRef:
                                                    properties:
                                                      containerName:
                                                        type: string
                                                      divisor:
                                                        anyOf:
                                                        - type: integer
                                                        - type: string
                                                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                        x-kubernetes-int-or-string: true
                                                      resource:
                                                        type: string
                                                    required:
                                                    - resource
                                                    type: object
                                                  secretKeyRef:
                                                    properties:
                                                      key:
                                                        type: string
                                                      name:
                                                        type: string
                                                      optional:
                                                        type: boolean
                                                    required:
                                                    - key
                                                    type: object
                                                type: object
                                            required:
                                            - name
                                            type: object
                                          type: array
                                        extraLabels:
                                          additionalProperties:
                                            type: string
                                          type: object
                                        extraVolumeMounts:
                                          items:
                                            properties:
                                              mountPath:
                                                type: string
                                              mountPropagation:
                                                type: string
                                              name:
                                                type: string
                                              readOnly:
                                                type: boolean
                                              subPath:
                                                type: string
                                              subPathExpr:
                                                type: string
                                            required:
                                            - mountPath
                                            - name
                                            type: object
                                          type: array
                                        extraVolumes:
                                          items:
                                            required:
                                            - name
                                            type: object
                                            x-kubernetes-preserve-unknown-fields: true
                                          type: array
                                        initContainers:
                                          items:
                                            required:
                                            - name
                                            type: object
                                            x-kubernetes-preserve-unknown-fields: true
                                          type: array
                                        mysqld:
                                          properties:
                                            configOverrides:
                                              type: string
                                            resources:
                                              properties:
                                                limits:
                                                  additionalProperties:
                                                    anyOf:
                                                    - type: integer
                                                    - type: string
                                                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                    x-kubernetes-int-or-string: true
                                                  type: object
                                                requests:
                                                  additionalProperties:
                                                    anyOf:
                                                    - type: integer
                                                    - type: string
                                                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                    x-kubernetes-int-or-string: true
                                                  type: object
                                              type: object
                                          required:
                                          - resources
                                          type: object
                                        replicas:
                                          format: int32
                                          minimum: 0
                                          type: integer
                                        sidecarContainers:
                                          items:
                                            required:
                                            - name
                                            type: object
                                            x-kubernetes-preserve-unknown-fields: true
                                          type: array
                                        tolerations:
                                          items:
                                            type: object
                                            x-kubernetes-preserve-unknown-fields: true
                                          type: array
                                        topologySpreadConstraints:
                                          items:
                                            required:
                                            - maxSkew
                                            - topologyKey
                                            - whenUnsatisfiable
                                            type: object
                                            x-kubernetes-preserve-unknown-fields: true
                                          type: array
                                        type:
                                          enum:
                                          - replica
                                          - rdonly
                                          - externalmaster
                                          - externalreplica
                                          - externalrdonly
                                          type: string
                                        vttablet:
                                          properties:
                                            extraFlags:
                                              additionalProperties:
                                                type: string
                                              type: object
                                            lifecycle:
                                              type: object
                                              x-kubernetes-preserve-unknown-fields: true
                                            resources:
                                              properties:
                                                limits:
                                                  additionalProperties:
                                                    anyOf:
                                                    - type: integer
                                                    - type: string
                                                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                    x-kubernetes-int-or-string: true
                                                  type: object
                                                requests:
                                                  additionalProperties:
                                                    anyOf:
                                                    - type: integer
                                                    - type: string
                                                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                    x-kubernetes-int-or-string: true
                                                  type: object
                                              type: object
                                          required:
                                          - resources
                                          type: object
                                      required:
                                      - cell
                                      - replicas
                                      - type
                                      - vttablet
                                      type: object
                                    type: array
                                    x-kubernetes-list-map-keys:
                                    - type
                                    - cell
                                    x-kubernetes-list-type: map
                                required:
                                - databaseInitScriptSecret
                                - keyRange
                                type: object
                              type: array
                          required:
                          - shards
                          type: object
                        equal:
                          properties:
                            parts:
                              format: int32
                              minimum: 1
                              type: integer
                            shardTemplate:
                              properties:
                                annotations:
                                  additionalProperties:
                                    type: string
                                  type: object
                                databaseInitScriptSecret:
                                  properties:
                                    key:
                                      type: string
                                    name:
                                      type: string
                                    volumeName:
                                      type: string
                                  required:
                                  - key
                                  type: object
                                replication:
                                  properties:
                                    enforceSemiSync:
                                      type: boolean
                                    initializeBackup:
                                      type: boolean
                                    initializeMaster:
                                      type: boolean
                                    recoverRestartedMaster:
                                      type: boolean
                                  type: object
                                tabletPools:
                                  items:
                                    properties:
                                      affinity:
                                        type: object
                                        x-kubernetes-preserve-unknown-fields: true
                                      annotations:
                                        additionalProperties:
                                          type: string
                                        type: object
                                      backupLocationName:
                                        type: string
                                      cell:
                                        maxLength: 63
                                        minLength: 1
                                        pattern: ^[A-Za-z0-9]([_.A-Za-z0-9]*[A-Za-z0-9])?$
                                        type: string
                                      dataVolumeClaimTemplate:
                                        properties:
                                          accessModes:
                                            items:
                                              type: string
                                            type: array
                                          resources:
                                            properties:
                                              limits:
                                                additionalProperties:
                                                  anyOf:
                                                  - type: integer
                                                  - type: string
                                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                  x-kubernetes-int-or-string: true
                                                type: object
                                              requests:
                                                additionalProperties:
                                                  anyOf:
                                                  - type: integer
                                                  - type: string
                                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                  x-kubernetes-int-or-string: true
                                                type: object
                                            type: object
                                          selector:
                                            properties:
                                              matchExpressions:
                                                items:
                                                  properties:
                                                    key:
                                                      type: string
                                                    operator:
                                                      type: string
                                                    values:
                                                      items:
                                                        type: string
                                                      type: array
                                                  required:
                                                  - key
                                                  - operator
                                                  type: object
                                                type: array
                                              matchLabels:
                                                additionalProperties:
                                                  type: string
                                                type: object
                                            type: object
                                          storageClassName:
                                            type: string
                                          volumeMode:
                                            type: string
                                          volumeName:
                                            type: string
                                        type: object
                                      externalDatastore:
                                        properties:
                                          credentialsSecret:
                                            properties:
                                              key:
                                                type: string
                                              name:
                                                type: string
                                              volumeName:
                                                type: string
                                            required:
                                            - key
                                            type: object
                                          database:
                                            type: string
                                          host:
                                            type: string
                                          port:
                                            format: int32
                                            maximum: 65535
                                            minimum: 1
                                            type: integer
                                          serverCACertSecret:
                                            properties:
                                              key:
                                                type: string
                                              name:
                                                type: string
                                              volumeName:
                                                type: string
                                            required:
                                            - key
                                            type: object
                                          user:
                                            type: string
                                        required:
                                        - credentialsSecret
                                        - database
                                        - host
                                        - port
                                        - user
                                        type: object
                                      extraEnv:
                                        items:
                                          properties:
                                            name:
                                              type: string
                                            value:
                                              type: string
                                            valueFrom:
                                              properties:
                                                configMapKeyRef:
                                                  properties:
                                                    key:
                                                      type: string
                                                    name:
                                                      type: string
                                                    optional:
                                                      type: boolean
                                                  required:
                                                  - key
                                                  type: object
                                                fieldRef:
                                                  properties:
                                                    apiVersion:
                                                      type: string
                                                    fieldPath:
                                                      type: string
                                                  required:
                                                  - fieldPath
                                                  type: object
                                                resourceFieldRef:
                                                  properties:
                                                    containerName:
                                                      type: string
                                                    divisor:
                                                      anyOf:
                                                      - type: integer
                                                      - type: string
                                                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                      x-kubernetes-int-or-string: true
                                                    resource:
                                                      type: string
                                                  required:
                                                  - resource
                                                  type: object
                                                secretKeyRef:
                                                  properties:
                                                    key:
                                                      type: string
                                                    name:
                                                      type: string
                                                    optional:
                                                      type: boolean
                                                  required:
                                                  - key
                                                  type: object
                                              type: object
                                          required:
                                          - name
                                          type: object
                                        type: array
                                      extraLabels:
                                        additionalProperties:
                                          type: string
                                        type: object
                                      extraVolumeMounts:
                                        items:
                                          properties:
                                            mountPath:
                                              type: string
                                            mountPropagation:
                                              type: string
                                            name:
                                              type: string
                                            readOnly:
                                              type: boolean
                                            subPath:
                                              type: string
                                            subPathExpr:
                                              type: string
                                          required:
                                          - mountPath
                                          - name
                                          type: object
                                        type: array
                                      extraVolumes:
                                        items:
                                          required:
                                          - name
                                          type: object
                                          x-kubernetes-preserve-unknown-fields: true
                                        type: array
                                      initContainers:
                                        items:
                                          required:
                                          - name
                                          type: object
                                          x-kubernetes-preserve-unknown-fields: true
                                        type: array
                                      mysqld:
                                        properties:
                                          configOverrides:
                                            type: string
                                          resources:
                                            properties:
                                              limits:
                                                additionalProperties:
                                                  anyOf:
                                                  - type: integer
                                                  - type: string
                                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                  x-kubernetes-int-or-string: true
                                                type: object
                                              requests:
                                                additionalProperties:
                                                  anyOf:
                                                  - type: integer
                                                  - type: string
                                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                  x-kubernetes-int-or-string: true
                                                type: object
                                            type: object
                                        required:
                                        - resources
                                        type: object
                                      replicas:
                                        format: int32
                                        minimum: 0
                                        type: integer
                                      sidecarContainers:
                                        items:
                                          required:
                                          - name
                                          type: object
                                          x-kubernetes-preserve-unknown-fields: true
                                        type: array
                                      tolerations:
                                        items:
                                          type: object
                                          x-kubernetes-preserve-unknown-fields: true
                                        type: array
                                      topologySpreadConstraints:
                                        items:
                                          required:
                                          - maxSkew
                                          - topologyKey
                                          - whenUnsatisfiable
                                          type: object
                                          x-kubernetes-preserve-unknown-fields: true
                                        type: array
                                      type:
                                        enum:
                                        - replica
                                        - rdonly
                                        - externalmaster
                                        - externalreplica
                                        - externalrdonly
                                        type: string
                                      vttablet:
                                        properties:
                                          extraFlags:
                                            additionalProperties:
                                              type: string
                                            type: object
                                          lifecycle:
                                            type: object
                                            x-kubernetes-preserve-unknown-fields: true
                                          resources:
                                            properties:
                                              limits:
                                                additionalProperties:
                                                  anyOf:
                                                  - type: integer
                                                  - type: string
                                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                  x-kubernetes-int-or-string: true
                                                type: object
                                              requests:
                                                additionalProperties:
                                                  anyOf:
                                                  - type: integer
                                                  - type: string
                                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                  x-kubernetes-int-or-string: true
                                                type: object
                                            type: object
                                        required:
                                        - resources
                                        type: object
                                    required:
                                    - cell
                                    - replicas
                                    - type
                                    - vttablet
                                    type: object
                                  type: array
                                  x-kubernetes-list-map-keys:
                                  - type
                                  - cell
                                  x-kubernetes-list-type: map
                              required:
                              - databaseInitScriptSecret
                              type: object
                          required:
                          - parts
                          type: object
                      type: object
                    maxItems: 2
                    minItems: 1
                    type: array
                  turndownPolicy:
                    enum:
                    - RequireIdle
                    - Immediate
                    type: string
                  vitessOrchestrator:
                    properties:
                      affinity:
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      annotations:
                        additionalProperties:
                          type: string
                        type: object
                      configSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                      extraEnv:
                        items:
                          properties:
                            name:
                              type: string
                            value:
                              type: string
                            valueFrom:
                              properties:
                                configMapKeyRef:
                                  properties:
                                    key:
                                      type: string
                                    name:
                                      type: string
                                    optional:
                                      type: boolean
                                  required:
                                  - key
                                  type: object
                                fieldRef:
                                  properties:
                                    apiVersion:
                                      type: string
                                    fieldPath:
                                      type: string
                                  required:
                                  - fieldPath
                                  type: object
                                resourceFieldRef:
                                  properties:
                                    containerName:
                                      type: string
                                    divisor:
                                      anyOf:
                                      - type: integer
                                      - type: string
                                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                      x-kubernetes-int-or-string: true
                                    resource:
                                      type: string
                                  required:
                                  - resource
                                  type: object
                                secretKeyRef:
                                  properties:
                                    key:
                                      type: string
                                    name:
                                      type: string
                                    optional:
                                      type: boolean
                                  required:
                                  - key
                                  type: object
                              type: object
                          required:
                          - name
                          type: object
                        type: array
                      extraFlags:
                        additionalProperties:
                          type: string
                        type: object
                      extraLabels:
                        additionalProperties:
                          type: string
                        type: object
                      extraVolumeMounts:
                        items:
                          properties:
                            mountPath:
                              type: string
                            mountPropagation:
                              type: string
                            name:
                              type: string
                            readOnly:
                              type: boolean
                            subPath:
                              type: string
                            subPathExpr:
                              type: string
                          required:
                          - mountPath
                          - name
                          type: object
                        type: array
                      extraVolumes:
                        items:
                          required:
                          - name
                          type: object
                          x-kubernetes-preserve-unknown-fields: true
                        type: array
                      initContainers:
                        items:
                          required:
                          - name
                          type: object
                          x-kubernetes-preserve-unknown-fields: true
                        type: array
                      resources:
                        properties:
                          limits:
                            additionalProperties:
                              anyOf:
                              - type: integer
                              - type: string
                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                              x-kubernetes-int-or-string: true
                            type: object
                          requests:
                            additionalProperties:
                              anyOf:
                              - type: integer
                              - type: string
                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                              x-kubernetes-int-or-string: true
                            type: object
                        type: object
                      service:
                        properties:
                          annotations:
                            additionalProperties:
                              type: string
                            type: object
                          clusterIP:
                            type: string
                        type: object
                      sidecarContainers:
                        items:
                          required:
                          - name
                          type: object
                          x-kubernetes-preserve-unknown-fields: true
                        type: array
                      tolerations:
                        items:
                          type: object
                          x-kubernetes-preserve-unknown-fields: true
                        type: array
                    required:
                    - configSecret
                    type: object
                required:
                - name
                - partitionings
                type: object
              type: array
            tabletService:
              properties:
                annotations:
                  additionalProperties:
                    type: string
                  type: object
                clusterIP:
                  type: string
              type: object
            topologyReconciliation:
              properties:
                pruneCells:
                  type: boolean
                pruneKeyspaces:
                  type: boolean
                pruneShardCells:
                  type: boolean
                pruneShards:
                  type: boolean
                pruneSrvKeyspaces:
                  type: boolean
                pruneTablets:
                  type: boolean
                registerCells:
                  type: boolean
                registerCellsAliases:
                  type: boolean
              type: object
            updateStrategy:
              properties:
                external:
                  properties:
                    allowResourceChanges:
                      items:
                        type: string
                      type: array
                  type: object
                type:
                  enum:
                  - External
                  - Immediate
                  type: string
              type: object
            vitessDashboard:
              properties:
                affinity:
                  type: object
                  x-kubernetes-preserve-unknown-fields: true
                annotations:
                  additionalProperties:
                    type: string
                  type: object
                cells:
                  items:
                    type: string
                  type: array
                extraEnv:
                  items:
                    properties:
                      name:
                        type: string
                      value:
                        type: string
                      valueFrom:
                        properties:
                          configMapKeyRef:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              optional:
                                type: boolean
                            required:
                            - key
                            type: object
                          fieldRef:
                            properties:
                              apiVersion:
                                type: string
                              fieldPath:
                                type: string
                            required:
                            - fieldPath
                            type: object
                          resourceFieldRef:
                            properties:
                              containerName:
                                type: string
                              divisor:
                                anyOf:
                                - type: integer
                                - type: string
                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                x-kubernetes-int-or-string: true
                              resource:
                                type: string
                            required:
                            - resource
                            type: object
                          secretKeyRef:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              optional:
                                type: boolean
                            required:
                            - key
                            type: object
                        type: object
                    required:
                    - name
                    type: object
                  type: array
                extraFlags:
                  additionalProperties:
                    type: string
                  type: object
                extraLabels:
                  additionalProperties:
                    type: string
                  type: object
                extraVolumeMounts:
                  items:
                    properties:
                      mountPath:
                        type: string
                      mountPropagation:
                        type: string
                      name:
                        type: string
                      readOnly:
                        type: boolean
                      subPath:
                        type: string
                      subPathExpr:
                        type: string
                    required:
                    - mountPath
                    - name
                    type: object
                  type: array
                extraVolumes:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                initContainers:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                replicas:
                  format: int32
                  type: integer
                resources:
                  properties:
                    limits:
                      additionalProperties:
                        anyOf:
                        - type: integer
                        - type: string
                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                        x-kubernetes-int-or-string: true
                      type: object
                    requests:
                      additionalProperties:
                        anyOf:
                        - type: integer
                        - type: string
                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                        x-kubernetes-int-or-string: true
                      type: object
                  type: object
                service:
                  properties:
                    annotations:
                      additionalProperties:
                        type: string
                      type: object
                    clusterIP:
                      type: string
                  type: object
                sidecarContainers:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                tolerations:
                  items:
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
              type: object
          required:
          - cells
          type: object
        status:
          properties:
            cells:
              additionalProperties:
                properties:
                  gatewayAvailable:
                    type: string
                  pendingChanges:
                    type: string
                type: object
              type: object
            gatewayServiceName:
              type: string
            globalLockserver:
              properties:
                etcd:
                  properties:
                    available:
                      type: string
                    clientServiceName:
                      type: string
                    observedGeneration:
                      format: int64
                      type: integer
                  type: object
              type: object
            keyspaces:
              additionalProperties:
                properties:
                  cells:
                    items:
                      type: string
                    type: array
                  desiredShards:
                    format: int32
                    type: integer
                  desiredTablets:
                    format: int32
                    type: integer
                  pendingChanges:
                    type: string
                  readyShards:
                    format: int32
                    type: integer
                  readyTablets:
                    format: int32
                    type: integer
                  shards:
                    format: int32
                    type: integer
                  tablets:
                    format: int32
                    type: integer
                  updatedShards:
                    format: int32
                    type: integer
                  updatedTablets:
                    format: int32
                    type: integer
                type: object
              type: object
            observedGeneration:
              format: int64
              type: integer
            orphanedCells:
              additionalProperties:
                properties:
                  message:
                    type: string
                  reason:
                    type: string
                required:
                - message
                - reason
                type: object
              type: object
            orphanedKeyspaces:
              additionalProperties:
                properties:
                  message:
                    type: string
                  reason:
                    type: string
                required:
                - message
                - reason
                type: object
              type: object
            vitessDashboard:
              properties:
                available:
                  type: string
                serviceName:
                  type: string
              type: object
          type: object
      type: object
  version: v2
  versions:
  - name: v2
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.0
  creationTimestamp: null
  name: vitesskeyspaces.planetscale.com
spec:
  group: planetscale.com
  names:
    kind: VitessKeyspace
    listKind: VitessKeyspaceList
    plural: vitesskeyspaces
    shortNames:
    - vtk
    singular: vitesskeyspace
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata:
          type: object
        spec:
          properties:
            annotations:
              additionalProperties:
                type: string
              type: object
            backupEngine:
              type: string
            backupLocations:
              items:
                properties:
                  annotations:
                    additionalProperties:
                      type: string
                    type: object
                  azblob:
                    properties:
                      account:
                        minLength: 1
                        type: string
                      authSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                      container:
                        minLength: 1
                        type: string
                      keyPrefix:
                        maxLength: 256
                        pattern: ^[^\r\n]*$
                        type: string
                    required:
                    - account
                    - authSecret
                    - container
                    type: object
                  ceph:
                    properties:
                      authSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                    required:
                    - authSecret
                    type: object
                  gcs:
                    properties:
                      authSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                      bucket:
                        minLength: 1
                        type: string
                      keyPrefix:
                        maxLength: 256
                        pattern: ^[^\r\n]*$
                        type: string
                    required:
                    - bucket
                    type: object
                  name:
                    maxLength: 63
                    pattern: ^[A-Za-z0-9]([A-Za-z0-9-_.]*[A-Za-z0-9])?$
                    type: string
                  s3:
                    properties:
                      authSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                      bucket:
                        minLength: 1
                        type: string
                      endpoint:
                        type: string
                      keyPrefix:
                        maxLength: 256
                        pattern: ^[^\r\n]*$
                        type: string
                      region:
                        minLength: 1
                        type: string
                    required:
                    - bucket
                    - region
                    type: object
                  volume:
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  volumeSubPath:
                    type: string
                type: object
              type: array
            databaseName:
              type: string
            extraVitessFlags:
              additionalProperties:
                type: string
              type: object
            globalLockserver:
              properties:
                address:
                  type: string
                implementation:
                  type: string
                rootPath:
                  type: string
              required:
              - address
              - implementation
              - rootPath
              type: object
            imagePullPolicies:
              properties:
                mysqld:
                  type: string
                mysqldExporter:
                  type: string
                vtbackup:
                  type: string
                vtctld:
                  type: string
                vtgate:
                  type: string
                vtorc:
                  type: string
                vttablet:
                  type: string
              type: object
            imagePullSecrets:
              items:
                properties:
                  name:
                    type: string
                type: object
              type: array
            images:
              properties:
                mysqld:
                  properties:
                    mariadb103Compatible:
                      type: string
                    mariadbCompatible:
                      type: string
                    mysql56Compatible:
                      type: string
                    mysql80Compatible:
                      type: string
                  type: object
                mysqldExporter:
                  type: string
                vtbackup:
                  type: string
                vtorc:
                  type: string
                vttablet:
                  type: string
              type: object
            name:
              maxLength: 63
              minLength: 1
              pattern: ^[A-Za-z0-9]([A-Za-z0-9-_.]*[A-Za-z0-9])?$
              type: string
            partitionings:
              items:
                properties:
                  custom:
                    properties:
                      shards:
                        items:
                          properties:
                            annotations:
                              additionalProperties:
                                type: string
                              type: object
                            databaseInitScriptSecret:
                              properties:
                                key:
                                  type: string
                                name:
                                  type: string
                                volumeName:
                                  type: string
                              required:
                              - key
                              type: object
                            keyRange:
                              properties:
                                end:
                                  pattern: ^([0-9a-f][0-9a-f])*$
                                  type: string
                                start:
                                  pattern: ^([0-9a-f][0-9a-f])*$
                                  type: string
                              type: object
                            replication:
                              properties:
                                enforceSemiSync:
                                  type: boolean
                                initializeBackup:
                                  type: boolean
                                initializeMaster:
                                  type: boolean
                                recoverRestartedMaster:
                                  type: boolean
                              type: object
                            tabletPools:
                              items:
                                properties:
                                  affinity:
                                    type: object
                                    x-kubernetes-preserve-unknown-fields: true
                                  annotations:
                                    additionalProperties:
                                      type: string
                                    type: object
                                  backupLocationName:
                                    type: string
                                  cell:
                                    maxLength: 63
                                    minLength: 1
                                    pattern: ^[A-Za-z0-9]([_.A-Za-z0-9]*[A-Za-z0-9])?$
                                    type: string
                                  dataVolumeClaimTemplate:
                                    properties:
                                      accessModes:
                                        items:
                                          type: string
                                        type: array
                                      resources:
                                        properties:
                                          limits:
                                            additionalProperties:
                                              anyOf:
                                              - type: integer
                                              - type: string
                                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                              x-kubernetes-int-or-string: true
                                            type: object
                                          requests:
                                            additionalProperties:
                                              anyOf:
                                              - type: integer
                                              - type: string
                                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                              x-kubernetes-int-or-string: true
                                            type: object
                                        type: object
                                      selector:
                                        properties:
                                          matchExpressions:
                                            items:
                                              properties:
                                                key:
                                                  type: string
                                                operator:
                                                  type: string
                                                values:
                                                  items:
                                                    type: string
                                                  type: array
                                              required:
                                              - key
                                              - operator
                                              type: object
                                            type: array
                                          matchLabels:
                                            additionalProperties:
                                              type: string
                                            type: object
                                        type: object
                                      storageClassName:
                                        type: string
                                      volumeMode:
                                        type: string
                                      volumeName:
                                        type: string
                                    type: object
                                  externalDatastore:
                                    properties:
                                      credentialsSecret:
                                        properties:
                                          key:
                                            type: string
                                          name:
                                            type: string
                                          volumeName:
                                            type: string
                                        required:
                                        - key
                                        type: object
                                      database:
                                        type: string
                                      host:
                                        type: string
                                      port:
                                        format: int32
                                        maximum: 65535
                                        minimum: 1
                                        type: integer
                                      serverCACertSecret:
                                        properties:
                                          key:
                                            type: string
                                          name:
                                            type: string
                                          volumeName:
                                            type: string
                                        required:
                                        - key
                                        type: object
                                      user:
                                        type: string
                                    required:
                                    - credentialsSecret
                                    - database
                                    - host
                                    - port
                                    - user
                                    type: object
                                  extraEnv:
                                    items:
                                      properties:
                                        name:
                                          type: string
                                        value:
                                          type: string
                                        valueFrom:
                                          properties:
                                            configMapKeyRef:
                                              properties:
                                                key:
                                                  type: string
                                                name:
                                                  type: string
                                                optional:
                                                  type: boolean
                                              required:
                                              - key
                                              type: object
                                            fieldRef:
                                              properties:
                                                apiVersion:
                                                  type: string
                                                fieldPath:
                                                  type: string
                                              required:
                                              - fieldPath
                                              type: object
                                            resourceFieldRef:
                                              properties:
                                                containerName:
                                                  type: string
                                                divisor:
                                                  anyOf:
                                                  - type: integer
                                                  - type: string
                                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                  x-kubernetes-int-or-string: true
                                                resource:
                                                  type: string
                                              required:
                                              - resource
                                              type: object
                                            secretKeyRef:
                                              properties:
                                                key:
                                                  type: string
                                                name:
                                                  type: string
                                                optional:
                                                  type: boolean
                                              required:
                                              - key
                                              type: object
                                          type: object
                                      required:
                                      - name
                                      type: object
                                    type: array
                                  extraLabels:
                                    additionalProperties:
                                      type: string
                                    type: object
                                  extraVolumeMounts:
                                    items:
                                      properties:
                                        mountPath:
                                          type: string
                                        mountPropagation:
                                          type: string
                                        name:
                                          type: string
                                        readOnly:
                                          type: boolean
                                        subPath:
                                          type: string
                                        subPathExpr:
                                          type: string
                                      required:
                                      - mountPath
                                      - name
                                      type: object
                                    type: array
                                  extraVolumes:
                                    items:
                                      required:
                                      - name
                                      type: object
                                      x-kubernetes-preserve-unknown-fields: true
                                    type: array
                                  initContainers:
                                    items:
                                      required:
                                      - name
                                      type: object
                                      x-kubernetes-preserve-unknown-fields: true
                                    type: array
                                  mysqld:
                                    properties:
                                      configOverrides:
                                        type: string
                                      resources:
                                        properties:
                                          limits:
                                            additionalProperties:
                                              anyOf:
                                              - type: integer
                                              - type: string
                                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                              x-kubernetes-int-or-string: true
                                            type: object
                                          requests:
                                            additionalProperties:
                                              anyOf:
                                              - type: integer
                                              - type: string
                                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                              x-kubernetes-int-or-string: true
                                            type: object
                                        type: object
                                    required:
                                    - resources
                                    type: object
                                  replicas:
                                    format: int32
                                    minimum: 0
                                    type: integer
                                  sidecarContainers:
                                    items:
                                      required:
                                      - name
                                      type: object
                                      x-kubernetes-preserve-unknown-fields: true
                                    type: array
                                  tolerations:
                                    items:
                                      type: object
                                      x-kubernetes-preserve-unknown-fields: true
                                    type: array
                                  topologySpreadConstraints:
                                    items:
                                      required:
                                      - maxSkew
                                      - topologyKey
                                      - whenUnsatisfiable
                                      type: object
                                      x-kubernetes-preserve-unknown-fields: true
                                    type: array
                                  type:
                                    enum:
                                    - replica
                                    - rdonly
                                    - externalmaster
                                    - externalreplica
                                    - externalrdonly
                                    type: string
                                  vttablet:
                                    properties:
                                      extraFlags:
                                        additionalProperties:
                                          type: string
                                        type: object
                                      lifecycle:
                                        type: object
                                        x-kubernetes-preserve-unknown-fields: true
                                      resources:
                                        properties:
                                          limits:
                                            additionalProperties:
                                              anyOf:
                                              - type: integer
                                              - type: string
                                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                              x-kubernetes-int-or-string: true
                                            type: object
                                          requests:
                                            additionalProperties:
                                              anyOf:
                                              - type: integer
                                              - type: string
                                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                              x-kubernetes-int-or-string: true
                                            type: object
                                        type: object
                                    required:
                                    - resources
                                    type: object
                                required:
                                - cell
                                - replicas
                                - type
                                - vttablet
                                type: object
                              type: array
                              x-kubernetes-list-map-keys:
                              - type
                              - cell
                              x-kubernetes-list-type: map
                          required:
                          - databaseInitScriptSecret
                          - keyRange
                          type: object
                        type: array
                    required:
                    - shards
                    type: object
                  equal:
                    properties:
                      parts:
                        format: int32
                        minimum: 1
                        type: integer
                      shardTemplate:
                        properties:
                          annotations:
                            additionalProperties:
                              type: string
                            type: object
                          databaseInitScriptSecret:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              volumeName:
                                type: string
                            required:
                            - key
                            type: object
                          replication:
                            properties:
                              enforceSemiSync:
                                type: boolean
                              initializeBackup:
                                type: boolean
                              initializeMaster:
                                type: boolean
                              recoverRestartedMaster:
                                type: boolean
                            type: object
                          tabletPools:
                            items:
                              properties:
                                affinity:
                                  type: object
                                  x-kubernetes-preserve-unknown-fields: true
                                annotations:
                                  additionalProperties:
                                    type: string
                                  type: object
                                backupLocationName:
                                  type: string
                                cell:
                                  maxLength: 63
                                  minLength: 1
                                  pattern: ^[A-Za-z0-9]([_.A-Za-z0-9]*[A-Za-z0-9])?$
                                  type: string
                                dataVolumeClaimTemplate:
                                  properties:
                                    accessModes:
                                      items:
                                        type: string
                                      type: array
                                    resources:
                                      properties:
                                        limits:
                                          additionalProperties:
                                            anyOf:
                                            - type: integer
                                            - type: string
                                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                            x-kubernetes-int-or-string: true
                                          type: object
                                        requests:
                                          additionalProperties:
                                            anyOf:
                                            - type: integer
                                            - type: string
                                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                            x-kubernetes-int-or-string: true
                                          type: object
                                      type: object
                                    selector:
                                      properties:
                                        matchExpressions:
                                          items:
                                            properties:
                                              key:
                                                type: string
                                              operator:
                                                type: string
                                              values:
                                                items:
                                                  type: string
                                                type: array
                                            required:
                                            - key
                                            - operator
                                            type: object
                                          type: array
                                        matchLabels:
                                          additionalProperties:
                                            type: string
                                          type: object
                                      type: object
                                    storageClassName:
                                      type: string
                                    volumeMode:
                                      type: string
                                    volumeName:
                                      type: string
                                  type: object
                                externalDatastore:
                                  properties:
                                    credentialsSecret:
                                      properties:
                                        key:
                                          type: string
                                        name:
                                          type: string
                                        volumeName:
                                          type: string
                                      required:
                                      - key
                                      type: object
                                    database:
                                      type: string
                                    host:
                                      type: string
                                    port:
                                      format: int32
                                      maximum: 65535
                                      minimum: 1
                                      type: integer
                                    serverCACertSecret:
                                      properties:
                                        key:
                                          type: string
                                        name:
                                          type: string
                                        volumeName:
                                          type: string
                                      required:
                                      - key
                                      type: object
                                    user:
                                      type: string
                                  required:
                                  - credentialsSecret
                                  - database
                                  - host
                                  - port
                                  - user
                                  type: object
                                extraEnv:
                                  items:
                                    properties:
                                      name:
                                        type: string
                                      value:
                                        type: string
                                      valueFrom:
                                        properties:
                                          configMapKeyRef:
                                            properties:
                                              key:
                                                type: string
                                              name:
                                                type: string
                                              optional:
                                                type: boolean
                                            required:
                                            - key
                                            type: object
                                          fieldRef:
                                            properties:
                                              apiVersion:
                                                type: string
                                              fieldPath:
                                                type: string
                                            required:
                                            - fieldPath
                                            type: object
                                          resourceFieldRef:
                                            properties:
                                              containerName:
                                                type: string
                                              divisor:
                                                anyOf:
                                                - type: integer
                                                - type: string
                                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                                x-kubernetes-int-or-string: true
                                              resource:
                                                type: string
                                            required:
                                            - resource
                                            type: object
                                          secretKeyRef:
                                            properties:
                                              key:
                                                type: string
                                              name:
                                                type: string
                                              optional:
                                                type: boolean
                                            required:
                                            - key
                                            type: object
                                        type: object
                                    required:
                                    - name
                                    type: object
                                  type: array
                                extraLabels:
                                  additionalProperties:
                                    type: string
                                  type: object
                                extraVolumeMounts:
                                  items:
                                    properties:
                                      mountPath:
                                        type: string
                                      mountPropagation:
                                        type: string
                                      name:
                                        type: string
                                      readOnly:
                                        type: boolean
                                      subPath:
                                        type: string
                                      subPathExpr:
                                        type: string
                                    required:
                                    - mountPath
                                    - name
                                    type: object
                                  type: array
                                extraVolumes:
                                  items:
                                    required:
                                    - name
                                    type: object
                                    x-kubernetes-preserve-unknown-fields: true
                                  type: array
                                initContainers:
                                  items:
                                    required:
                                    - name
                                    type: object
                                    x-kubernetes-preserve-unknown-fields: true
                                  type: array
                                mysqld:
                                  properties:
                                    configOverrides:
                                      type: string
                                    resources:
                                      properties:
                                        limits:
                                          additionalProperties:
                                            anyOf:
                                            - type: integer
                                            - type: string
                                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                            x-kubernetes-int-or-string: true
                                          type: object
                                        requests:
                                          additionalProperties:
                                            anyOf:
                                            - type: integer
                                            - type: string
                                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                            x-kubernetes-int-or-string: true
                                          type: object
                                      type: object
                                  required:
                                  - resources
                                  type: object
                                replicas:
                                  format: int32
                                  minimum: 0
                                  type: integer
                                sidecarContainers:
                                  items:
                                    required:
                                    - name
                                    type: object
                                    x-kubernetes-preserve-unknown-fields: true
                                  type: array
                                tolerations:
                                  items:
                                    type: object
                                    x-kubernetes-preserve-unknown-fields: true
                                  type: array
                                topologySpreadConstraints:
                                  items:
                                    required:
                                    - maxSkew
                                    - topologyKey
                                    - whenUnsatisfiable
                                    type: object
                                    x-kubernetes-preserve-unknown-fields: true
                                  type: array
                                type:
                                  enum:
                                  - replica
                                  - rdonly
                                  - externalmaster
                                  - externalreplica
                                  - externalrdonly
                                  type: string
                                vttablet:
                                  properties:
                                    extraFlags:
                                      additionalProperties:
                                        type: string
                                      type: object
                                    lifecycle:
                                      type: object
                                      x-kubernetes-preserve-unknown-fields: true
                                    resources:
                                      properties:
                                        limits:
                                          additionalProperties:
                                            anyOf:
                                            - type: integer
                                            - type: string
                                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                            x-kubernetes-int-or-string: true
                                          type: object
                                        requests:
                                          additionalProperties:
                                            anyOf:
                                            - type: integer
                                            - type: string
                                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                            x-kubernetes-int-or-string: true
                                          type: object
                                      type: object
                                  required:
                                  - resources
                                  type: object
                              required:
                              - cell
                              - replicas
                              - type
                              - vttablet
                              type: object
                            type: array
                            x-kubernetes-list-map-keys:
                            - type
                            - cell
                            x-kubernetes-list-type: map
                        required:
                        - databaseInitScriptSecret
                        type: object
                    required:
                    - parts
                    type: object
                type: object
              maxItems: 2
              minItems: 1
              type: array
            topologyReconciliation:
              properties:
                pruneCells:
                  type: boolean
                pruneKeyspaces:
                  type: boolean
                pruneShardCells:
                  type: boolean
                pruneShards:
                  type: boolean
                pruneSrvKeyspaces:
                  type: boolean
                pruneTablets:
                  type: boolean
                registerCells:
                  type: boolean
                registerCellsAliases:
                  type: boolean
              type: object
            turndownPolicy:
              enum:
              - RequireIdle
              - Immediate
              type: string
            updateStrategy:
              properties:
                external:
                  properties:
                    allowResourceChanges:
                      items:
                        type: string
                      type: array
                  type: object
                type:
                  enum:
                  - External
                  - Immediate
                  type: string
              type: object
            vitessOrchestrator:
              properties:
                affinity:
                  type: object
                  x-kubernetes-preserve-unknown-fields: true
                annotations:
                  additionalProperties:
                    type: string
                  type: object
                configSecret:
                  properties:
                    key:
                      type: string
                    name:
                      type: string
                    volumeName:
                      type: string
                  required:
                  - key
                  type: object
                extraEnv:
                  items:
                    properties:
                      name:
                        type: string
                      value:
                        type: string
                      valueFrom:
                        properties:
                          configMapKeyRef:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              optional:
                                type: boolean
                            required:
                            - key
                            type: object
                          fieldRef:
                            properties:
                              apiVersion:
                                type: string
                              fieldPath:
                                type: string
                            required:
                            - fieldPath
                            type: object
                          resourceFieldRef:
                            properties:
                              containerName:
                                type: string
                              divisor:
                                anyOf:
                                - type: integer
                                - type: string
                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                x-kubernetes-int-or-string: true
                              resource:
                                type: string
                            required:
                            - resource
                            type: object
                          secretKeyRef:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              optional:
                                type: boolean
                            required:
                            - key
                            type: object
                        type: object
                    required:
                    - name
                    type: object
                  type: array
                extraFlags:
                  additionalProperties:
                    type: string
                  type: object
                extraLabels:
                  additionalProperties:
                    type: string
                  type: object
                extraVolumeMounts:
                  items:
                    properties:
                      mountPath:
                        type: string
                      mountPropagation:
                        type: string
                      name:
                        type: string
                      readOnly:
                        type: boolean
                      subPath:
                        type: string
                      subPathExpr:
                        type: string
                    required:
                    - mountPath
                    - name
                    type: object
                  type: array
                extraVolumes:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                initContainers:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                resources:
                  properties:
                    limits:
                      additionalProperties:
                        anyOf:
                        - type: integer
                        - type: string
                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                        x-kubernetes-int-or-string: true
                      type: object
                    requests:
                      additionalProperties:
                        anyOf:
                        - type: integer
                        - type: string
                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                        x-kubernetes-int-or-string: true
                      type: object
                  type: object
                service:
                  properties:
                    annotations:
                      additionalProperties:
                        type: string
                      type: object
                    clusterIP:
                      type: string
                  type: object
                sidecarContainers:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                tolerations:
                  items:
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
              required:
              - configSecret
              type: object
            zoneMap:
              additionalProperties:
                type: string
              type: object
          required:
          - globalLockserver
          - name
          - partitionings
          - zoneMap
          type: object
        status:
          properties:
            conditions:
              items:
                properties:
                  lastTransitionTime:
                    format: date-time
                    type: string
                  message:
                    type: string
                  reason:
                    type: string
                  status:
                    enum:
                    - "True"
                    - "False"
                    - Unknown
                    type: string
                  type:
                    type: string
                required:
                - status
                - type
                type: object
              type: array
            idle:
              type: string
            observedGeneration:
              format: int64
              type: integer
            orphanedShards:
              additionalProperties:
                properties:
                  message:
                    type: string
                  reason:
                    type: string
                required:
                - message
                - reason
                type: object
              type: object
            partitionings:
              items:
                properties:
                  desiredShards:
                    format: int32
                    type: integer
                  desiredTablets:
                    format: int32
                    type: integer
                  readyShards:
                    format: int32
                    type: integer
                  readyTablets:
                    format: int32
                    type: integer
                  servingWrites:
                    type: string
                  shardNames:
                    items:
                      type: string
                    type: array
                  tablets:
                    format: int32
                    type: integer
                  updatedTablets:
                    format: int32
                    type: integer
                type: object
              type: array
            resharding:
              properties:
                copyProgress:
                  type: integer
                sourceShards:
                  items:
                    type: string
                  type: array
                state:
                  type: string
                targetShards:
                  items:
                    type: string
                  type: array
                workflow:
                  type: string
              required:
              - state
              - workflow
              type: object
            shards:
              additionalProperties:
                properties:
                  cells:
                    items:
                      type: string
                    type: array
                  desiredTablets:
                    format: int32
                    type: integer
                  hasMaster:
                    type: string
                  pendingChanges:
                    type: string
                  readyTablets:
                    format: int32
                    type: integer
                  servingWrites:
                    type: string
                  tablets:
                    format: int32
                    type: integer
                  updatedTablets:
                    format: int32
                    type: integer
                type: object
              type: object
          type: object
      type: object
  version: v2
  versions:
  - name: v2
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.0
  creationTimestamp: null
  name: vitessshards.planetscale.com
spec:
  group: planetscale.com
  names:
    kind: VitessShard
    listKind: VitessShardList
    plural: vitessshards
    shortNames:
    - vts
    singular: vitessshard
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata:
          type: object
        spec:
          properties:
            annotations:
              additionalProperties:
                type: string
              type: object
            backupEngine:
              type: string
            backupLocations:
              items:
                properties:
                  annotations:
                    additionalProperties:
                      type: string
                    type: object
                  azblob:
                    properties:
                      account:
                        minLength: 1
                        type: string
                      authSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                      container:
                        minLength: 1
                        type: string
                      keyPrefix:
                        maxLength: 256
                        pattern: ^[^\r\n]*$
                        type: string
                    required:
                    - account
                    - authSecret
                    - container
                    type: object
                  ceph:
                    properties:
                      authSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                    required:
                    - authSecret
                    type: object
                  gcs:
                    properties:
                      authSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                      bucket:
                        minLength: 1
                        type: string
                      keyPrefix:
                        maxLength: 256
                        pattern: ^[^\r\n]*$
                        type: string
                    required:
                    - bucket
                    type: object
                  name:
                    maxLength: 63
                    pattern: ^[A-Za-z0-9]([A-Za-z0-9-_.]*[A-Za-z0-9])?$
                    type: string
                  s3:
                    properties:
                      authSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                      bucket:
                        minLength: 1
                        type: string
                      endpoint:
                        type: string
                      keyPrefix:
                        maxLength: 256
                        pattern: ^[^\r\n]*$
                        type: string
                      region:
                        minLength: 1
                        type: string
                    required:
                    - bucket
                    - region
                    type: object
                  volume:
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  volumeSubPath:
                    type: string
                type: object
              type: array
            databaseInitScriptSecret:
              properties:
                key:
                  type: string
                name:
                  type: string
                volumeName:
                  type: string
              required:
              - key
              type: object
            databaseName:
              type: string
            extraVitessFlags:
              additionalProperties:
                type: string
              type: object
            globalLockserver:
              properties:
                address:
                  type: string
                implementation:
                  type: string
                rootPath:
                  type: string
              required:
              - address
              - implementation
              - rootPath
              type: object
            imagePullPolicies:
              properties:
                mysqld:
                  type: string
                mysqldExporter:
                  type: string
                vtbackup:
                  type: string
                vtctld:
                  type: string
                vtgate:
                  type: string
                vtorc:
                  type: string
                vttablet:
                  type: string
              type: object
            imagePullSecrets:
              items:
                properties:
                  name:
                    type: string
                type: object
              type: array
            images:
              properties:
                mysqld:
                  properties:
                    mariadb103Compatible:
                      type: string
                    mariadbCompatible:
                      type: string
                    mysql56Compatible:
                      type: string
                    mysql80Compatible:
                      type: string
                  type: object
                mysqldExporter:
                  type: string
                vtbackup:
                  type: string
                vtorc:
                  type: string
                vttablet:
                  type: string
              type: object
            keyRange:
              properties:
                end:
                  pattern: ^([0-9a-f][0-9a-f])*$
                  type: string
                start:
                  pattern: ^([0-9a-f][0-9a-f])*$
                  type: string
              type: object
            name:
              type: string
            replication:
              properties:
                enforceSemiSync:
                  type: boolean
                initializeBackup:
                  type: boolean
                initializeMaster:
                  type: boolean
                recoverRestartedMaster:
                  type: boolean
              type: object
            tabletPools:
              items:
                properties:
                  affinity:
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  annotations:
                    additionalProperties:
                      type: string
                    type: object
                  backupLocationName:
                    type: string
                  cell:
                    maxLength: 63
                    minLength: 1
                    pattern: ^[A-Za-z0-9]([_.A-Za-z0-9]*[A-Za-z0-9])?$
                    type: string
                  dataVolumeClaimTemplate:
                    properties:
                      accessModes:
                        items:
                          type: string
                        type: array
                      resources:
                        properties:
                          limits:
                            additionalProperties:
                              anyOf:
                              - type: integer
                              - type: string
                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                              x-kubernetes-int-or-string: true
                            type: object
                          requests:
                            additionalProperties:
                              anyOf:
                              - type: integer
                              - type: string
                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                              x-kubernetes-int-or-string: true
                            type: object
                        type: object
                      selector:
                        properties:
                          matchExpressions:
                            items:
                              properties:
                                key:
                                  type: string
                                operator:
                                  type: string
                                values:
                                  items:
                                    type: string
                                  type: array
                              required:
                              - key
                              - operator
                              type: object
                            type: array
                          matchLabels:
                            additionalProperties:
                              type: string
                            type: object
                        type: object
                      storageClassName:
                        type: string
                      volumeMode:
                        type: string
                      volumeName:
                        type: string
                    type: object
                  externalDatastore:
                    properties:
                      credentialsSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                      database:
                        type: string
                      host:
                        type: string
                      port:
                        format: int32
                        maximum: 65535
                        minimum: 1
                        type: integer
                      serverCACertSecret:
                        properties:
                          key:
                            type: string
                          name:
                            type: string
                          volumeName:
                            type: string
                        required:
                        - key
                        type: object
                      user:
                        type: string
                    required:
                    - credentialsSecret
                    - database
                    - host
                    - port
                    - user
                    type: object
                  extraEnv:
                    items:
                      properties:
                        name:
                          type: string
                        value:
                          type: string
                        valueFrom:
                          properties:
                            configMapKeyRef:
                              properties:
                                key:
                                  type: string
                                name:
                                  type: string
                                optional:
                                  type: boolean
                              required:
                              - key
                              type: object
                            fieldRef:
                              properties:
                                apiVersion:
                                  type: string
                                fieldPath:
                                  type: string
                              required:
                              - fieldPath
                              type: object
                            resourceFieldRef:
                              properties:
                                containerName:
                                  type: string
                                divisor:
                                  anyOf:
                                  - type: integer
                                  - type: string
                                  pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                  x-kubernetes-int-or-string: true
                                resource:
                                  type: string
                              required:
                              - resource
                              type: object
                            secretKeyRef:
                              properties:
                                key:
                                  type: string
                                name:
                                  type: string
                                optional:
                                  type: boolean
                              required:
                              - key
                              type: object
                          type: object
                      required:
                      - name
                      type: object
                    type: array
                  extraLabels:
                    additionalProperties:
                      type: string
                    type: object
                  extraVolumeMounts:
                    items:
                      properties:
                        mountPath:
                          type: string
                        mountPropagation:
                          type: string
                        name:
                          type: string
                        readOnly:
                          type: boolean
                        subPath:
                          type: string
                        subPathExpr:
                          type: string
                      required:
                      - mountPath
                      - name
                      type: object
                    type: array
                  extraVolumes:
                    items:
                      required:
                      - name
                      type: object
                      x-kubernetes-preserve-unknown-fields: true
                    type: array
                  initContainers:
                    items:
                      required:
                      - name
                      type: object
                      x-kubernetes-preserve-unknown-fields: true
                    type: array
                  mysqld:
                    properties:
                      configOverrides:
                        type: string
                      resources:
                        properties:
                          limits:
                            additionalProperties:
                              anyOf:
                              - type: integer
                              - type: string
                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                              x-kubernetes-int-or-string: true
                            type: object
                          requests:
                            additionalProperties:
                              anyOf:
                              - type: integer
                              - type: string
                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                              x-kubernetes-int-or-string: true
                            type: object
                        type: object
                    required:
                    - resources
                    type: object
                  replicas:
                    format: int32
                    minimum: 0
                    type: integer
                  sidecarContainers:
                    items:
                      required:
                      - name
                      type: object
                      x-kubernetes-preserve-unknown-fields: true
                    type: array
                  tolerations:
                    items:
                      type: object
                      x-kubernetes-preserve-unknown-fields: true
                    type: array
                  topologySpreadConstraints:
                    items:
                      required:
                      - maxSkew
                      - topologyKey
                      - whenUnsatisfiable
                      type: object
                      x-kubernetes-preserve-unknown-fields: true
                    type: array
                  type:
                    enum:
                    - replica
                    - rdonly
                    - externalmaster
                    - externalreplica
                    - externalrdonly
                    type: string
                  vttablet:
                    properties:
                      extraFlags:
                        additionalProperties:
                          type: string
                        type: object
                      lifecycle:
                        type: object
                        x-kubernetes-preserve-unknown-fields: true
                      resources:
                        properties:
                          limits:
                            additionalProperties:
                              anyOf:
                              - type: integer
                              - type: string
                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                              x-kubernetes-int-or-string: true
                            type: object
                          requests:
                            additionalProperties:
                              anyOf:
                              - type: integer
                              - type: string
                              pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                              x-kubernetes-int-or-string: true
                            type: object
                        type: object
                    required:
                    - resources
                    type: object
                required:
                - cell
                - replicas
                - type
                - vttablet
                type: object
              type: array
              x-kubernetes-list-map-keys:
              - type
              - cell
              x-kubernetes-list-type: map
            topologyReconciliation:
              properties:
                pruneCells:
                  type: boolean
                pruneKeyspaces:
                  type: boolean
                pruneShardCells:
                  type: boolean
                pruneShards:
                  type: boolean
                pruneSrvKeyspaces:
                  type: boolean
                pruneTablets:
                  type: boolean
                registerCells:
                  type: boolean
                registerCellsAliases:
                  type: boolean
              type: object
            updateStrategy:
              properties:
                external:
                  properties:
                    allowResourceChanges:
                      items:
                        type: string
                      type: array
                  type: object
                type:
                  enum:
                  - External
                  - Immediate
                  type: string
              type: object
            vitessOrchestrator:
              properties:
                affinity:
                  type: object
                  x-kubernetes-preserve-unknown-fields: true
                annotations:
                  additionalProperties:
                    type: string
                  type: object
                configSecret:
                  properties:
                    key:
                      type: string
                    name:
                      type: string
                    volumeName:
                      type: string
                  required:
                  - key
                  type: object
                extraEnv:
                  items:
                    properties:
                      name:
                        type: string
                      value:
                        type: string
                      valueFrom:
                        properties:
                          configMapKeyRef:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              optional:
                                type: boolean
                            required:
                            - key
                            type: object
                          fieldRef:
                            properties:
                              apiVersion:
                                type: string
                              fieldPath:
                                type: string
                            required:
                            - fieldPath
                            type: object
                          resourceFieldRef:
                            properties:
                              containerName:
                                type: string
                              divisor:
                                anyOf:
                                - type: integer
                                - type: string
                                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                                x-kubernetes-int-or-string: true
                              resource:
                                type: string
                            required:
                            - resource
                            type: object
                          secretKeyRef:
                            properties:
                              key:
                                type: string
                              name:
                                type: string
                              optional:
                                type: boolean
                            required:
                            - key
                            type: object
                        type: object
                    required:
                    - name
                    type: object
                  type: array
                extraFlags:
                  additionalProperties:
                    type: string
                  type: object
                extraLabels:
                  additionalProperties:
                    type: string
                  type: object
                extraVolumeMounts:
                  items:
                    properties:
                      mountPath:
                        type: string
                      mountPropagation:
                        type: string
                      name:
                        type: string
                      readOnly:
                        type: boolean
                      subPath:
                        type: string
                      subPathExpr:
                        type: string
                    required:
                    - mountPath
                    - name
                    type: object
                  type: array
                extraVolumes:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                initContainers:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                resources:
                  properties:
                    limits:
                      additionalProperties:
                        anyOf:
                        - type: integer
                        - type: string
                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                        x-kubernetes-int-or-string: true
                      type: object
                    requests:
                      additionalProperties:
                        anyOf:
                        - type: integer
                        - type: string
                        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                        x-kubernetes-int-or-string: true
                      type: object
                  type: object
                service:
                  properties:
                    annotations:
                      additionalProperties:
                        type: string
                      type: object
                    clusterIP:
                      type: string
                  type: object
                sidecarContainers:
                  items:
                    required:
                    - name
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
                tolerations:
                  items:
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                  type: array
              required:
              - configSecret
              type: object
            zoneMap:
              additionalProperties:
                type: string
              type: object
          required:
          - databaseInitScriptSecret
          - globalLockserver
          - images
          - keyRange
          - name
          - zoneMap
          type: object
        status:
          properties:
            backupLocations:
              items:
                properties:
                  completeBackups:
                    format: int32
                    type: integer
                  incompleteBackups:
                    format: int32
                    type: integer
                  latestCompleteBackupTime:
                    format: date-time
                    type: string
                  name:
                    type: string
                required:
                - completeBackups
                - incompleteBackups
                type: object
              type: array
            cells:
              items:
                type: string
              type: array
            conditions:
              additionalProperties:
                properties:
                  lastTransitionTime:
                    format: date-time
                    type: string
                  message:
                    type: string
                  reason:
                    type: string
                  status:
                    enum:
                    - "True"
                    - "False"
                    - Unknown
                    type: string
                required:
                - status
                type: object
              type: object
            hasInitialBackup:
              type: string
            hasMaster:
              type: string
            idle:
              type: string
            lowestPodGeneration:
              format: int64
              type: integer
            masterAlias:
              type: string
            observedGeneration:
              format: int64
              type: integer
            orphanedTablets:
              additionalProperties:
                properties:
                  message:
                    type: string
                  reason:
                    type: string
                required:
                - message
                - reason
                type: object
              type: object
            servingWrites:
              type: string
            tablets:
              additionalProperties:
                properties:
                  available:
                    type: string
                  dataVolumeBound:
                    type: string
                  index:
                    format: int32
                    type: integer
                  pendingChanges:
                    type: string
                  poolType:
                    type: string
                  ready:
                    type: string
                  running:
                    type: string
                  type:
                    type: string
                type: object
              type: object
            vitessOrchestrator:
              properties:
                available:
                  type: string
                serviceName:
                  type: string
              type: object
          type: object
      type: object
  version: v2
  versions:
  - name: v2
    served: true
    storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: vitess-operator
  namespace: operators-vitess-operator
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: vitess-operator
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - services
  - endpoints
  - persistentvolumeclaims
  - events
  - configmaps
  - secrets
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
- apiGroups:
  - apps
  resources:
  - deployments
  - daemonsets
  - replicasets
  - statefulsets
  verbs:
  - '*'
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - '*'
- apiGroups:
  - apps
  resourceNames:
  - vitess-operator
  resources:
  - deployments/finalizers
  verbs:
  - update
- apiGroups:
  - planetscale.com
  resources:
  - vitessclusters
  - vitessclusters/status
  - vitessclusters/finalizers
  - vitesscells
  - vitesscells/status
  - vitesscells/finalizers
  - vitesskeyspaces
  - vitesskeyspaces/status
  - vitesskeyspaces/finalizers
  - vitessshards
  - vitessshards/status
  - vitessshards/finalizers
  - etcdlockservers
  - etcdlockservers/status
  - etcdlockservers/finalizers
  - vitessbackups
  - vitessbackups/status
  - vitessbackups/finalizers
  - vitessbackupstorages
  - vitessbackupstorages/status
  - vitessbackupstorages/finalizers
  verbs:
  - '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: vitess-operator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: vitess-operator
subjects:
- kind: ServiceAccount
  name: vitess-operator
---
apiVersion: scheduling.k8s.io/v1beta1
description: Vitess components (vttablet, vtgate, vtctld, etcd)
globalDefault: false
kind: PriorityClass
metadata:
  name: vitess
value: 1000
---
apiVersion: scheduling.k8s.io/v1beta1
description: The vitess-operator control plane.
globalDefault: false
kind: PriorityClass
metadata:
  name: vitess-operator-control-plane
value: 5000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: vitess-operator
  namespace: operators-vitess-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      app: vitess-operator
  template:
    metadata:
      labels:
        app: vitess-operator
    spec:
      containers:
      - args:
        - --logtostderr
        - -v=4
        command:
        - vitess-operator
        env:
        - name: WATCH_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: PS_OPERATOR_POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: PS_OPERATOR_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: OPERATOR_NAME
          value: vitess-operator
        image: planetscale/vitess-operator:v2.5.0
        name: vitess-operator
        resources:
          limits:
            memory: 128Mi
          requests:
            cpu: 100m
            memory: 128Mi
      priorityClassName: vitess-operator-control-plane
      serviceAccountName: vitess-operator

And that's my 101_initial_cluster.yaml:

# The following example is minimalist. The security policies
# and resource specifications are not meant to be used in production.
# Please refer to the operator documentation for recommendations on
# production settings.
apiVersion: planetscale.com/v2
kind: VitessCluster
metadata:
  name: example
  namespace: operators-vitess-example
spec:
  images:
    vtctld: vitess/lite:v12.0.0
    vtgate: vitess/lite:v12.0.0
    vttablet: vitess/lite:v12.0.0
    vtbackup: vitess/lite:v12.0.0
    mysqld:
      mysql56Compatible: vitess/lite:v12.0.0
    mysqldExporter: prom/mysqld-exporter:v0.11.0
  cells:
  - name: zone1
    gateway:
      authentication:
        static:
          secret:
            name: example-cluster-config
            key: users.json
      replicas: 1
      resources:
        requests:
          cpu: 100m
          memory: 256Mi
        limits:
          memory: 256Mi
  vitessDashboard:
    cells:
    - zone1
    extraFlags:
      security_policy: read-only
    replicas: 1
    resources:
      limits:
        memory: 128Mi
      requests:
        cpu: 100m
        memory: 128Mi

  keyspaces:
  - name: commerce
    turndownPolicy: Immediate
    partitionings:
    - equal:
        parts: 1
        shardTemplate:
          databaseInitScriptSecret:
            name: example-cluster-config
            key: init_db.sql
          replication:
            enforceSemiSync: false
          tabletPools:
          - cell: zone1
            type: replica
            replicas: 2
            vttablet:
              extraFlags:
                db_charset: utf8mb4
              resources:
                limits:
                  memory: 256Mi
                requests:
                  cpu: 100m
                  memory: 256Mi
            mysqld:
              resources:
                limits:
                  memory: 256Mi
                requests:
                  cpu: 100m
                  memory: 256Mi
            dataVolumeClaimTemplate:
              accessModes: ["ReadWriteOnce"]
              resources:
                requests:
                  storage: 10Gi
  updateStrategy:
    type: Immediate
---
apiVersion: v1
kind: Secret
metadata:
  name: example-cluster-config
  namespace: operators-vitess-example
type: Opaque
stringData:
  users.json: |
    {
      "user": [{
        "UserData": "user",
        "Password": ""
      }]
    }
  init_db.sql: |
    # This file is executed immediately after mysql_install_db,
    # to initialize a fresh data directory.

    ###############################################################################
    # Equivalent of mysql_secure_installation
    ###############################################################################

    # Changes during the init db should not make it to the binlog.
    # They could potentially create errant transactions on replicas.
    SET sql_log_bin = 0;
    # Remove anonymous users.
    DELETE FROM mysql.user WHERE User = '';

    # Disable remote root access (only allow UNIX socket).
    DELETE FROM mysql.user WHERE User = 'root' AND Host != 'localhost';

    # Remove test database.
    DROP DATABASE IF EXISTS test;

    ###############################################################################
    # Vitess defaults
    ###############################################################################

    # Vitess-internal database.
    CREATE DATABASE IF NOT EXISTS _vt;
    # Note that definitions of local_metadata and shard_metadata should be the same
    # as in production which is defined in go/vt/mysqlctl/metadata_tables.go.
    CREATE TABLE IF NOT EXISTS _vt.local_metadata (
      name VARCHAR(255) NOT NULL,
      value VARCHAR(255) NOT NULL,
      db_name VARBINARY(255) NOT NULL,
      PRIMARY KEY (db_name, name)
      ) ENGINE=InnoDB;
    CREATE TABLE IF NOT EXISTS _vt.shard_metadata (
      name VARCHAR(255) NOT NULL,
      value MEDIUMBLOB NOT NULL,
      db_name VARBINARY(255) NOT NULL,
      PRIMARY KEY (db_name, name)
      ) ENGINE=InnoDB;

    # Admin user with all privileges.
    CREATE USER 'vt_dba'@'localhost';
    GRANT ALL ON *.* TO 'vt_dba'@'localhost';
    GRANT GRANT OPTION ON *.* TO 'vt_dba'@'localhost';

    # User for app traffic, with global read-write access.
    CREATE USER 'vt_app'@'localhost';
    GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, FILE,
      REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
      LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW,
      SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER
      ON *.* TO 'vt_app'@'localhost';

    # User for app debug traffic, with global read access.
    CREATE USER 'vt_appdebug'@'localhost';
    GRANT SELECT, SHOW DATABASES, PROCESS ON *.* TO 'vt_appdebug'@'localhost';

    # User for administrative operations that need to be executed as non-SUPER.
    # Same permissions as vt_app here.
    CREATE USER 'vt_allprivs'@'localhost';
    GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, FILE,
      REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
      LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW,
      SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER
      ON *.* TO 'vt_allprivs'@'localhost';

    # User for slave replication connections.
    # TODO: Should we set a password on this since it allows remote connections?
    CREATE USER 'vt_repl'@'%';
    GRANT REPLICATION SLAVE ON *.* TO 'vt_repl'@'%';

    # User for Vitess filtered replication (binlog player).
    # Same permissions as vt_app.
    CREATE USER 'vt_filtered'@'localhost';
    GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, FILE,
      REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
      LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW,
      SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER
      ON *.* TO 'vt_filtered'@'localhost';

    # User for Orchestrator (https://github.com/openark/orchestrator).
    # TODO: Reenable when the password is randomly generated.
    #CREATE USER 'orc_client_user'@'%' IDENTIFIED BY 'orc_client_user_password';
    #GRANT SUPER, PROCESS, REPLICATION SLAVE, RELOAD
    #  ON *.* TO 'orc_client_user'@'%';
    #GRANT SELECT
    #  ON _vt.* TO 'orc_client_user'@'%';

    FLUSH PRIVILEGES;

    RESET SLAVE ALL;
    RESET MASTER;

If you search for "namespace:" you can see the diffs compared with the examples from the upstream.

Is it possible to manage clusters/run in other namespaces? I've found this: https://giters.com/vitessio/vitess/issues/8172

Thank you in advance!

Best regards,
Kai

missing backup args for vtctld

Following the discussion on Slack https://vitess.slack.com/archives/C0PQY0PTK/p1635093608043400.

vtctld is clearly missing the backup parameters like --s3_backup_aws_endpoint=s3.endpoint.com.

Document in README and websites the supported/recommended Kubernetes version

According to @enisoc on Slack:

the recommendation of k8s 1.14 matches the pinned 2.0.0 version of the operator that the vitess docs currently use. if you use the latest operator from master, we recommend k8s 1.16 or something +/- 1 minor version.

i.e. 1.15 through 1.17

I don't see this documented on the README, or on the Vitess or PlanetScale websites.

Please also document the support policy schedule for Kubernetes versions:

When will a Kubernetes version start being supported?
When will a Kubernetes version start being the primary recommendation / the version that the operator tests against.
When will a Kubernetes version stop being supported? (e.g. stop acting on compatibility issues, actively tell people to avoid that version, and/or known incompatibilities may start being introduced)
For the clouds that you explicitly support (GKE, EKS, etc.), will you guarantee that you support at least one Kubernetes version that overlaps with the versions they offer?

Ability to set database's user authorization for allowing access to specific tables

Vitess topic described in here: https://vitess.io/docs/user-guides/configuration-advanced/authorization/

I was unable to find a description for vitess-operator to do that.

Annotations not being removed on VTTablets

When updating annotations for a keyspace:

....
          annotations:
            prometheus.io/path: /metrics
            prometheus.io/port: "15000"
            prometheus.io/scrape: "true"
          backupLocationName: gcs
          cell: useast4b0
          dataVolumeClaimTemplate:
....

to remove:

....
          annotations:
            prometheus.io/path: /metrics
            prometheus.io/scrape: "true"
          backupLocationName: gcs
          cell: useast4b0
          dataVolumeClaimTemplate:
....

The VTTablet pods end up in an inconsistent state and retain the old annotation:

...
  annotations:
....
    drain.planetscale.com/supported: ensure that the tablet is not a master
    planetscale.com/observed-shard-generation: "8"
    prometheus.io/path: /metrics
    prometheus.io/port: "15000"
    prometheus.io/scrape: "true"
...

and even after removing them manually, the operator will try to bring it back:

  annotations:
...
    drain.planetscale.com/supported: ensure that the tablet is not a master
    planetscale.com/observed-shard-generation: "8"
    prometheus.io/path: /metrics
    prometheus.io/scrape: "true"
    rollout.planetscale.com/scheduled: |
      metadata:
        annotations:
          prometheus.io/port: "15000"

Panic in pkg/operator/vttablet/pvc.go

A user has reported a panic with this stack trace:

1 runtime.go:69] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:76
/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:65
/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:51
/usr/local/go/src/runtime/panic.go:679
/usr/local/go/src/runtime/panic.go:199
/usr/local/go/src/runtime/signal_unix.go:394
/go/src/planetscale.dev/vitess-operator/pkg/operator/vttablet/pvc.go:34
/go/src/planetscale.dev/vitess-operator/pkg/controller/vitessshard/reconcile_backup_job.go:156
/go/src/planetscale.dev/vitess-operator/pkg/operator/reconciler/object.go:166
/go/src/planetscale.dev/vitess-operator/pkg/operator/reconciler/object_set.go:87
/go/src/planetscale.dev/vitess-operator/pkg/controller/vitessshard/reconcile_backup_job.go:152
/go/src/planetscale.dev/vitess-operator/pkg/controller/vitessshard/vitessshard_controller.go:198
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:215
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:158
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:134
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:88
/usr/local/go/src/runtime/asm_amd64.s:1357
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1b0bf43]

Is there a schedule to support newer Kubernetes versions > 1.17?

Given that the Helm chart has been deprecated I decided the operator was a better alternative. We are using Kubernetes 1.18.6 and the operator does not work in our cluster. I happened upon a compatibility link to this project in slack and discovered the latest version of Kubernetes vitess is compatible with is 1.17. Are there specific plans to make the operator compatible with newer Kubernetes versions.

BTW, it would had been useful if Kubernetes version compatibility information was in the vitessio/vitess project, directly or via a link to this project.

Add an option to use k8s as the topology server

We recently merged the k8s topology into Vitess. This allows Vitess to run without requiring a dedicated topology service. It's actually so useful that it's the default for the 2.X Helm charts.

It should be pretty easy to update the operator to support the k8s topo. It will just need to create the following extra resources:

The vitesstopologynode crd (could be done during operator installation)
ServiceAccounts for all component pods that need to use the topology.
RBAC for the ServiceAccounts that allow them full access to vitesstopologynodes
Set the right topo flags, it should just be -topo_implementation=k8s the root doesn't change and the server_address doesn't matter because the components will use their in-cluster kubeconfig to detect everything else by default.

Resharding Workflow Status Updates

We need the following status updates pulled from the new Workflow Show command:

Status indicating which shards are actively serving (MasterIsServing field from Workflow Show return)

example: servingShards: -80,80-

Status showing whether we are Copying or Running for a given workflow.

example: workflowState: Copying

Status boolean field to indicate whether we have gone above or below an acceptable max vreplication lag to allow for switching writes. As of now that acceptable value is 10s. We could allow configuring this value in the future, but for v1, let's just stick to this hard coded value.

example: replicationLagUnsafe: true

Blockers

Item 3 is currently blocked until this PR is merged: vitessio/vitess#6635

panic when backupLocation is nil

Hi all!

I'm seeing the following crash trying to bootstrap a cluster using an externally managed MySQL

time="2020-08-19T20:33:11Z" level=info msg="Reconciling VitessCluster" VitessCluster=a controller=VitessCluster namespace=vitess
time="2020-08-19T20:33:11Z" level=info msg="Reconciling EtcdLockserver" controller=EtcdLockserver etcdlockserver=a-etcd-cf3bf24b namespace=vitess
time="2020-08-19T20:33:11Z" level=info msg="Reconciling VitessCell" controller=VitessCell namespace=vitess vitesscell=a-useast1-aaa38ac7
time="2020-08-19T20:33:11Z" level=info msg="Reconciling VitessKeyspace" controller=VitessKeyspace namespace=vitess vitesskeyspace=a-a-d182b884
time="2020-08-19T20:33:11Z" level=info msg="Reconciling VitessKeyspace" controller=VitessKeyspace namespace=vitess vitesskeyspace=a-actiontext-d8e43c31
time="2020-08-19T20:33:11Z" level=info msg="connecting to Vitess topology server" address="a-etcd-cf3bf24b-client:2379" component=toposerver.connpool implementation=etcd2 rootPath=/vitess/a/global
time="2020-08-19T20:33:11Z" level=info msg="successfully connected to Vitess topology server" address="a-etcd-cf3bf24b-client:2379" component=toposerver.connpool implementation=etcd2 rootPath=/vitess/a/global
E0819 20:33:11.557558       1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
goroutine 669 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1fac740, 0x3b64de0)
	/Users/mkent/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:74 +0xa3
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/Users/mkent/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:48 +0x82
panic(0x1fac740, 0x3b64de0)
	/usr/local/Cellar/go/1.14.3/libexec/src/runtime/panic.go:969 +0x166
planetscale.dev/vitess-operator/pkg/controller/vitessshard.vttabletSpecs(0xc001115000, 0xc00062ec90, 0x23158b1, 0x15, 0xc001033618)
	vitess-operator/pkg/controller/vitessshard/reconcile_tablets.go:309 +0x4d2
planetscale.dev/vitess-operator/pkg/controller/vitessshard.(*ReconcileVitessShard).reconcileTablets(0xc00099c440, 0x27b1140, 0xc000fba480, 0xc001115000, 0x0, 0x0, 0x0, 0x0)
	vitess-operator/pkg/controller/vitessshard/reconcile_tablets.go:82 +0x380
planetscale.dev/vitess-operator/pkg/controller/vitessshard.(*ReconcileVitessShard).Reconcile(0xc00099c440, 0xc000f18968, 0x6, 0xc000df8aa0, 0x1e, 0x0, 0x0, 0x0, 0x0)
	vitess-operator/pkg/controller/vitessshard/vitessshard_controller.go:184 +0x68a
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0008a9e00, 0x2058dc0, 0xc000c5be20, 0x1061800)
	/Users/mkent/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:256 +0x161
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0008a9e00, 0x0)
	/Users/mkent/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:232 +0xae
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc0008a9e00)
	/Users/mkent/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:211 +0x2b
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc00058e240)
	/Users/mkent/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:152 +0x5f
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00058e240, 0x3b9aca00, 0x0, 0x1000000023f0101, 0xc00054ad20)
	/Users/mkent/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:153 +0xf8
k8s.io/apimachinery/pkg/util/wait.Until(0xc00058e240, 0x3b9aca00, 0xc00054ad20)
	/Users/mkent/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:88 +0x4d
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
	/Users/mkent/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:193 +0x305
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x1c75a72]

looks to be the issue hinted at in https://github.com/planetscale/vitess-operator/pull/116/files#r472334564

How Does Partitionings->Custom->Shards-KeyRange Work?

Another newbie question:
If key range is defined as follows:
- keyRange:
start: "40"
end: "80"
What ends up being created is that a single range without -40 or 80-. Does -Start End- need to be defined?
My first goal is to be able to add more shards to an existing keyspace via the operator. If I add another cell, how does the operator know which keyspace to put the shard in?
My next goal is to be able to add shards AND extend the key range. For example add new shards in 80-c0 . Does keyRange support something like:
- keyRange:
start: "40"
end: "80"
start: "80"
end: "c0"

AFAICT, the partitionings->equal does NOT support what I want to accomplish because it sets keyspace. ex -80 80- My understanding is the use of "-" prevents me from doing this??

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.