Customers, Products, and Suppliers do not flow to Google Big Query about confluent-hybrid-cloud-workshop

[chadmott]

@tmcgrath I suspect this is why you were getting the error in Data Studio... the queries are joining on IDs that (at least for me) do not exist

[chadmott]

Quick update --- adding the ID fields to the tables in BigQuery, and then re-starting the connector has data flowing in. Seems for whatever reason the ID does not exist in the Schemas...

{
  "connect.name": "io.confluent.ksql.avro_schemas.KsqlDataSourceSchema",
  "fields": [
    {
      "default": null,
      "name": "FIRST_NAME",
      "type": [
        "null",
        "string"
      ]
    },
    {
      "default": null,
      "name": "LAST_NAME",
      "type": [
        "null",
        "string"
      ]
    },
    {
      "default": null,
      "name": "EMAIL",
      "type": [
        "null",
        "string"
      ]
    },
    {
      "default": null,
      "name": "CITY",
      "type": [
        "null",
        "string"
      ]
    },
    {
      "default": null,
      "name": "COUNTRY",
      "type": [
        "null",
        "string"
      ]
    },
    {
      "default": null,
      "name": "SOURCEDC",
      "type": [
        "null",
        "string"
      ]
    }
  ],
  "name": "KsqlDataSourceSchema",
  "namespace": "io.confluent.ksql.avro_schemas",
  "type": "record"
}

if there is no schema here, it makes sense why it wouldn't show up in BigQuery but... why then did the data flow after I manually added the ID to big query?