Playground for Kafka/Confluent Docker experimentations...
βΉοΈ How to run
- π Kafka Connectors
- βοΈ Confluent Cloud
- π Confluent Replicator and Mirror Maker 2
- π Environments
- Confluent Commercial
- CP-Ansible Playground
- πΎ Other Playgrounds
- π Useful Resources
Quick start examples from Confluent docs but in Docker version for ease of use.
Connector | Product Category | Latest Version (*) | Type | Release Date | ![]() |
---|---|---|---|---|---|
![]() |
Message Queue | 1.1.6 | Confluent Subscription | 2020-08-21 | β |
![]() |
Message Queue | 5.5.2 | Confluent Subscription | 2020-09-30 | β |
Analytics | 1.0.4 | Confluent Subscription | 2020-06-03 | β | |
Analytics | 1.1.2 | Confluent Subscription | 2020-06-08 | β | |
Database | 1.1.2 | Confluent Subscription | 2020-10-07 | β | |
Message Queue | 1.3.0 | Confluent Subscription | 2020-08-31 | β | |
![]() |
Data Warehouse | 1.0.2 | Confluent Subscription | 2020-04-09 | β |
![]() |
Data Warehouse | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Datastore | 5.5.2 | Confluent Community License | 2020-09-30 | β |
![]() |
Datastore | 1.3.2 | Confluent Subscription | 2020-06-26 | β |
Message Queue | 1.0.6 | Confluent Subscription | 2020-09-23 | β | |
![]() |
Message Queue | 1.0.0-preview | Confluent Subscription | 2020-07-08 | β |
![]() |
Database | 1.0.1 | Confluent Subscription | 2020-01-07 | β |
![]() |
Database | 1.0.1 | Confluent Subscription | 2020-01-07 | β |
SaaS Apps | 1.0.4 | Confluent Subscription | 2020-10-01 | β | |
![]() |
Datastore | 1.5.0 | Confluent Subscription | 2020-10-07 | β |
![]() |
Datastore | 1.3.2 | Confluent Subscription | 2020-06-26 | β |
![]() |
Datastore | 1.5.0 | Confluent Subscription | 2020-10-07 | β |
![]() |
Datastore | 1.5.0 | Confluent Subscription | 2020-10-07 | β |
![]() |
Message Queue | 1.0.3 | Confluent Subscription | 2020-03-28 | β |
![]() |
SaaS Apps | 1.0.7 | Confluent Subscription | 2020-08-31 | β |
![]() |
Analytics | 1.0.3 | Confluent Subscription | 2020-10-08 | β |
![]() |
Message Queue | 1.1.0 | Confluent Subscription | 2020-03-13 | β |
![]() |
Data Warehouse | 1.0.4 | Confluent Subscription | 2020-10-08 | β |
![]() |
Database | 1.2.2 | Confluent Subscription | 2020-06-19 | β |
Database | 3.4.8 | Open Source (Couchbase) | β | ||
Database | 3.4.8 | Open Source (Couchbase) | β | ||
![]() |
CDC | 1.2.2 | Open Source (Debezium Community) | β | |
![]() |
CDC | 1.2.2 | Open Source (Debezium Community) | β | |
![]() |
CDC | 1.2.2 | Open Source (Debezium Community) | β | |
![]() |
CDC | 1.2.2 | Open Source (Debezium Community) | β | |
![]() |
Logs | 1.1.1 | Confluent Subscription | 2019-10-18 | β |
![]() |
Logs | 1.1.1 | Confluent Subscription | 2019-10-18 | β |
![]() |
Analytics | 1.1.2 | Confluent Subscription | 2020-06-08 | β |
![]() |
Analytics | 10.0.1 | Confluent Community License | 2020-10-04 | β |
![]() |
Datastore | 1.0.3-preview | Confluent Subscription | 2020-10-01 | β |
![]() |
Datastore | 1.0.3-preview | Confluent Subscription | 2020-10-01 | β |
![]() |
Database | 1.0.1 | Confluent Subscription | 2019-11-19 | β |
![]() |
SaaS Apps | 1.0.1 | Confluent Subscription | 2020-06-05 | β |
![]() |
Data Warehouse | 1.6.1 | Open Source (WePay) | β | |
![]() |
Database | 1.0.5 | Confluent Subscription | 2020-07-20 | β |
![]() |
SaaS Apps | 1.1.1 | Confluent Subscription | 2020-09-22 | β |
![]() |
Message Queue | 1.0.3 | Confluent Subscription | 2020-10-06 | β |
![]() |
Database | 1.0.2 | Confluent Subscription | 2019-10-30 | β |
![]() |
Datastore | 5.5.3 | Confluent Subscription | 2020-10-01 | β |
![]() |
Datastore | 1.3.2 | Confluent Subscription | 2020-06-26 | β |
Database | 1.1.1 | Confluent Subscription | 2020-01-07 | β | |
Database | 1.1.1 | Confluent Subscription | 2020-01-07 | β | |
![]() |
Database | 1.0.5 | Confluent Subscription | 2020-07-20 | β |
Datastore | 1.3.2 | Confluent Subscription | 2020-06-26 | β | |
Datastore | 1.3.2 | Confluent Subscription | 2020-06-26 | β | |
Datastore | 5.5.2 | Confluent Community License | 2020-09-30 | β | |
Datastore | 1.0.5 | Confluent Subscription | 2020-03-04 | β | |
![]() |
SaaS Apps | 1.0.16 | Confluent Subscription | 2020-08-03 | β |
![]() |
Message Queue | 1.3.0 | Confluent Subscription | 2020-09-15 | β |
![]() |
Message Queue | 5.5.2 | Confluent Subscription | 2020-09-30 | β |
Database | 1.2.1 | Confluent Subscription | 2020-09-30 | β | |
Database | 1.2.1 | Confluent Subscription | 2020-09-30 | β | |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
Database | 10.0.0 | Confluent Community License | 2020-10-07 | β |
![]() |
SaaS Apps | 1.0.0-preview | Confluent Subscription | 2020-03-30 | β |
![]() |
Message Queue | 1.3.0 | Confluent Subscription | 2020-09-15 | β |
![]() |
Message Queue | 1.3.0 | Confluent Subscription | 2020-09-15 | β |
![]() |
Message Queue | 1.3.0 | Confluent Subscription | 2020-09-15 | β |
![]() |
Message Queue | 5.5.2 | Confluent Subscription | 2020-09-30 | β |
![]() |
Datastore | 1.1.1 | Confluent Subscription | 2020-02-10 | β |
![]() |
Datastore | 5.5.2 | Confluent Community License | 2020-09-30 | β |
![]() |
Database | 1.2.0 | Open Source (MongoDB) | 2020-07-08 | β |
![]() |
Database | 1.2.0 | Open Source (MongoDB) | 2020-07-08 | β |
![]() |
IoT | 1.3.0 | Confluent Subscription | 2020-08-28 | β |
![]() |
IoT | 1.3.0 | Confluent Subscription | 2020-08-28 | β |
![]() |
Database | 1.0.9 | Open Source (Neo4j, Inc.) | 2020-09-02 | β |
![]() |
Database | 1.0.2 | Confluent Subscription | 2019-08-20 | β |
![]() |
SaaS Apps | 1.0.1 | Confluent Subscription | 2020-07-20 | β |
![]() |
Analytics | 1.1.2-preview | Confluent Subscription | 2020-06-08 | β |
Message Queue | 1.3.2-preview | Confluent Subscription | 2020-10-06 | β | |
Message Queue | 1.3.2 | Confluent Subscription | 2020-10-06 | β | |
![]() |
Database | 0.0.2.11 | Open Source (Jeremy Custenborder) | 2020-01-22 | β |
![]() |
SaaS Apps | 1.7.2 | Confluent Subscription | 2020-10-14 | β |
![]() |
SaaS Apps | 1.7.2 | Confluent Subscription | 2020-10-14 | β |
![]() |
SaaS Apps | 1.7.2 | Confluent Subscription | 2020-10-14 | β |
![]() |
SaaS Apps | 1.7.2 | Confluent Subscription | 2020-10-14 | β |
![]() |
SaaS Apps | 1.7.2 | Confluent Subscription | 2020-10-14 | β |
![]() |
SaaS Apps | 1.7.2 | Confluent Subscription | 2020-10-14 | β |
![]() |
SaaS Apps | 1.7.2 | Confluent Subscription | 2020-10-14 | β |
![]() |
SaaS Apps | 2.0.1 | Confluent Subscription | 2020-07-28 | β |
![]() |
SaaS Apps | 2.0.1 | Confluent Subscription | 2020-07-28 | β |
![]() |
Datastore | 2.1.2 | Confluent Subscription | 2020-10-07 | β |
![]() |
Datastore | 2.1.2 | Confluent Subscription | 2020-10-07 | β |
![]() |
IoT | 1.1.2 | Confluent Subscription | 2020-04-09 | β |
![]() |
Data Warehouse | 1.5.0 | Open Source (Snowflake, Inc.) | 2020-09-30 | β |
![]() |
Message Queue | 1.3.0 | Confluent Subscription | 2020-09-15 | β |
![]() |
Message Queue | 1.2.0 | Confluent Subscription | 2020-08-26 | β |
![]() |
Analytics | 2.0 | Open Source (Splunk) | β | |
![]() |
Analytics | 1.0.2 | Confluent Subscription | 2020-01-08 | β |
![]() |
Datastore | 2.0.46 | Open Source (Jeremy Custenborder) | 2020-08-26 | β |
![]() |
Logs | 1.3.1 | Confluent Subscription | 2020-07-22 | β |
![]() |
Message Queue | 1.3.0 | Confluent Subscription | 2020-09-15 | β |
![]() |
Message Queue | 1.2.0 | Confluent Subscription | 2020-08-26 | β |
![]() |
Database | 1.2.2 | Confluent Subscription | 2020-09-07 | β |
![]() |
SaaS Apps | 1.0.2 | Confluent Subscription | 2020-10-08 | β |
* You can change default connector version by setting CONNECTOR_TAG
environment variable before starting a test, get more details here
- How to connect your components to Confluent Cloud
- How to monitor your Confluent Cloud cluster
- How to restrict access
- etc...
ServiceNow source
ServiceNow sink
MongoDB source
Firebase
- Using cp-ansible with Confluent Cloud
- Demo using dabz/ccloudexporter in order to pull Metrics API data from Confluent Cloud cluster and export it to Prometheus (Grafana dashboard is also available)
.NET client (producer/consumer)
Go client (producer/consumer)
kafka-admin Managing topics and ACLs using matt-mangia/kafka-admin
Confluent Replicator OnPrem to cloud and Cloud to Cloud examples
Multi-Cluster Schema Registry with hybrid configuration (onprem/confluent cloud)
- Confluent REST Proxy Security Plugin with Principal Propagation
- Migrate Schemas to Confluent Cloud using Confluent Replicator
Using Multi-Data-Center setup with US
πΊπΈ and EUROPE
πͺπΊ clusters.
Using Confluent Replicator as connector
- Using PLAINTEXT
- Using SASL_PLAIN
- Using Kerberos
- πΎ Using Confluent Replicator as executable
- Using PLAINTEXT
- Using SASL_PLAIN
- Using Kerberos
Using Mirror Maker 2
- Using PLAINTEXT
Single cluster:
- PLAINTEXT: no security
- SASL_PLAIN: no SSL encryption, SASL/PLAIN authentication
- SASL/SCRAM no SSL encryption, SASL/SCRAM-SHA-256 authentication
- SASL_SSL: SSL encryption, SASL/PLAIN authentication
- 2WAY_SSL: SSL encryption, SSL authentication
- Kerberos: no SSL encryption, Kerberos GSSAPI authentication
- SSL_Kerberos SSL encryption, Kerberos GSSAPI authentication
- LDAP Authorizer with SASL/PLAIN no SSL encryption, SASL/PLAIN authentication, LDAP Authorizer for ACL authorization
- RBAC with SASL/PLAIN RBAC with no SSL encryption, SASL/PLAIN authentication
Multi-Data-Center setup:
- PLAINTEXT: no security
- SASL_PLAIN: no SSL encryption, SASL/PLAIN authentication
- Kerberos: no SSL encryption, Kerberos GSSAPI authentication
- Control Center
- Tiered Storage
- Tiered storage with AWS S3
- Tiered storage with Minio (unsupported)
- Confluent Rebalancer
- JMS Client
- RBAC with SASL/PLAIN RBAC with no SSL encryption, SASL/PLAIN authentication
- Audit Logs
- Confluent REST Proxy Security Plugin with SASL_SSL and 2WAY_SSL Principal Propagation
Easily play with Confluent Platform Ansible playbooks by using Ubuntu based Docker images generated daily from this cp-ansible-playground repository
There is also a Confluent Cloud version available here
- Confluent Replicator [also with SASL_SSL and 2WAY_SSL]
- Testing Separate principals (
connector.client.config.override.policy
) for Source connector (SFTP source) - Testing Separate principals (
connector.client.config.override.policy
) for Source connector (SFTP sink) - How to write logs to files when using docker-compose
- Publish logs to kafka with Elastic Filebeat
.NET basic producer
Monitor Confluent Platform with Datadog
- A Kafka Story π: A step by step guide to use Kafka ecosystem (Kafka Connect, KSQL, Java Consumers/Producers, etc..) with Docker
- Kafka Boom Boom π₯: An attempt to break kafka
- Kafka Security playbook π: demonstrates various security configurations with Docker
- MDC and single views π: Multi-Data-Center setup using Confluent Replicator
- Kafka Platform Prometheus π: Simple demo of how to monitor Kafka Platform using Prometheus and Grafana.