Comments (7)
use tag release-1.12.0
from flink-connector-clickhouse.
and, sink to clickhouse, lost some data
But when I use datastream jdbc connector , No data loss occurred
from flink-connector-clickhouse.
This info is not enough for me to find the problem.
If it happens accidentally, it may be caused by dirty data (I'm not sure).
This exception occurs during sending data to the server-side in clickhouse-jdbc.jar
, you'd better do some breakpoint tests, where the exception occurred:
from flink-connector-clickhouse.
I tried three test cases.
case 1: use datastream jdbc connector;
successed sink to clickhouse 9408 datas
case 2: use tag release-1.12.0
successed sink to clickhouse 8578 datas, lost 830 datas , no exception occurred
I still can't find the reason for the loss of data.
case 3: use this source code (https://github.com/liekkassmile/flink-connector-clickhouse)
successed sink to clickhouse 9408 datas
from flink-connector-clickhouse.
OK, can you email or WX me to talk some details about this?
from flink-connector-clickhouse.
WX:snake_kong
from flink-connector-clickhouse.
Tag: release-1.12.0 has a thread-safe bug that causes this problem, bugfix commit: bb4f078, 3451515
Use branch release-1.12
to avoid this issue and I have deleted the tag to prevent more people encounter this.
from flink-connector-clickhouse.
Related Issues (20)
- flinkcdc 同步到clickhouse 任务运行一段时间后报错 HOT 2
- 配置properties.*属性,报错 HOT 1
- Contribute Clickhouse Connector To Apache Flink HOT 10
- Support Flink 1.17
- Support ClickHouse JDBC 0.6.0 HOT 1
- properties.* HOT 1
- 使用请教 HOT 5
- 我使用sql连接clickhouse查询的时候也出现了java.lang.ClassNotFoundException: org.apache.flink.connector.clickhouse.internal.ClickHouseBatchInputFormat HOT 1
- jar 包放进flink lib 目录出现classnotfind HOT 6
- flink cdc to clickhouse HOT 1
- 请问要在flink/lib目录放哪些包才可以运行呢? HOT 1
- clickhouse表中有bool类型,FlinkSQL读取报错? 请问这个该如何解决呢? HOT 1
- 有不有考虑实现折叠的功能 HOT 1
- 内存溢出
- 似乎写入速度很慢 HOT 8
- `select count(field) from table` failed when using flink table api HOT 4
- java.lang.ClassNotFoundException: org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat HOT 3
- [Feature request] Support lookup join in Flink 1.15 HOT 1
- flink job完成之后,ck端数据还没有插入完成 HOT 2
- sink.update-strategy不生效 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flink-connector-clickhouse.