Comments (4)
flinkx版本tag r1.5.0 依然报错
./bin/flinkx -mode yarn -job /etlx/jobs/mysql_to_hdfs.json -plugin /flinkx/plugins -flinkconf /flink-1.5.0/conf -yarnconf /etc/hadoop
Exception in thread "main" java.lang.RuntimeException: Unable to get ClusterClient status from Application Client at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:183) at org.apache.flink.yarn.YarnClusterClient.waitForClusterToBeReady(YarnClusterClient.java:247) at org.apache.flink.client.program.ClusterClient.runDetached(ClusterClient.java:513) at org.apache.flink.yarn.YarnClusterClient.submitJob(YarnClusterClient.java:155) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:464) at org.apache.flink.client.program.DetachedEnvironment.finalizeExecute(DetachedEnvironment.java:77) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:410) at com.dtstack.flinkx.launcher.Launcher.main(Launcher.java:100) Caused by: org.apache.flink.util.FlinkException: Could not connect to the leading JobManager. Please check that the JobManager is running. at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:932) at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:178) ... 7 more Caused by: org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException: Could not retrieve the leader gateway. at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:83) at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:927) ... 8 more Caused by: java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds] at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223) at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227) at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190) at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) at scala.concurrent.Await$.result(package.scala:190) at scala.concurrent.Await.result(package.scala) at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:81) ... 9 more 15:19:51.609 [main-SendThread(10.3.2.64:2181)] DEBUG org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x26a2e3084f57b8c after 10ms
from chunjun.
flinkx版本tag r1.5.0 依然报错
./bin/flinkx -mode yarn -job /etlx/jobs/mysql_to_hdfs.json -plugin /flinkx/plugins -flinkconf /flink-1.5.0/conf -yarnconf /etc/hadoop
Exception in thread "main" java.lang.RuntimeException: Unable to get ClusterClient status from Application Client at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:183) at org.apache.flink.yarn.YarnClusterClient.waitForClusterToBeReady(YarnClusterClient.java:247) at org.apache.flink.client.program.ClusterClient.runDetached(ClusterClient.java:513) at org.apache.flink.yarn.YarnClusterClient.submitJob(YarnClusterClient.java:155) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:464) at org.apache.flink.client.program.DetachedEnvironment.finalizeExecute(DetachedEnvironment.java:77) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:410) at com.dtstack.flinkx.launcher.Launcher.main(Launcher.java:100) Caused by: org.apache.flink.util.FlinkException: Could not connect to the leading JobManager. Please check that the JobManager is running. at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:932) at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:178) ... 7 more Caused by: org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException: Could not retrieve the leader gateway. at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:83) at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:927) ... 8 more Caused by: java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds] at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223) at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227) at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190) at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) at scala.concurrent.Await$.result(package.scala:190) at scala.concurrent.Await.result(package.scala) at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:81) ... 9 more 15:19:51.609 [main-SendThread(10.3.2.64:2181)] DEBUG org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x26a2e3084f57b8c after 10ms
yarn上有没有启动一个flink session
from chunjun.
flinkx版本tag r1.5.0 依然报错
./bin/flinkx -mode yarn -job /etlx/jobs/mysql_to_hdfs.json -plugin /flinkx/plugins -flinkconf /flink-1.5.0/conf -yarnconf /etc/hadoop
Exception in thread "main" java.lang.RuntimeException: Unable to get ClusterClient status from Application Client at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:183) at org.apache.flink.yarn.YarnClusterClient.waitForClusterToBeReady(YarnClusterClient.java:247) at org.apache.flink.client.program.ClusterClient.runDetached(ClusterClient.java:513) at org.apache.flink.yarn.YarnClusterClient.submitJob(YarnClusterClient.java:155) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:464) at org.apache.flink.client.program.DetachedEnvironment.finalizeExecute(DetachedEnvironment.java:77) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:410) at com.dtstack.flinkx.launcher.Launcher.main(Launcher.java:100) Caused by: org.apache.flink.util.FlinkException: Could not connect to the leading JobManager. Please check that the JobManager is running. at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:932) at org.apache.flink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClient.java:178) ... 7 more Caused by: org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException: Could not retrieve the leader gateway. at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:83) at org.apache.flink.client.program.ClusterClient.getJobManagerGateway(ClusterClient.java:927) ... 8 more Caused by: java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds] at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223) at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227) at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190) at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) at scala.concurrent.Await$.result(package.scala:190) at scala.concurrent.Await.result(package.scala) at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:81) ... 9 more 15:19:51.609 [main-SendThread(10.3.2.64:2181)] DEBUG org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x26a2e3084f57b8c after 10ms
yarn上有没有启动一个flink session
目前用yarn方式提交只支持session模式,需要预先启动一个flink session,名称为默认的"Flink session cluster"才可以,后面会支持PerJob模式在yarn上提交任务
from chunjun.
已支持PerJob模式
from chunjun.
Related Issues (20)
- 纯钧如何使用Kubernetes Operator提交
- [Bug] [chunjun-client] 本地local模式提交执行任务情况下connector 的加载错误
- [Feature][ftp] 支持读取中文路径、优化读取execl
- [Bug] [ddl] maven package failed
- Release v1.12.8-cross join unnest使用报错
- [mongodb-x]-mongo维表连接失败
- [Feature][hbase] sql模式下新增特性
- 本地执行mysqlcdcreader到streamwriter,GenericRowData类型无法支持rowSize! HOT 1
- [Feature][ftp] sql模式下,支持指定字段的列索引,支持指定sheetNo工作表
- [Feature][Module Name] support hbase incremental read
- [Question][chunjun-oracle] oracle的clob类型,使用transformer报错
- [Bug] [chunjun-connector-binlog] Skipping unrecognized binlog event Unknown type:41 from: mysql-bin.xxx
- [Bug] [Module Name] Bug title mysql-mysql中加入transformer操作,sql可能有点长报错
- [Feature][gbase8s] 增加支持gbase8s版本connector HOT 7
- [Feature][connector][mysql] The connector supports MySQL Driver 8.x
- [Feature][chunjun-core] Supports capturing dirty data from the source and when the source sends it downstream
- [Bug] [sqlserver] 在 yarn-pre-job 模式找不到 sqlserver 相关驱动
- [Bug] [hbase] The HBase unit test failed.
- [Question][chunjun-connector-starrocks] Data size check HOT 1
- [Bug] [StarRocks] streamLoad报错502时,统计指标错误统计了未成功写入的记录数
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chunjun.