首页 > 其他 > 详细

启动Flink的yarn-session模式出错

时间:2021-08-05 16:11:47      阅读:80      评论:0      收藏:0      [点我收藏+]

  问题如下:bin/yarn-session.sh -d 报错

 The program finished with the following exception:

org.apache.flink.client.deployment.ClusterDeploymentException: Couldn‘t deploy Yarn session cluster
    at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:425)
    at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:606)
    at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:860)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
    at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
    at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:860)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: The number of requested virtual cores for application master 1 exceeds the maximum number of virtual cores 0 available in the Yarn Cluster.
    at org.apache.flink.yarn.YarnClusterDescriptor.isReadyForDeployment(YarnClusterDescriptor.java:326)
    at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:534)
    at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:418)
    ... 7 more

  解析:日志中显示的大概原因就行启动失败,资源不够,但是我启动的时候并没有指定任何参数,使用的是默认值,申请的资源大小绝对不会超过可用的资源大小,完全够用,所以报错原因跟运行时指定的参数无关,那么我们该如何解决呢?

技术分享图片

  解决:之前测试时启动太多次Filnk的yarn-session模式,所以断定是临时文件过多导致,故删除HDFS中的Flink的临时文件即可,你的文件存在哪就删除哪,我存在/flink/yarn/ha目录下,所以直接删除即可,可以在flink的conf目录下的flink-conf.yaml查看HDFS的存储路径

技术分享图片

  再次启动,发现启动成功

bin/yarn-session.sh -d

技术分享图片

启动Flink的yarn-session模式出错

原文:https://www.cnblogs.com/LzMingYueShanPao/p/15103250.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!