spark-submit --master yarn --deploy-mode cluster --class org.resourcesubmit.spark.app.RSKafkaToHive --num-executors 1 --executor-memory 1G --executor-cores 2 --driver-memory 512m --conf yarn.nodemanager.resource.memory-mb=512 --conf yarn.scheduler.maximum-allocation-mb=1024 --conf spark.executor.memoryOverhead=1024 --conf spark.core.connection.ack.wait.timeout=500 --conf mapreduce.fileoutputcommitter.marksuccessfuljobs=false oss://jars/sensor/ResourceSubmitETL-1.0-SNAPSHOT-jar-with-dependencies.jar
1. -- conf 这种参数配置形式 ‘=’前后不能有空格
2.声明jar 包 一定要放在最后,之后只能跟 要传入的参数,其他的全部失效
附上参考链接 https://www.cnblogs.com/weiweifeng/p/8073553.html
原文:https://www.cnblogs.com/yangxusun9/p/13559435.html