在Windows下面运行hadoop的MapReduce程序的方法：
1.下载hadoop的安装包，这里使用的是"hadoop-2.6.4.tar.gz":
2.将安装包直接解压到D盘根目录:
3.配置环境变量：
4.下载hadoop的eclipse插件，并将插件放到eclipse的plugins目录下：
5.打开Eclipse，选择菜单"Window"-->"Preferences"，在左侧找到"Hadoop Map/Reduce"，
在右侧选择hadoop的目录：
6.打开菜单"Window"中的"Show View"窗口，选择"Map/Reduce Locations"：
7：在打开的"Map/Reduce Locations"面板中，点击小象图标，打开新建配置窗口：
8.按照下图所示，填写hadoop集群的主机地址和端口：
9.新创建的hadoop集群连接配置，右上角的齿轮可以修改配置信息：
10.打开菜单"Window"中的"Show View"窗口，找到"Project Explorer"：
11.在"Project Explorer"面板中找到"DFS Locations"，展开下面的菜单就可以连接上HDFS，
可以直接看到HDFS中的目录和文件：
12.在"Project Explorer"面板中点击鼠标右键，选择新建，就可以创建"Map/Reduce"项目了：
13.下面我们创建了一个名为"hadoop-test"的项目，可以看到它自动帮我们导入了很多的jar包：
14.在项目的src下面创建log4j.properties文件，内容如下：
log4j.rootLogger=debug,stdout,R log4j.appender.stdout=org.apache.log4j.ConsoleAppender log4j.appender.stdout.layout=org.apache.log4j.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%5p - %m%n log4j.appender.R=org.apache.log4j.RollingFileAppender log4j.appender.R.File=mapreduce_test.log log4j.appender.R.MaxFileSize=1MB log4j.appender.R.MaxBackupIndex=1 log4j.appender.R.layout=org.apache.log4j.PatternLayout log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n log4j.logger.com.codefutures=DEBUG
15.在src下面创建org.apache.hadoop.io.nativeio包，并将hadoop源码中的NativeIO.java类拷贝到该包下（这一步是从网上看的，不知道有什么用）。
NativeIO.java类所在路径为：hadoop-2.6.4-src\hadoop-common-project\hadoop-common\src\main\java\org\apache\hadoop\io\nativeio
16.修改NativeIO.java类中的如下部分，返回"true"即可：
17.下面来写一个hadoop的MapReduce程序，经典的wordcount（单词计数）程序，代码如下（共3个java类）：
（1）WordCountDriver.java类：
1 package com.xuebusi.hadoop.mr.windows; 2 3 import org.apache.hadoop.conf.Configuration; 4 import org.apache.hadoop.fs.Path; 5 import org.apache.hadoop.io.IntWritable; 6 import org.apache.hadoop.io.Text; 7 import org.apache.hadoop.mapreduce.Job; 8 import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; 9 import org.apache.hadoop.mapreduce.lib.input.TextInputFormat; 10 import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; 11 12 public class WordCountDriver { 13 14 public static void main(String[] args) throws Exception { 15 16 Configuration conf = new Configuration(); 17 Job job = Job.getInstance(conf); 18 19 job.setJarByClass(WordCountDriver.class); 20 21 job.setMapperClass(WordCountMapper.class); 22 job.setReducerClass(WordCountReducer.class); 23 24 job.setMapOutputKeyClass(Text.class); 25 job.setMapOutputValueClass(IntWritable.class); 26 27 job.setOutputKeyClass(Text.class); 28 job.setOutputValueClass(IntWritable.class); 29 30 job.setInputFormatClass(TextInputFormat.class); 31 32 FileInputFormat.setInputPaths(job, new Path("c:/wordcount/input")); 33 FileOutputFormat.setOutputPath(job, new Path("c:/wordcount/output")); 34 35 boolean res = job.waitForCompletion(true); 36 System.exit(res ? 0 : 1); 37 38 } 39 40 }
（2）WordCountMapper.java类：
1 package com.xuebusi.hadoop.mr.windows; 2 3 import java.io.IOException; 4 5 import org.apache.hadoop.io.IntWritable; 6 import org.apache.hadoop.io.LongWritable; 7 import org.apache.hadoop.io.Text; 8 import org.apache.hadoop.mapreduce.Mapper; 9 10 public class WordCountMapper extends Mapper<LongWritable, Text, Text, IntWritable> { 11 12 @Override 13 protected void map(LongWritable key, Text value, Context context) 14 throws IOException, InterruptedException { 15 // TODO Auto-generated method stub 16 //super.map(key, value, context); 17 18 String line = value.toString(); 19 String[] words = line.split(" "); 20 21 for (String word : words) { 22 context.write(new Text(word), new IntWritable(1)); 23 } 24 } 25 }
（3）WordCountReducer.java类：
1 package com.xuebusi.hadoop.mr.windows; 2 3 import java.io.IOException; 4 5 import org.apache.hadoop.io.IntWritable; 6 import org.apache.hadoop.io.Text; 7 import org.apache.hadoop.mapreduce.Reducer; 8 9 public class WordCountReducer extends Reducer<Text, IntWritable, Text, IntWritable> { 10 11 @Override 12 protected void reduce(Text key, Iterable<IntWritable> values, Context context) 13 throws IOException, InterruptedException { 14 // TODO Auto-generated method stub 15 // super.reduce(arg0, arg1, arg2); 16 17 int count = 0; 18 for (IntWritable value : values) { 19 count += value.get(); 20 } 21 22 context.write(new Text(key), new IntWritable(count)); 23 } 24 25 }
18.准备点数据，在"c:/wordcount/input"目录下面创建"words.txt"文件，输入一些文本，并用空格隔开：
18.在Eclipse中，运行WordCountDriver类中的main方法（右键Run As-->Java Application），可以在控制台看到如下日志信息：
1 DEBUG - Handling deprecation for dfs.image.compression.codec 2 DEBUG - Handling deprecation for mapreduce.job.reduces 3 DEBUG - Handling deprecation for mapreduce.job.complete.cancel.delegation.tokens 4 DEBUG - Handling deprecation for mapreduce.jobhistory.recovery.store.class 5 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.search.filter.user 6 DEBUG - Handling deprecation for dfs.namenode.enable.retrycache 7 DEBUG - Handling deprecation for yarn.nodemanager.sleep-delay-before-sigkill.ms 8 DEBUG - Handling deprecation for mapreduce.jobhistory.joblist.cache.size 9 DEBUG - Handling deprecation for mapreduce.tasktracker.healthchecker.interval 10 DEBUG - Handling deprecation for mapreduce.jobtracker.heartbeats.in.second 11 DEBUG - Handling deprecation for mapreduce.jobtracker.persist.jobstatus.dir 12 DEBUG - Handling deprecation for dfs.namenode.backup.http-address 13 DEBUG - Handling deprecation for hadoop.rpc.protection 14 DEBUG - Handling deprecation for dfs.client.mmap.enabled 15 DEBUG - Handling deprecation for yarn.app.mapreduce.am.container.log.backups 16 DEBUG - Handling deprecation for ftp.stream-buffer-size 17 DEBUG - Handling deprecation for dfs.namenode.https-address 18 DEBUG - Handling deprecation for yarn.timeline-service.address 19 DEBUG - Handling deprecation for dfs.ha.log-roll.period 20 DEBUG - Handling deprecation for yarn.nodemanager.recovery.enabled 21 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.numinputfiles 22 DEBUG - Handling deprecation for hadoop.security.groups.negative-cache.secs 23 DEBUG - Handling deprecation for yarn.resourcemanager.admin.client.thread-count 24 DEBUG - Handling deprecation for dfs.datanode.fsdatasetcache.max.threads.per.volume 25 DEBUG - Handling deprecation for file.client-write-packet-size 26 DEBUG - Handling deprecation for hadoop.http.authentication.simple.anonymous.allowed 27 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.path 28 DEBUG - Handling deprecation for yarn.resourcemanager.proxy-user-privileges.enabled 29 DEBUG - Handling deprecation for dfs.datanode.drop.cache.behind.reads 30 DEBUG - Handling deprecation for yarn.nodemanager.log.retain-seconds 31 DEBUG - Handling deprecation for dfs.image.transfer.bandwidthPerSec 32 DEBUG - Handling deprecation for yarn.resourcemanager.work-preserving-recovery.scheduling-wait-ms 33 DEBUG - Handling deprecation for hadoop.registry.zk.session.timeout.ms 34 DEBUG - Handling deprecation for dfs.datanode.slow.io.warning.threshold.ms 35 DEBUG - Handling deprecation for mapreduce.tasktracker.instrumentation 36 DEBUG - Handling deprecation for ha.failover-controller.cli-check.rpc-timeout.ms 37 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.hierarchy 38 DEBUG - Handling deprecation for dfs.namenode.write.stale.datanode.ratio 39 DEBUG - Handling deprecation for hadoop.security.groups.cache.warn.after.ms 40 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.timeout-ms 41 DEBUG - Handling deprecation for mapreduce.jobhistory.client.thread-count 42 DEBUG - Handling deprecation for io.mapfile.bloom.size 43 DEBUG - Handling deprecation for yarn.nodemanager.docker-container-executor.exec-name 44 DEBUG - Handling deprecation for yarn.resourcemanager.work-preserving-recovery.enabled 45 DEBUG - Handling deprecation for dfs.ha.fencing.ssh.connect-timeout 46 DEBUG - Handling deprecation for yarn.resourcemanager.zk-num-retries 47 DEBUG - Handling deprecation for hadoop.registry.zk.root 48 DEBUG - Handling deprecation for s3.bytes-per-checksum 49 DEBUG - Handling deprecation for yarn.app.mapreduce.am.container.log.limit.kb 50 DEBUG - Handling deprecation for dfs.namenode.edit.log.autoroll.check.interval.ms 51 DEBUG - Handling deprecation for fs.automatic.close 52 DEBUG - Handling deprecation for fs.trash.interval 53 DEBUG - Handling deprecation for dfs.journalnode.https-address 54 DEBUG - Handling deprecation for yarn.timeline-service.ttl-ms 55 DEBUG - Handling deprecation for hadoop.security.authentication 56 DEBUG - Handling deprecation for fs.defaultFS 57 DEBUG - Handling deprecation for nfs.rtmax 58 DEBUG - Handling deprecation for hadoop.ssl.server.conf 59 DEBUG - Handling deprecation for ipc.client.connect.max.retries 60 DEBUG - Handling deprecation for yarn.resourcemanager.delayed.delegation-token.removal-interval-ms 61 DEBUG - Handling deprecation for dfs.journalnode.http-address 62 DEBUG - Handling deprecation for dfs.namenode.xattrs.enabled 63 DEBUG - Handling deprecation for dfs.datanode.shared.file.descriptor.paths 64 DEBUG - Handling deprecation for mapreduce.jobtracker.taskscheduler 65 DEBUG - Handling deprecation for mapreduce.job.speculative.speculativecap 66 DEBUG - Handling deprecation for yarn.timeline-service.store-class 67 DEBUG - Handling deprecation for yarn.am.liveness-monitor.expiry-interval-ms 68 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.compress 69 DEBUG - Handling deprecation for dfs.user.home.dir.prefix 70 DEBUG - Handling deprecation for net.topology.node.switch.mapping.impl 71 DEBUG - Handling deprecation for dfs.namenode.replication.considerLoad 72 DEBUG - Handling deprecation for dfs.namenode.fs-limits.min-block-size 73 DEBUG - Handling deprecation for fs.swift.impl 74 DEBUG - Handling deprecation for dfs.namenode.audit.loggers 75 DEBUG - Handling deprecation for mapreduce.job.max.split.locations 76 DEBUG - Handling deprecation for yarn.resourcemanager.address 77 DEBUG - Handling deprecation for mapreduce.job.counters.max 78 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.enabled 79 DEBUG - Handling deprecation for dfs.client.block.write.retries 80 DEBUG - Handling deprecation for dfs.short.circuit.shared.memory.watcher.interrupt.check.ms 81 DEBUG - Handling deprecation for dfs.namenode.resource.checked.volumes.minimum 82 DEBUG - Handling deprecation for io.map.index.interval 83 DEBUG - Handling deprecation for mapred.child.java.opts 84 DEBUG - Handling deprecation for mapreduce.tasktracker.local.dir.minspacestart 85 DEBUG - Handling deprecation for mapreduce.client.progressmonitor.pollinterval 86 DEBUG - Handling deprecation for dfs.client.https.keystore.resource 87 DEBUG - Handling deprecation for mapreduce.task.profile.map.params 88 DEBUG - Handling deprecation for mapreduce.jobtracker.tasktracker.maxblacklists 89 DEBUG - Handling deprecation for mapreduce.job.queuename 90 DEBUG - Handling deprecation for hadoop.registry.zk.quorum 91 DEBUG - Handling deprecation for yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts 92 DEBUG - Handling deprecation for yarn.nodemanager.localizer.address 93 DEBUG - Handling deprecation for io.mapfile.bloom.error.rate 94 DEBUG - Handling deprecation for yarn.nodemanager.delete.thread-count 95 DEBUG - Handling deprecation for mapreduce.job.split.metainfo.maxsize 96 DEBUG - Handling deprecation for yarn.scheduler.maximum-allocation-vcores 97 DEBUG - Handling deprecation for mapred.mapper.new-api 98 DEBUG - Handling deprecation for mapreduce.job.dir 99 DEBUG - Handling deprecation for yarn.app.mapreduce.am.resource.mb 100 DEBUG - Handling deprecation for dfs.datanode.dns.nameserver 101 DEBUG - Handling deprecation for dfs.client.slow.io.warning.threshold.ms 102 DEBUG - Handling deprecation for mapreduce.job.reducer.preempt.delay.sec 103 DEBUG - Handling deprecation for yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb 104 DEBUG - Handling deprecation for mapreduce.map.output.compress.codec 105 DEBUG - Handling deprecation for dfs.namenode.accesstime.precision 106 DEBUG - Handling deprecation for fs.s3a.connection.maximum 107 DEBUG - Handling deprecation for mapreduce.map.log.level 108 DEBUG - Handling deprecation for io.seqfile.compress.blocksize 109 DEBUG - Handling deprecation for ipc.client.ping 110 DEBUG - Handling deprecation for mapreduce.tasktracker.taskcontroller 111 DEBUG - Handling deprecation for hadoop.security.groups.cache.secs 112 DEBUG - Handling deprecation for dfs.datanode.cache.revocation.timeout.ms 113 DEBUG - Handling deprecation for dfs.client.context 114 DEBUG - Handling deprecation for mapreduce.input.lineinputformat.linespermap 115 DEBUG - Handling deprecation for mapreduce.job.end-notification.max.attempts 116 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user 117 DEBUG - Handling deprecation for yarn.nodemanager.webapp.address 118 DEBUG - Handling deprecation for mapreduce.job.submithostname 119 DEBUG - Handling deprecation for mapreduce.jobhistory.recovery.enable 120 DEBUG - Handling deprecation for mapreduce.jobtracker.expire.trackers.interval 121 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.address 122 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.num.refill.threads 123 DEBUG - Handling deprecation for yarn.nodemanager.health-checker.interval-ms 124 DEBUG - Handling deprecation for mapreduce.jobhistory.loadedjobs.cache.size 125 DEBUG - Handling deprecation for hadoop.security.authorization 126 DEBUG - Handling deprecation for mapreduce.job.map.output.collector.class 127 DEBUG - Handling deprecation for mapreduce.am.max-attempts 128 DEBUG - Handling deprecation for fs.ftp.host 129 DEBUG - Handling deprecation for fs.s3a.attempts.maximum 130 DEBUG - Handling deprecation for yarn.app.mapreduce.am.scheduler.heartbeat.interval-ms 131 DEBUG - Handling deprecation for mapreduce.ifile.readahead 132 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.monitor.enable 133 DEBUG - Handling deprecation for yarn.resourcemanager.zk-retry-interval-ms 134 DEBUG - Handling deprecation for ha.zookeeper.session-timeout.ms 135 DEBUG - Handling deprecation for mapreduce.tasktracker.taskmemorymanager.monitoringinterval 136 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.parallelcopies 137 DEBUG - Handling deprecation for dfs.client.mmap.retry.timeout.ms 138 DEBUG - Handling deprecation for mapreduce.map.skip.maxrecords 139 DEBUG - Handling deprecation for mapreduce.job.output.value.class 140 DEBUG - Handling deprecation for dfs.namenode.avoid.read.stale.datanode 141 DEBUG - Handling deprecation for dfs.https.enable 142 DEBUG - Handling deprecation for yarn.timeline-service.webapp.https.address 143 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.read.timeout 144 DEBUG - Handling deprecation for dfs.namenode.list.encryption.zones.num.responses 145 DEBUG - Handling deprecation for yarn.nodemanager.remote-app-log-dir-suffix 146 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.compress.codec 147 DEBUG - Handling deprecation for mapreduce.jobtracker.instrumentation 148 DEBUG - Handling deprecation for dfs.blockreport.intervalMsec 149 DEBUG - Handling deprecation for ipc.client.connect.retry.interval 150 DEBUG - Handling deprecation for mapreduce.reduce.speculative 151 DEBUG - Handling deprecation for mapreduce.jobhistory.keytab 152 DEBUG - Handling deprecation for mapreduce.jobhistory.datestring.cache.size 153 DEBUG - Handling deprecation for dfs.datanode.balance.bandwidthPerSec 154 DEBUG - Handling deprecation for file.blocksize 155 DEBUG - Handling deprecation for yarn.resourcemanager.admin.address 156 DEBUG - Handling deprecation for mapreduce.map.cpu.vcores 157 DEBUG - Handling deprecation for yarn.nodemanager.container-monitor.procfs-tree.smaps-based-rss.enabled 158 DEBUG - Handling deprecation for yarn.resourcemanager.configuration.provider-class 159 DEBUG - Handling deprecation for yarn.resourcemanager.resource-tracker.address 160 DEBUG - Handling deprecation for mapreduce.tasktracker.local.dir.minspacekill 161 DEBUG - Handling deprecation for mapreduce.jobtracker.staging.root.dir 162 DEBUG - Handling deprecation for mapreduce.jobtracker.retiredjobs.cache.size 163 DEBUG - Handling deprecation for hadoop.registry.rm.enabled 164 DEBUG - Handling deprecation for ipc.client.connect.max.retries.on.timeouts 165 DEBUG - Handling deprecation for ha.zookeeper.acl 166 DEBUG - Handling deprecation for hadoop.security.crypto.codec.classes.aes.ctr.nopadding 167 DEBUG - Handling deprecation for yarn.nodemanager.local-dirs 168 DEBUG - Handling deprecation for mapreduce.app-submission.cross-platform 169 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.connect.timeout 170 DEBUG - Handling deprecation for dfs.block.access.key.update.interval 171 DEBUG - Handling deprecation for rpc.metrics.quantile.enable 172 DEBUG - Handling deprecation for dfs.block.access.token.lifetime 173 DEBUG - Handling deprecation for mapreduce.job.end-notification.retry.attempts 174 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-xattrs-per-inode 175 DEBUG - Handling deprecation for mapreduce.jobtracker.system.dir 176 DEBUG - Handling deprecation for dfs.client.file-block-storage-locations.timeout.millis 177 DEBUG - Handling deprecation for yarn.nodemanager.admin-env 178 DEBUG - Handling deprecation for mapreduce.jobtracker.jobhistory.block.size 179 DEBUG - Handling deprecation for yarn.log-aggregation.retain-seconds 180 DEBUG - Handling deprecation for mapreduce.tasktracker.indexcache.mb 181 DEBUG - Handling deprecation for yarn.timeline-service.handler-thread-count 182 DEBUG - Handling deprecation for dfs.namenode.checkpoint.check.period 183 DEBUG - Handling deprecation for yarn.resourcemanager.hostname 184 DEBUG - Handling deprecation for dfs.client.block.write.replace-datanode-on-failure.enable 185 DEBUG - Handling deprecation for net.topology.impl 186 DEBUG - Handling deprecation for dfs.datanode.directoryscan.interval 187 DEBUG - Handling deprecation for fs.s3a.multipart.purge.age 188 DEBUG - Handling deprecation for hadoop.security.java.secure.random.algorithm 189 DEBUG - Handling deprecation for yarn.nodemanager.container-monitor.interval-ms 190 DEBUG - Handling deprecation for dfs.default.chunk.view.size 191 DEBUG - Handling deprecation for fs.s3a.multipart.threshold 192 DEBUG - Handling deprecation for mapreduce.job.speculative.slownodethreshold 193 DEBUG - Handling deprecation for mapreduce.job.reduce.slowstart.completedmaps 194 DEBUG - Handling deprecation for mapred.reducer.new-api 195 DEBUG - Handling deprecation for hadoop.security.instrumentation.requires.admin 196 DEBUG - Handling deprecation for io.compression.codec.bzip2.library 197 DEBUG - Handling deprecation for hadoop.http.authentication.signature.secret.file 198 DEBUG - Handling deprecation for hadoop.registry.secure 199 DEBUG - Handling deprecation for dfs.namenode.safemode.min.datanodes 200 DEBUG - Handling deprecation for yarn.nodemanager.localizer.cache.target-size-mb 201 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.inputdir 202 DEBUG - Handling deprecation for mapreduce.reduce.maxattempts 203 DEBUG - Handling deprecation for dfs.datanode.https.address 204 DEBUG - Handling deprecation for s3native.replication 205 DEBUG - Handling deprecation for dfs.namenode.inotify.max.events.per.rpc 206 DEBUG - Handling deprecation for dfs.namenode.path.based.cache.retry.interval.ms 207 DEBUG - Handling deprecation for mapreduce.reduce.skip.proc.count.autoincr 208 DEBUG - Handling deprecation for dfs.datanode.cache.revocation.polling.ms 209 DEBUG - Handling deprecation for mapreduce.jobhistory.cleaner.interval-ms 210 DEBUG - Handling deprecation for file.replication 211 DEBUG - Handling deprecation for hadoop.hdfs.configuration.version 212 DEBUG - Handling deprecation for ipc.client.idlethreshold 213 DEBUG - Handling deprecation for hadoop.tmp.dir 214 DEBUG - Handling deprecation for yarn.resourcemanager.store.class 215 DEBUG - Handling deprecation for mapreduce.jobhistory.address 216 DEBUG - Handling deprecation for mapreduce.jobtracker.restart.recover 217 DEBUG - Handling deprecation for mapreduce.cluster.local.dir 218 DEBUG - Handling deprecation for yarn.client.nodemanager-client-async.thread-pool-max-size 219 DEBUG - Handling deprecation for dfs.namenode.decommission.nodes.per.interval 220 DEBUG - Handling deprecation for mapreduce.job.inputformat.class 221 DEBUG - Handling deprecation for yarn.nodemanager.resource.cpu-vcores 222 DEBUG - Handling deprecation for dfs.namenode.reject-unresolved-dn-topology-mapping 223 DEBUG - Handling deprecation for dfs.namenode.delegation.key.update-interval 224 DEBUG - Handling deprecation for fs.s3.buffer.dir 225 DEBUG - Handling deprecation for dfs.client.read.shortcircuit.streams.cache.expiry.ms 226 DEBUG - Handling deprecation for dfs.namenode.support.allow.format 227 DEBUG - Handling deprecation for yarn.nodemanager.remote-app-log-dir 228 DEBUG - Handling deprecation for mapreduce.map.memory.mb 229 DEBUG - Handling deprecation for dfs.namenode.edit.log.autoroll.multiplier.threshold 230 DEBUG - Handling deprecation for hadoop.work.around.non.threadsafe.getpwuid 231 DEBUG - Handling deprecation for mapreduce.task.profile.reduce.params 232 DEBUG - Handling deprecation for yarn.timeline-service.client.max-retries 233 DEBUG - Handling deprecation for dfs.ha.automatic-failover.enabled 234 DEBUG - Handling deprecation for dfs.namenode.edits.noeditlogchannelflush 235 DEBUG - Handling deprecation for dfs.namenode.stale.datanode.interval 236 DEBUG - Handling deprecation for mapreduce.shuffle.transfer.buffer.size 237 DEBUG - Handling deprecation for mapreduce.jobtracker.persist.jobstatus.active 238 DEBUG - Handling deprecation for dfs.namenode.logging.level 239 DEBUG - Handling deprecation for yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs 240 DEBUG - Handling deprecation for yarn.nodemanager.log-dirs 241 DEBUG - Handling deprecation for ha.health-monitor.sleep-after-disconnect.ms 242 DEBUG - Handling deprecation for yarn.resourcemanager.fs.state-store.uri 243 DEBUG - Handling deprecation for dfs.namenode.checkpoint.edits.dir 244 DEBUG - Handling deprecation for yarn.resourcemanager.keytab 245 DEBUG - Handling deprecation for hadoop.rpc.socket.factory.class.default 246 DEBUG - Handling deprecation for yarn.resourcemanager.fail-fast 247 DEBUG - Handling deprecation for dfs.datanode.http.address 248 DEBUG - Handling deprecation for mapreduce.task.profile 249 DEBUG - Handling deprecation for mapreduce.jobhistory.move.interval-ms 250 DEBUG - Handling deprecation for dfs.namenode.edits.dir 251 DEBUG - Handling deprecation for dfs.storage.policy.enabled 252 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.size 253 DEBUG - Handling deprecation for hadoop.fuse.timer.period 254 DEBUG - Handling deprecation for mapreduce.jobhistory.http.policy 255 DEBUG - Handling deprecation for mapreduce.jobhistory.intermediate-done-dir 256 DEBUG - Handling deprecation for mapreduce.map.skip.proc.count.autoincr 257 DEBUG - Handling deprecation for fs.AbstractFileSystem.viewfs.impl 258 DEBUG - Handling deprecation for mapreduce.job.speculative.slowtaskthreshold 259 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled 260 DEBUG - Handling deprecation for s3native.stream-buffer-size 261 DEBUG - Handling deprecation for yarn.nodemanager.delete.debug-delay-sec 262 DEBUG - Handling deprecation for dfs.secondary.namenode.kerberos.internal.spnego.principal 263 DEBUG - Handling deprecation for dfs.datanode.available-space-volume-choosing-policy.balanced-space-threshold 264 DEBUG - Handling deprecation for fs.s3n.multipart.uploads.block.size 265 DEBUG - Handling deprecation for dfs.namenode.safemode.threshold-pct 266 DEBUG - Handling deprecation for mapreduce.ifile.readahead.bytes 267 DEBUG - Handling deprecation for yarn.scheduler.maximum-allocation-mb 268 DEBUG - Handling deprecation for ipc.client.fallback-to-simple-auth-allowed 269 DEBUG - Handling deprecation for fs.har.impl.disable.cache 270 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.read-cache-size 271 DEBUG - Handling deprecation for yarn.timeline-service.hostname 272 DEBUG - Handling deprecation for s3native.bytes-per-checksum 273 DEBUG - Handling deprecation for mapreduce.job.committer.setup.cleanup.needed 274 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms 275 DEBUG - Handling deprecation for fs.s3a.paging.maximum 276 DEBUG - Handling deprecation for yarn.client.nodemanager-connect.retry-interval-ms 277 DEBUG - Handling deprecation for yarn.nodemanager.log-aggregation.compression-type 278 DEBUG - Handling deprecation for yarn.app.mapreduce.am.job.committer.commit-window 279 DEBUG - Handling deprecation for hadoop.http.authentication.type 280 DEBUG - Handling deprecation for dfs.client.failover.sleep.base.millis 281 DEBUG - Handling deprecation for mapreduce.job.submithostaddress 282 DEBUG - Handling deprecation for yarn.nodemanager.vmem-check-enabled 283 DEBUG - Handling deprecation for hadoop.jetty.logs.serve.aliases 284 DEBUG - Handling deprecation for ha.failover-controller.graceful-fence.rpc-timeout.ms 285 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.input.buffer.percent 286 DEBUG - Handling deprecation for dfs.datanode.max.transfer.threads 287 DEBUG - Handling deprecation for mapreduce.reduce.merge.inmem.threshold 288 DEBUG - Handling deprecation for mapreduce.task.io.sort.mb 289 DEBUG - Handling deprecation for dfs.namenode.acls.enabled 290 DEBUG - Handling deprecation for yarn.client.application-client-protocol.poll-interval-ms 291 DEBUG - Handling deprecation for hadoop.security.kms.client.authentication.retry-count 292 DEBUG - Handling deprecation for dfs.namenode.handler.count 293 DEBUG - Handling deprecation for yarn.resourcemanager.connect.max-wait.ms 294 DEBUG - Handling deprecation for dfs.namenode.retrycache.heap.percent 295 DEBUG - Handling deprecation for yarn.timeline-service.enabled 296 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users 297 DEBUG - Handling deprecation for hadoop.ssl.client.conf 298 DEBUG - Handling deprecation for yarn.resourcemanager.container.liveness-monitor.interval-ms 299 DEBUG - Handling deprecation for yarn.nodemanager.vmem-pmem-ratio 300 DEBUG - Handling deprecation for mapreduce.client.completion.pollinterval 301 DEBUG - Handling deprecation for yarn.app.mapreduce.client.max-retries 302 DEBUG - Handling deprecation for hadoop.ssl.enabled 303 DEBUG - Handling deprecation for fs.client.resolve.remote.symlinks 304 DEBUG - Handling deprecation for fs.AbstractFileSystem.hdfs.impl 305 DEBUG - Handling deprecation for mapreduce.tasktracker.reduce.tasks.maximum 306 DEBUG - Handling deprecation for yarn.nodemanager.hostname 307 DEBUG - Handling deprecation for mapreduce.reduce.input.buffer.percent 308 DEBUG - Handling deprecation for fs.s3a.multipart.purge 309 DEBUG - Handling deprecation for yarn.app.mapreduce.am.command-opts 310 DEBUG - Handling deprecation for dfs.namenode.invalidate.work.pct.per.iteration 311 DEBUG - Handling deprecation for dfs.bytes-per-checksum 312 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.https.address 313 DEBUG - Handling deprecation for dfs.replication 314 DEBUG - Handling deprecation for dfs.datanode.block.id.layout.upgrade.threads 315 DEBUG - Handling deprecation for mapreduce.shuffle.ssl.file.buffer.size 316 DEBUG - Handling deprecation for dfs.namenode.list.cache.directives.num.responses 317 DEBUG - Handling deprecation for dfs.permissions.enabled 318 DEBUG - Handling deprecation for mapreduce.jobtracker.maxtasks.perjob 319 DEBUG - Handling deprecation for dfs.datanode.use.datanode.hostname 320 DEBUG - Handling deprecation for mapreduce.task.userlog.limit.kb 321 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-directory-items 322 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.low-watermark 323 DEBUG - Handling deprecation for fs.s3a.buffer.dir 324 DEBUG - Handling deprecation for s3.client-write-packet-size 325 DEBUG - Handling deprecation for hadoop.user.group.static.mapping.overrides 326 DEBUG - Handling deprecation for mapreduce.shuffle.max.threads 327 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.expiry 328 DEBUG - Handling deprecation for dfs.client.failover.sleep.max.millis 329 DEBUG - Handling deprecation for mapreduce.job.maps 330 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-component-length 331 DEBUG - Handling deprecation for yarn.fail-fast 332 DEBUG - Handling deprecation for hadoop.ssl.enabled.protocols 333 DEBUG - Handling deprecation for s3.blocksize 334 DEBUG - Handling deprecation for mapreduce.map.output.compress 335 DEBUG - Handling deprecation for dfs.namenode.edits.journal-plugin.qjournal 336 DEBUG - Handling deprecation for dfs.namenode.datanode.registration.ip-hostname-check 337 DEBUG - Handling deprecation for yarn.nodemanager.pmem-check-enabled 338 DEBUG - Handling deprecation for dfs.client.short.circuit.replica.stale.threshold.ms 339 DEBUG - Handling deprecation for dfs.client.https.need-auth 340 DEBUG - Handling deprecation for yarn.scheduler.minimum-allocation-mb 341 DEBUG - Handling deprecation for mapreduce.jobhistory.max-age-ms 342 DEBUG - Handling deprecation for yarn.timeline-service.http-authentication.type 343 DEBUG - Handling deprecation for ftp.replication 344 DEBUG - Handling deprecation for dfs.namenode.secondary.https-address 345 DEBUG - Handling deprecation for dfs.blockreport.split.threshold 346 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.split.minsize 347 DEBUG - Handling deprecation for fs.s3n.block.size 348 DEBUG - Handling deprecation for mapreduce.job.token.tracking.ids.enabled 349 DEBUG - Handling deprecation for yarn.ipc.rpc.class 350 DEBUG - Handling deprecation for dfs.namenode.num.extra.edits.retained 351 DEBUG - Handling deprecation for yarn.nodemanager.localizer.cache.cleanup.interval-ms 352 DEBUG - Handling deprecation for hadoop.http.staticuser.user 353 DEBUG - Handling deprecation for mapreduce.jobhistory.move.thread-count 354 DEBUG - Handling deprecation for fs.s3a.multipart.size 355 DEBUG - Handling deprecation for mapreduce.job.jvm.numtasks 356 DEBUG - Handling deprecation for mapreduce.task.profile.maps 357 DEBUG - Handling deprecation for dfs.datanode.max.locked.memory 358 DEBUG - Handling deprecation for dfs.cachereport.intervalMsec 359 DEBUG - Handling deprecation for mapreduce.shuffle.port 360 DEBUG - Handling deprecation for yarn.resourcemanager.nodemanager.minimum.version 361 DEBUG - Handling deprecation for mapreduce.shuffle.connection-keep-alive.timeout 362 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.merge.percent 363 DEBUG - Handling deprecation for mapreduce.jobtracker.http.address 364 DEBUG - Handling deprecation for mapreduce.task.skip.start.attempts 365 DEBUG - Handling deprecation for yarn.resourcemanager.connect.retry-interval.ms 366 DEBUG - Handling deprecation for yarn.scheduler.minimum-allocation-vcores 367 DEBUG - Handling deprecation for mapreduce.task.io.sort.factor 368 DEBUG - Handling deprecation for dfs.namenode.checkpoint.dir 369 DEBUG - Handling deprecation for nfs.exports.allowed.hosts 370 DEBUG - Handling deprecation for tfile.fs.input.buffer.size 371 DEBUG - Handling deprecation for fs.s3.block.size 372 DEBUG - Handling deprecation for tfile.io.chunk.size 373 DEBUG - Handling deprecation for fs.s3n.multipart.copy.block.size 374 DEBUG - Handling deprecation for io.serializations 375 DEBUG - Handling deprecation for yarn.resourcemanager.max-completed-applications 376 DEBUG - Handling deprecation for mapreduce.jobhistory.principal 377 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.outputdir 378 DEBUG - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.zk-base-path 379 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.interval-ms 380 DEBUG - Handling deprecation for mapreduce.job.end-notification.retry.interval 381 DEBUG - Handling deprecation for dfs.namenode.backup.address 382 DEBUG - Handling deprecation for fs.s3n.multipart.uploads.enabled 383 DEBUG - Handling deprecation for io.seqfile.sorter.recordlimit 384 DEBUG - Handling deprecation for dfs.block.access.token.enable 385 DEBUG - Handling deprecation for s3native.client-write-packet-size 386 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-xattr-size 387 DEBUG - Handling deprecation for ftp.bytes-per-checksum 388 DEBUG - Handling deprecation for hadoop.security.group.mapping 389 DEBUG - Handling deprecation for dfs.client.domain.socket.data.traffic 390 DEBUG - Handling deprecation for dfs.client.read.shortcircuit.streams.cache.size 391 DEBUG - Handling deprecation for fs.s3a.connection.timeout 392 DEBUG - Handling deprecation for mapreduce.job.end-notification.max.retry.interval 393 DEBUG - Handling deprecation for yarn.acl.enable 394 DEBUG - Handling deprecation for yarn.nm.liveness-monitor.expiry-interval-ms 395 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.list-status.num-threads 396 DEBUG - Handling deprecation for dfs.client.mmap.cache.size 397 DEBUG - Handling deprecation for mapreduce.tasktracker.map.tasks.maximum 398 DEBUG - Handling deprecation for yarn.timeline-service.ttl-enable 399 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.resources-handler.class 400 DEBUG - Handling deprecation for dfs.namenode.max.objects 401 DEBUG - Handling deprecation for yarn.resourcemanager.state-store.max-completed-applications 402 DEBUG - Handling deprecation for dfs.namenode.delegation.token.max-lifetime 403 DEBUG - Handling deprecation for mapreduce.job.classloader 404 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size 405 DEBUG - Handling deprecation for mapreduce.job.hdfs-servers 406 DEBUG - Handling deprecation for dfs.datanode.hdfs-blocks-metadata.enabled 407 DEBUG - Handling deprecation for mapreduce.tasktracker.dns.nameserver 408 DEBUG - Handling deprecation for dfs.datanode.readahead.bytes 409 DEBUG - Handling deprecation for dfs.datanode.bp-ready.timeout 410 DEBUG - Handling deprecation for mapreduce.job.ubertask.maxreduces 411 DEBUG - Handling deprecation for dfs.image.compress 412 DEBUG - Handling deprecation for mapreduce.shuffle.ssl.enabled 413 DEBUG - Handling deprecation for yarn.log-aggregation-enable 414 DEBUG - Handling deprecation for mapreduce.tasktracker.report.address 415 DEBUG - Handling deprecation for mapreduce.tasktracker.http.threads 416 DEBUG - Handling deprecation for dfs.stream-buffer-size 417 DEBUG - Handling deprecation for tfile.fs.output.buffer.size 418 DEBUG - Handling deprecation for fs.permissions.umask-mode 419 DEBUG - Handling deprecation for dfs.client.datanode-restart.timeout 420 DEBUG - Handling deprecation for dfs.namenode.resource.du.reserved 421 DEBUG - Handling deprecation for yarn.resourcemanager.am.max-attempts 422 DEBUG - Handling deprecation for yarn.nodemanager.resource.percentage-physical-cpu-limit 423 DEBUG - Handling deprecation for ha.failover-controller.graceful-fence.connection.retries 424 DEBUG - Handling deprecation for dfs.datanode.drop.cache.behind.writes 425 DEBUG - Handling deprecation for mapreduce.map.output.value.class 426 DEBUG - Handling deprecation for hadoop.common.configuration.version 427 DEBUG - Handling deprecation for mapreduce.job.ubertask.enable 428 DEBUG - Handling deprecation for yarn.app.mapreduce.am.resource.cpu-vcores 429 DEBUG - Handling deprecation for dfs.namenode.replication.work.multiplier.per.iteration 430 DEBUG - Handling deprecation for mapreduce.job.acl-modify-job 431 DEBUG - Handling deprecation for io.seqfile.local.dir 432 DEBUG - Handling deprecation for yarn.resourcemanager.system-metrics-publisher.enabled 433 DEBUG - Handling deprecation for fs.s3.sleepTimeSeconds 434 DEBUG - Handling deprecation for mapreduce.client.output.filter 435 INFO - Submitting tokens for job: job_local1553403857_0001 436 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331) 437 DEBUG - Handling deprecation for all properties in config... 438 DEBUG - Handling deprecation for dfs.datanode.data.dir 439 DEBUG - Handling deprecation for dfs.namenode.checkpoint.txns 440 DEBUG - Handling deprecation for s3.replication 441 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.compress.type 442 DEBUG - Handling deprecation for mapreduce.jobtracker.jobhistory.lru.cache.size 443 DEBUG - Handling deprecation for dfs.datanode.failed.volumes.tolerated 444 DEBUG - Handling deprecation for hadoop.http.filter.initializers 445 DEBUG - Handling deprecation for mapreduce.cluster.temp.dir 446 DEBUG - Handling deprecation for yarn.nodemanager.keytab 447 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.memory.limit.percent 448 DEBUG - Handling deprecation for dfs.namenode.checkpoint.max-retries 449 DEBUG - Handling deprecation for nfs.mountd.port 450 DEBUG - Handling deprecation for hadoop.registry.zk.retry.times 451 DEBUG - Handling deprecation for yarn.resourcemanager.zk-acl 452 DEBUG - Handling deprecation for mapreduce.reduce.skip.maxgroups 453 DEBUG - Handling deprecation for ipc.ping.interval 454 DEBUG - Handling deprecation for dfs.https.server.keystore.resource 455 DEBUG - Handling deprecation for yarn.app.mapreduce.task.container.log.backups 456 DEBUG - Handling deprecation for hadoop.http.authentication.kerberos.keytab 457 DEBUG - Handling deprecation for yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage 458 DEBUG - Handling deprecation for dfs.client.block.write.replace-datanode-on-failure.best-effort 459 DEBUG - Handling deprecation for yarn.nodemanager.localizer.client.thread-count 460 DEBUG - Handling deprecation for mapreduce.jobhistory.done-dir 461 DEBUG - Handling deprecation for mapreduce.framework.name 462 DEBUG - Handling deprecation for ha.failover-controller.new-active.rpc-timeout.ms 463 DEBUG - Handling deprecation for ha.health-monitor.check-interval.ms 464 DEBUG - Handling deprecation for io.file.buffer.size 465 DEBUG - Handling deprecation for mapreduce.shuffle.max.connections 466 DEBUG - Handling deprecation for dfs.namenode.resource.check.interval 467 DEBUG - Handling deprecation for dfs.namenode.path.based.cache.block.map.allocation.percent 468 DEBUG - Handling deprecation for mapreduce.task.tmp.dir 469 DEBUG - Handling deprecation for dfs.namenode.checkpoint.period 470 DEBUG - Handling deprecation for dfs.client.mmap.cache.timeout.ms 471 DEBUG - Handling deprecation for ipc.client.kill.max 472 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.class 473 DEBUG - Handling deprecation for mapreduce.jobtracker.taskcache.levels 474 DEBUG - Handling deprecation for s3.stream-buffer-size 475 DEBUG - Handling deprecation for dfs.namenode.secondary.http-address 476 DEBUG - Handling deprecation for yarn.client.nodemanager-connect.max-wait-ms 477 DEBUG - Handling deprecation for dfs.namenode.decommission.interval 478 DEBUG - Handling deprecation for dfs.namenode.http-address 479 DEBUG - Handling deprecation for mapreduce.task.files.preserve.failedtasks 480 DEBUG - Handling deprecation for dfs.encrypt.data.transfer 481 DEBUG - Handling deprecation for yarn.resourcemanager.ha.enabled 482 DEBUG - Handling deprecation for dfs.datanode.address 483 DEBUG - Handling deprecation for dfs.namenode.avoid.write.stale.datanode 484 DEBUG - Handling deprecation for hadoop.http.authentication.token.validity 485 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.search.filter.group 486 DEBUG - Handling deprecation for dfs.client.failover.max.attempts 487 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.monitor.policies 488 DEBUG - Handling deprecation for hadoop.registry.zk.connection.timeout.ms 489 DEBUG - Handling deprecation for hadoop.security.crypto.cipher.suite 490 DEBUG - Handling deprecation for yarn.timeline-service.http-authentication.simple.anonymous.allowed 491 DEBUG - Handling deprecation for mapreduce.task.profile.params 492 DEBUG - Handling deprecation for yarn.resourcemanager.fs.state-store.retry-policy-spec 493 DEBUG - Handling deprecation for yarn.admin.acl 494 DEBUG - Handling deprecation for yarn.nodemanager.local-cache.max-files-per-directory 495 DEBUG - Handling deprecation for yarn.client.failover-retries-on-socket-timeouts 496 DEBUG - Handling deprecation for dfs.namenode.retrycache.expirytime.millis 497 DEBUG - Handling deprecation for yarn.resourcemanager.nodemanagers.heartbeat-interval-ms 498 DEBUG - Handling deprecation for dfs.client.failover.connection.retries.on.timeouts 499 DEBUG - Handling deprecation for yarn.client.failover-proxy-provider 500 DEBUG - Handling deprecation for mapreduce.map.sort.spill.percent 501 DEBUG - Handling deprecation for file.stream-buffer-size 502 DEBUG - Handling deprecation for dfs.webhdfs.enabled 503 DEBUG - Handling deprecation for ipc.client.connection.maxidletime 504 DEBUG - Handling deprecation for mapreduce.task.combine.progress.records 505 DEBUG - Handling deprecation for mapreduce.job.output.key.class 506 DEBUG - Handling deprecation for mapreduce.jobtracker.persist.jobstatus.hours 507 DEBUG - Handling deprecation for dfs.image.transfer.chunksize 508 DEBUG - Handling deprecation for yarn.nodemanager.address 509 DEBUG - Handling deprecation for dfs.datanode.ipc.address 510 DEBUG - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.embedded 511 DEBUG - Handling deprecation for mapreduce.jobhistory.recovery.store.fs.uri 512 DEBUG - Handling deprecation for yarn.resourcemanager.zk-state-store.parent-path 513 DEBUG - Handling deprecation for yarn.app.mapreduce.am.job.task.listener.thread-count 514 DEBUG - Handling deprecation for nfs.dump.dir 515 DEBUG - Handling deprecation for dfs.namenode.list.cache.pools.num.responses 516 DEBUG - Handling deprecation for dfs.client.read.shortcircuit 517 DEBUG - Handling deprecation for dfs.namenode.safemode.extension 518 DEBUG - Handling deprecation for ha.zookeeper.parent-znode 519 DEBUG - Handling deprecation for yarn.nodemanager.container-executor.class 520 DEBUG - Handling deprecation for io.skip.checksum.errors 521 DEBUG - Handling deprecation for dfs.namenode.path.based.cache.refresh.interval.ms 522 DEBUG - Handling deprecation for dfs.encrypt.data.transfer.cipher.key.bitlength 523 DEBUG - Handling deprecation for mapreduce.job.user.name 524 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.client.thread-count 525 DEBUG - Handling deprecation for yarn.nodemanager.recovery.dir 526 DEBUG - Handling deprecation for mapreduce.job.emit-timeline-data 527 DEBUG - Handling deprecation for hadoop.http.authentication.kerberos.principal 528 DEBUG - Handling deprecation for mapreduce.reduce.log.level 529 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.user-pattern 530 DEBUG - Handling deprecation for fs.s3.maxRetries 531 DEBUG - Handling deprecation for yarn.nodemanager.resourcemanager.minimum.version 532 DEBUG - Handling deprecation for ipc.client.rpc-timeout.ms 533 DEBUG - Handling deprecation for hadoop.kerberos.kinit.command 534 DEBUG - Handling deprecation for yarn.log-aggregation.retain-check-interval-seconds 535 DEBUG - Handling deprecation for yarn.nodemanager.process-kill-wait.ms 536 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.mount 537 DEBUG - Handling deprecation for mapreduce.map.output.key.class 538 DEBUG - Handling deprecation for mapreduce.job.working.dir 539 DEBUG - Handling deprecation for dfs.namenode.name.dir.restore 540 DEBUG - Handling deprecation for mapreduce.jobtracker.handler.count 541 DEBUG - Handling deprecation for mapreduce.jobhistory.admin.address 542 DEBUG - Handling deprecation for yarn.app.mapreduce.client-am.ipc.max-retries 543 DEBUG - Handling deprecation for dfs.client.use.datanode.hostname 544 DEBUG - Handling deprecation for hadoop.util.hash.type 545 DEBUG - Handling deprecation for dfs.datanode.available-space-volume-choosing-policy.balanced-space-preference-fraction 546 DEBUG - Handling deprecation for dfs.datanode.dns.interface 547 DEBUG - Handling deprecation for io.seqfile.lazydecompress 548 DEBUG - Handling deprecation for dfs.namenode.lazypersist.file.scrub.interval.sec 549 DEBUG - Handling deprecation for yarn.nodemanager.disk-health-checker.min-healthy-disks 550 DEBUG - Handling deprecation for yarn.client.max-cached-nodemanagers-proxies 551 DEBUG - Handling deprecation for mapreduce.job.maxtaskfailures.per.tracker 552 DEBUG - Handling deprecation for mapreduce.tasktracker.healthchecker.script.timeout 553 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.search.attr.group.name 554 DEBUG - Handling deprecation for fs.df.interval 555 DEBUG - Handling deprecation for hadoop.security.crypto.buffer.size 556 DEBUG - Handling deprecation for dfs.namenode.kerberos.internal.spnego.principal 557 DEBUG - Handling deprecation for dfs.client.cached.conn.retry 558 DEBUG - Handling deprecation for mapreduce.job.reduce.class 559 DEBUG - Handling deprecation for mapreduce.job.map.class 560 DEBUG - Handling deprecation for mapreduce.job.reduce.shuffle.consumer.plugin.class 561 DEBUG - Handling deprecation for mapreduce.jobtracker.address 562 DEBUG - Handling deprecation for mapreduce.tasktracker.tasks.sleeptimebeforesigkill 563 DEBUG - Handling deprecation for dfs.journalnode.rpc-address 564 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-blocks-per-file 565 DEBUG - Handling deprecation for mapreduce.job.acl-view-job 566 DEBUG - Handling deprecation for yarn.app.mapreduce.am.job.committer.cancel-timeout 567 DEBUG - Handling deprecation for dfs.client.block.write.replace-datanode-on-failure.policy 568 DEBUG - Handling deprecation for mapreduce.shuffle.connection-keep-alive.enable 569 DEBUG - Handling deprecation for dfs.namenode.replication.interval 570 DEBUG - Handling deprecation for mapreduce.jobhistory.minicluster.fixed.ports 571 DEBUG - Handling deprecation for dfs.namenode.num.checkpoints.retained 572 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.address 573 DEBUG - Handling deprecation for mapreduce.tasktracker.http.address 574 DEBUG - Handling deprecation for mapreduce.jobhistory.admin.acl 575 DEBUG - Handling deprecation for dfs.datanode.directoryscan.threads 576 DEBUG - Handling deprecation for mapreduce.reduce.memory.mb 577 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.ssl 578 DEBUG - Handling deprecation for dfs.http.policy 579 DEBUG - Handling deprecation for mapreduce.task.merge.progress.records 580 DEBUG - Handling deprecation for dfs.heartbeat.interval 581 DEBUG - Handling deprecation for yarn.resourcemanager.system-metrics-publisher.dispatcher.pool-size 582 DEBUG - Handling deprecation for yarn.resourcemanager.recovery.enabled 583 DEBUG - Handling deprecation for net.topology.script.number.args 584 DEBUG - Handling deprecation for mapreduce.local.clientfactory.class.name 585 DEBUG - Handling deprecation for dfs.client-write-packet-size 586 DEBUG - Handling deprecation for yarn.dispatcher.drain-events.timeout 587 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.directory.search.timeout 588 DEBUG - Handling deprecation for io.native.lib.available 589 DEBUG - Handling deprecation for dfs.client.failover.connection.retries 590 DEBUG - Handling deprecation for yarn.nodemanager.disk-health-checker.interval-ms 591 DEBUG - Handling deprecation for dfs.blocksize 592 DEBUG - Handling deprecation for dfs.client.use.legacy.blockreader.local 593 DEBUG - Handling deprecation for yarn.resourcemanager.container-tokens.master-key-rolling-interval-secs 594 DEBUG - Handling deprecation for fs.s3a.connection.ssl.enabled 595 DEBUG - Handling deprecation for mapreduce.jobhistory.webapp.address 596 DEBUG - Handling deprecation for yarn.resourcemanager.resource-tracker.client.thread-count 597 DEBUG - Handling deprecation for yarn.client.failover-retries 598 DEBUG - Handling deprecation for hadoop.registry.jaas.context 599 DEBUG - Handling deprecation for dfs.blockreport.initialDelay 600 DEBUG - Handling deprecation for yarn.nodemanager.aux-services.mapreduce_shuffle.class 601 DEBUG - Handling deprecation for ha.health-monitor.rpc-timeout.ms 602 DEBUG - Handling deprecation for yarn.resourcemanager.zk-timeout-ms 603 DEBUG - Handling deprecation for mapreduce.reduce.markreset.buffer.percent 604 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.start-time-read-cache-size 605 DEBUG - Handling deprecation for dfs.ha.tail-edits.period 606 DEBUG - Handling deprecation for yarn.resourcemanager.client.thread-count 607 DEBUG - Handling deprecation for yarn.nodemanager.health-checker.script.timeout-ms 608 DEBUG - Handling deprecation for file.bytes-per-checksum 609 DEBUG - Handling deprecation for dfs.replication.max 610 DEBUG - Handling deprecation for dfs.namenode.max.extra.edits.segments.retained 611 DEBUG - Handling deprecation for io.map.index.skip 612 DEBUG - Handling deprecation for yarn.timeline-service.client.retry-interval-ms 613 DEBUG - Handling deprecation for mapreduce.task.timeout 614 DEBUG - Handling deprecation for dfs.datanode.du.reserved 615 DEBUG - Handling deprecation for mapreduce.reduce.cpu.vcores 616 DEBUG - Handling deprecation for dfs.support.append 617 DEBUG - Handling deprecation for dfs.client.file-block-storage-locations.num-threads 618 DEBUG - Handling deprecation for ftp.blocksize 619 DEBUG - Handling deprecation for yarn.nodemanager.container-manager.thread-count 620 DEBUG - Handling deprecation for ipc.server.listen.queue.size 621 DEBUG - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.enabled 622 DEBUG - Handling deprecation for hadoop.ssl.hostname.verifier 623 DEBUG - Handling deprecation for nfs.server.port 624 DEBUG - Handling deprecation for mapreduce.tasktracker.dns.interface 625 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.search.attr.member 626 DEBUG - Handling deprecation for mapreduce.job.userlog.retain.hours 627 DEBUG - Handling deprecation for mapreduce.tasktracker.outofband.heartbeat 628 DEBUG - Handling deprecation for fs.s3a.impl 629 DEBUG - Handling deprecation for hadoop.registry.system.acls 630 DEBUG - Handling deprecation for yarn.nodemanager.resource.memory-mb 631 DEBUG - Handling deprecation for yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds 632 DEBUG - Handling deprecation for dfs.webhdfs.user.provider.user.pattern 633 DEBUG - Handling deprecation for dfs.namenode.delegation.token.renew-interval 634 DEBUG - Handling deprecation for hadoop.ssl.keystores.factory.class 635 DEBUG - Handling deprecation for hadoop.registry.zk.retry.ceiling.ms 636 DEBUG - Handling deprecation for yarn.http.policy 637 DEBUG - Handling deprecation for dfs.datanode.sync.behind.writes 638 DEBUG - Handling deprecation for nfs.wtmax 639 DEBUG - Handling deprecation for fs.AbstractFileSystem.har.impl 640 DEBUG - Handling deprecation for dfs.client.read.shortcircuit.skip.checksum 641 DEBUG - Handling deprecation for hadoop.security.random.device.file.path 642 DEBUG - Handling deprecation for mapreduce.map.maxattempts 643 DEBUG - Handling deprecation for yarn.timeline-service.webapp.address 644 DEBUG - Handling deprecation for dfs.datanode.handler.count 645 DEBUG - Handling deprecation for hadoop.ssl.require.client.cert 646 DEBUG - Handling deprecation for ftp.client-write-packet-size 647 DEBUG - Handling deprecation for dfs.client.write.exclude.nodes.cache.expiry.interval.millis 648 DEBUG - Handling deprecation for mapreduce.jobhistory.cleaner.enable 649 DEBUG - Handling deprecation for fs.du.interval 650 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.retry-delay.max.ms 651 DEBUG - Handling deprecation for mapreduce.task.profile.reduces 652 DEBUG - Handling deprecation for ha.health-monitor.connect-retry-interval.ms 653 DEBUG - Handling deprecation for hadoop.fuse.connection.timeout 654 DEBUG - Handling deprecation for dfs.permissions.superusergroup 655 DEBUG - Handling deprecation for mapreduce.jobtracker.jobhistory.task.numberprogresssplits 656 DEBUG - Handling deprecation for fs.ftp.host.port 657 DEBUG - Handling deprecation for mapreduce.map.speculative 658 DEBUG - Handling deprecation for dfs.datanode.data.dir.perm 659 DEBUG - Handling deprecation for mapreduce.client.submit.file.replication 660 DEBUG - Handling deprecation for dfs.namenode.startup.delay.block.deletion.sec 661 DEBUG - Handling deprecation for s3native.blocksize 662 DEBUG - Handling deprecation for mapreduce.job.ubertask.maxmaps 663 DEBUG - Handling deprecation for dfs.namenode.replication.min 664 DEBUG - Handling deprecation for mapreduce.cluster.acls.enabled 665 DEBUG - Handling deprecation for hadoop.security.uid.cache.secs 666 DEBUG - Handling deprecation for nfs.allow.insecure.ports 667 DEBUG - Handling deprecation for yarn.nodemanager.localizer.fetch.thread-count 668 DEBUG - Handling deprecation for map.sort.class 669 DEBUG - Handling deprecation for fs.trash.checkpoint.interval 670 DEBUG - Handling deprecation for mapred.queue.default.acl-administer-jobs 671 DEBUG - Handling deprecation for yarn.timeline-service.generic-application-history.max-applications 672 DEBUG - Handling deprecation for dfs.image.transfer.timeout 673 DEBUG - Handling deprecation for dfs.namenode.name.dir 674 DEBUG - Handling deprecation for ipc.client.connect.timeout 675 DEBUG - Handling deprecation for yarn.app.mapreduce.am.staging-dir 676 DEBUG - Handling deprecation for fs.AbstractFileSystem.file.impl 677 DEBUG - Handling deprecation for yarn.nodemanager.env-whitelist 678 DEBUG - Handling deprecation for hadoop.registry.zk.retry.interval.ms 679 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage 680 DEBUG - Handling deprecation for yarn.timeline-service.keytab 681 DEBUG - Handling deprecation for dfs.image.compression.codec 682 DEBUG - Handling deprecation for mapreduce.job.reduces 683 DEBUG - Handling deprecation for mapreduce.job.complete.cancel.delegation.tokens 684 DEBUG - Handling deprecation for mapreduce.jobhistory.recovery.store.class 685 DEBUG - Handling deprecation for hadoop.security.group.mapping.ldap.search.filter.user 686 DEBUG - Handling deprecation for dfs.namenode.enable.retrycache 687 DEBUG - Handling deprecation for yarn.nodemanager.sleep-delay-before-sigkill.ms 688 DEBUG - Handling deprecation for mapreduce.jobhistory.joblist.cache.size 689 DEBUG - Handling deprecation for mapreduce.tasktracker.healthchecker.interval 690 DEBUG - Handling deprecation for mapreduce.jobtracker.heartbeats.in.second 691 DEBUG - Handling deprecation for mapreduce.jobtracker.persist.jobstatus.dir 692 DEBUG - Handling deprecation for dfs.namenode.backup.http-address 693 DEBUG - Handling deprecation for hadoop.rpc.protection 694 DEBUG - Handling deprecation for dfs.client.mmap.enabled 695 DEBUG - Handling deprecation for yarn.app.mapreduce.am.container.log.backups 696 DEBUG - Handling deprecation for ftp.stream-buffer-size 697 DEBUG - Handling deprecation for dfs.namenode.https-address 698 DEBUG - Handling deprecation for yarn.timeline-service.address 699 DEBUG - Handling deprecation for dfs.ha.log-roll.period 700 DEBUG - Handling deprecation for yarn.nodemanager.recovery.enabled 701 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.numinputfiles 702 DEBUG - Handling deprecation for hadoop.security.groups.negative-cache.secs 703 DEBUG - Handling deprecation for yarn.resourcemanager.admin.client.thread-count 704 DEBUG - Handling deprecation for dfs.datanode.fsdatasetcache.max.threads.per.volume 705 DEBUG - Handling deprecation for file.client-write-packet-size 706 DEBUG - Handling deprecation for hadoop.http.authentication.simple.anonymous.allowed 707 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.path 708 DEBUG - Handling deprecation for yarn.resourcemanager.proxy-user-privileges.enabled 709 DEBUG - Handling deprecation for dfs.datanode.drop.cache.behind.reads 710 DEBUG - Handling deprecation for yarn.nodemanager.log.retain-seconds 711 DEBUG - Handling deprecation for dfs.image.transfer.bandwidthPerSec 712 DEBUG - Handling deprecation for yarn.resourcemanager.work-preserving-recovery.scheduling-wait-ms 713 DEBUG - Handling deprecation for hadoop.registry.zk.session.timeout.ms 714 DEBUG - Handling deprecation for dfs.datanode.slow.io.warning.threshold.ms 715 DEBUG - Handling deprecation for mapreduce.tasktracker.instrumentation 716 DEBUG - Handling deprecation for ha.failover-controller.cli-check.rpc-timeout.ms 717 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.cgroups.hierarchy 718 DEBUG - Handling deprecation for dfs.namenode.write.stale.datanode.ratio 719 DEBUG - Handling deprecation for hadoop.security.groups.cache.warn.after.ms 720 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.timeout-ms 721 DEBUG - Handling deprecation for mapreduce.jobhistory.client.thread-count 722 DEBUG - Handling deprecation for io.mapfile.bloom.size 723 DEBUG - Handling deprecation for yarn.nodemanager.docker-container-executor.exec-name 724 DEBUG - Handling deprecation for yarn.resourcemanager.work-preserving-recovery.enabled 725 DEBUG - Handling deprecation for dfs.ha.fencing.ssh.connect-timeout 726 DEBUG - Handling deprecation for yarn.resourcemanager.zk-num-retries 727 DEBUG - Handling deprecation for hadoop.registry.zk.root 728 DEBUG - Handling deprecation for s3.bytes-per-checksum 729 DEBUG - Handling deprecation for yarn.app.mapreduce.am.container.log.limit.kb 730 DEBUG - Handling deprecation for dfs.namenode.edit.log.autoroll.check.interval.ms 731 DEBUG - Handling deprecation for fs.automatic.close 732 DEBUG - Handling deprecation for fs.trash.interval 733 DEBUG - Handling deprecation for dfs.journalnode.https-address 734 DEBUG - Handling deprecation for yarn.timeline-service.ttl-ms 735 DEBUG - Handling deprecation for hadoop.security.authentication 736 DEBUG - Handling deprecation for fs.defaultFS 737 DEBUG - Handling deprecation for nfs.rtmax 738 DEBUG - Handling deprecation for hadoop.ssl.server.conf 739 DEBUG - Handling deprecation for ipc.client.connect.max.retries 740 DEBUG - Handling deprecation for yarn.resourcemanager.delayed.delegation-token.removal-interval-ms 741 DEBUG - Handling deprecation for dfs.journalnode.http-address 742 DEBUG - Handling deprecation for dfs.namenode.xattrs.enabled 743 DEBUG - Handling deprecation for dfs.datanode.shared.file.descriptor.paths 744 DEBUG - Handling deprecation for mapreduce.jobtracker.taskscheduler 745 DEBUG - Handling deprecation for mapreduce.job.speculative.speculativecap 746 DEBUG - Handling deprecation for yarn.timeline-service.store-class 747 DEBUG - Handling deprecation for yarn.am.liveness-monitor.expiry-interval-ms 748 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.compress 749 DEBUG - Handling deprecation for dfs.user.home.dir.prefix 750 DEBUG - Handling deprecation for net.topology.node.switch.mapping.impl 751 DEBUG - Handling deprecation for dfs.namenode.replication.considerLoad 752 DEBUG - Handling deprecation for dfs.namenode.fs-limits.min-block-size 753 DEBUG - Handling deprecation for fs.swift.impl 754 DEBUG - Handling deprecation for dfs.namenode.audit.loggers 755 DEBUG - Handling deprecation for mapreduce.job.max.split.locations 756 DEBUG - Handling deprecation for yarn.resourcemanager.address 757 DEBUG - Handling deprecation for mapreduce.job.counters.max 758 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.enabled 759 DEBUG - Handling deprecation for dfs.client.block.write.retries 760 DEBUG - Handling deprecation for dfs.short.circuit.shared.memory.watcher.interrupt.check.ms 761 DEBUG - Handling deprecation for dfs.namenode.resource.checked.volumes.minimum 762 DEBUG - Handling deprecation for io.map.index.interval 763 DEBUG - Handling deprecation for mapred.child.java.opts 764 DEBUG - Handling deprecation for mapreduce.tasktracker.local.dir.minspacestart 765 DEBUG - Handling deprecation for mapreduce.client.progressmonitor.pollinterval 766 DEBUG - Handling deprecation for dfs.client.https.keystore.resource 767 DEBUG - Handling deprecation for mapreduce.task.profile.map.params 768 DEBUG - Handling deprecation for mapreduce.jobtracker.tasktracker.maxblacklists 769 DEBUG - Handling deprecation for mapreduce.job.queuename 770 DEBUG - Handling deprecation for hadoop.registry.zk.quorum 771 DEBUG - Handling deprecation for yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts 772 DEBUG - Handling deprecation for yarn.nodemanager.localizer.address 773 DEBUG - Handling deprecation for io.mapfile.bloom.error.rate 774 DEBUG - Handling deprecation for yarn.nodemanager.delete.thread-count 775 DEBUG - Handling deprecation for mapreduce.job.split.metainfo.maxsize 776 DEBUG - Handling deprecation for yarn.scheduler.maximum-allocation-vcores 777 DEBUG - Handling deprecation for mapred.mapper.new-api 778 DEBUG - Handling deprecation for mapreduce.job.dir 779 DEBUG - Handling deprecation for yarn.app.mapreduce.am.resource.mb 780 DEBUG - Handling deprecation for dfs.datanode.dns.nameserver 781 DEBUG - Handling deprecation for dfs.client.slow.io.warning.threshold.ms 782 DEBUG - Handling deprecation for mapreduce.job.reducer.preempt.delay.sec 783 DEBUG - Handling deprecation for yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb 784 DEBUG - Handling deprecation for mapreduce.map.output.compress.codec 785 DEBUG - Handling deprecation for dfs.namenode.accesstime.precision 786 DEBUG - Handling deprecation for fs.s3a.connection.maximum 787 DEBUG - Handling deprecation for mapreduce.map.log.level 788 DEBUG - Handling deprecation for io.seqfile.compress.blocksize 789 DEBUG - Handling deprecation for ipc.client.ping 790 DEBUG - Handling deprecation for mapreduce.tasktracker.taskcontroller 791 DEBUG - Handling deprecation for hadoop.security.groups.cache.secs 792 DEBUG - Handling deprecation for dfs.datanode.cache.revocation.timeout.ms 793 DEBUG - Handling deprecation for dfs.client.context 794 DEBUG - Handling deprecation for mapreduce.input.lineinputformat.linespermap 795 DEBUG - Handling deprecation for mapreduce.job.end-notification.max.attempts 796 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user 797 DEBUG - Handling deprecation for yarn.nodemanager.webapp.address 798 DEBUG - Handling deprecation for mapreduce.job.submithostname 799 DEBUG - Handling deprecation for mapreduce.jobhistory.recovery.enable 800 DEBUG - Handling deprecation for mapreduce.jobtracker.expire.trackers.interval 801 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.address 802 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.num.refill.threads 803 DEBUG - Handling deprecation for yarn.nodemanager.health-checker.interval-ms 804 DEBUG - Handling deprecation for mapreduce.jobhistory.loadedjobs.cache.size 805 DEBUG - Handling deprecation for hadoop.security.authorization 806 DEBUG - Handling deprecation for mapreduce.job.map.output.collector.class 807 DEBUG - Handling deprecation for mapreduce.am.max-attempts 808 DEBUG - Handling deprecation for fs.ftp.host 809 DEBUG - Handling deprecation for fs.s3a.attempts.maximum 810 DEBUG - Handling deprecation for yarn.app.mapreduce.am.scheduler.heartbeat.interval-ms 811 DEBUG - Handling deprecation for mapreduce.ifile.readahead 812 DEBUG - Handling deprecation for yarn.resourcemanager.scheduler.monitor.enable 813 DEBUG - Handling deprecation for yarn.resourcemanager.zk-retry-interval-ms 814 DEBUG - Handling deprecation for ha.zookeeper.session-timeout.ms 815 DEBUG - Handling deprecation for mapreduce.tasktracker.taskmemorymanager.monitoringinterval 816 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.parallelcopies 817 DEBUG - Handling deprecation for dfs.client.mmap.retry.timeout.ms 818 DEBUG - Handling deprecation for mapreduce.map.skip.maxrecords 819 DEBUG - Handling deprecation for mapreduce.job.output.value.class 820 DEBUG - Handling deprecation for dfs.namenode.avoid.read.stale.datanode 821 DEBUG - Handling deprecation for dfs.https.enable 822 DEBUG - Handling deprecation for yarn.timeline-service.webapp.https.address 823 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.read.timeout 824 DEBUG - Handling deprecation for dfs.namenode.list.encryption.zones.num.responses 825 DEBUG - Handling deprecation for yarn.nodemanager.remote-app-log-dir-suffix 826 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.compress.codec 827 DEBUG - Handling deprecation for mapreduce.jobtracker.instrumentation 828 DEBUG - Handling deprecation for dfs.blockreport.intervalMsec 829 DEBUG - Handling deprecation for ipc.client.connect.retry.interval 830 DEBUG - Handling deprecation for mapreduce.reduce.speculative 831 DEBUG - Handling deprecation for mapreduce.jobhistory.keytab 832 DEBUG - Handling deprecation for mapreduce.jobhistory.datestring.cache.size 833 DEBUG - Handling deprecation for dfs.datanode.balance.bandwidthPerSec 834 DEBUG - Handling deprecation for file.blocksize 835 DEBUG - Handling deprecation for yarn.resourcemanager.admin.address 836 DEBUG - Handling deprecation for mapreduce.map.cpu.vcores 837 DEBUG - Handling deprecation for yarn.nodemanager.container-monitor.procfs-tree.smaps-based-rss.enabled 838 DEBUG - Handling deprecation for yarn.resourcemanager.configuration.provider-class 839 DEBUG - Handling deprecation for yarn.resourcemanager.resource-tracker.address 840 DEBUG - Handling deprecation for mapreduce.tasktracker.local.dir.minspacekill 841 DEBUG - Handling deprecation for mapreduce.jobtracker.staging.root.dir 842 DEBUG - Handling deprecation for mapreduce.jobtracker.retiredjobs.cache.size 843 DEBUG - Handling deprecation for hadoop.registry.rm.enabled 844 DEBUG - Handling deprecation for ipc.client.connect.max.retries.on.timeouts 845 DEBUG - Handling deprecation for ha.zookeeper.acl 846 DEBUG - Handling deprecation for hadoop.security.crypto.codec.classes.aes.ctr.nopadding 847 DEBUG - Handling deprecation for yarn.nodemanager.local-dirs 848 DEBUG - Handling deprecation for mapreduce.app-submission.cross-platform 849 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.connect.timeout 850 DEBUG - Handling deprecation for dfs.block.access.key.update.interval 851 DEBUG - Handling deprecation for rpc.metrics.quantile.enable 852 DEBUG - Handling deprecation for dfs.block.access.token.lifetime 853 DEBUG - Handling deprecation for mapreduce.job.end-notification.retry.attempts 854 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-xattrs-per-inode 855 DEBUG - Handling deprecation for mapreduce.jobtracker.system.dir 856 DEBUG - Handling deprecation for dfs.client.file-block-storage-locations.timeout.millis 857 DEBUG - Handling deprecation for yarn.nodemanager.admin-env 858 DEBUG - Handling deprecation for mapreduce.jobtracker.jobhistory.block.size 859 DEBUG - Handling deprecation for yarn.log-aggregation.retain-seconds 860 DEBUG - Handling deprecation for mapreduce.tasktracker.indexcache.mb 861 DEBUG - Handling deprecation for yarn.timeline-service.handler-thread-count 862 DEBUG - Handling deprecation for dfs.namenode.checkpoint.check.period 863 DEBUG - Handling deprecation for yarn.resourcemanager.hostname 864 DEBUG - Handling deprecation for dfs.client.block.write.replace-datanode-on-failure.enable 865 DEBUG - Handling deprecation for net.topology.impl 866 DEBUG - Handling deprecation for dfs.datanode.directoryscan.interval 867 DEBUG - Handling deprecation for fs.s3a.multipart.purge.age 868 DEBUG - Handling deprecation for hadoop.security.java.secure.random.algorithm 869 DEBUG - Handling deprecation for yarn.nodemanager.container-monitor.interval-ms 870 DEBUG - Handling deprecation for dfs.default.chunk.view.size 871 DEBUG - Handling deprecation for fs.s3a.multipart.threshold 872 DEBUG - Handling deprecation for mapreduce.job.speculative.slownodethreshold 873 DEBUG - Handling deprecation for mapreduce.job.reduce.slowstart.completedmaps 874 DEBUG - Handling deprecation for mapred.reducer.new-api 875 DEBUG - Handling deprecation for hadoop.security.instrumentation.requires.admin 876 DEBUG - Handling deprecation for io.compression.codec.bzip2.library 877 DEBUG - Handling deprecation for hadoop.http.authentication.signature.secret.file 878 DEBUG - Handling deprecation for hadoop.registry.secure 879 DEBUG - Handling deprecation for dfs.namenode.safemode.min.datanodes 880 DEBUG - Handling deprecation for yarn.nodemanager.localizer.cache.target-size-mb 881 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.inputdir 882 DEBUG - Handling deprecation for mapreduce.reduce.maxattempts 883 DEBUG - Handling deprecation for dfs.datanode.https.address 884 DEBUG - Handling deprecation for s3native.replication 885 DEBUG - Handling deprecation for dfs.namenode.inotify.max.events.per.rpc 886 DEBUG - Handling deprecation for dfs.namenode.path.based.cache.retry.interval.ms 887 DEBUG - Handling deprecation for mapreduce.reduce.skip.proc.count.autoincr 888 DEBUG - Handling deprecation for dfs.datanode.cache.revocation.polling.ms 889 DEBUG - Handling deprecation for mapreduce.jobhistory.cleaner.interval-ms 890 DEBUG - Handling deprecation for file.replication 891 DEBUG - Handling deprecation for hadoop.hdfs.configuration.version 892 DEBUG - Handling deprecation for ipc.client.idlethreshold 893 DEBUG - Handling deprecation for hadoop.tmp.dir 894 DEBUG - Handling deprecation for yarn.resourcemanager.store.class 895 DEBUG - Handling deprecation for mapreduce.jobhistory.address 896 DEBUG - Handling deprecation for mapreduce.jobtracker.restart.recover 897 DEBUG - Handling deprecation for mapreduce.cluster.local.dir 898 DEBUG - Handling deprecation for yarn.client.nodemanager-client-async.thread-pool-max-size 899 DEBUG - Handling deprecation for dfs.namenode.decommission.nodes.per.interval 900 DEBUG - Handling deprecation for mapreduce.job.inputformat.class 901 DEBUG - Handling deprecation for yarn.nodemanager.resource.cpu-vcores 902 DEBUG - Handling deprecation for dfs.namenode.reject-unresolved-dn-topology-mapping 903 DEBUG - Handling deprecation for dfs.namenode.delegation.key.update-interval 904 DEBUG - Handling deprecation for fs.s3.buffer.dir 905 DEBUG - Handling deprecation for dfs.client.read.shortcircuit.streams.cache.expiry.ms 906 DEBUG - Handling deprecation for dfs.namenode.support.allow.format 907 DEBUG - Handling deprecation for yarn.nodemanager.remote-app-log-dir 908 DEBUG - Handling deprecation for mapreduce.map.memory.mb 909 DEBUG - Handling deprecation for dfs.namenode.edit.log.autoroll.multiplier.threshold 910 DEBUG - Handling deprecation for hadoop.work.around.non.threadsafe.getpwuid 911 DEBUG - Handling deprecation for mapreduce.task.profile.reduce.params 912 DEBUG - Handling deprecation for yarn.timeline-service.client.max-retries 913 DEBUG - Handling deprecation for dfs.ha.automatic-failover.enabled 914 DEBUG - Handling deprecation for dfs.namenode.edits.noeditlogchannelflush 915 DEBUG - Handling deprecation for dfs.namenode.stale.datanode.interval 916 DEBUG - Handling deprecation for mapreduce.shuffle.transfer.buffer.size 917 DEBUG - Handling deprecation for mapreduce.jobtracker.persist.jobstatus.active 918 DEBUG - Handling deprecation for dfs.namenode.logging.level 919 DEBUG - Handling deprecation for yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs 920 DEBUG - Handling deprecation for yarn.nodemanager.log-dirs 921 DEBUG - Handling deprecation for ha.health-monitor.sleep-after-disconnect.ms 922 DEBUG - Handling deprecation for yarn.resourcemanager.fs.state-store.uri 923 DEBUG - Handling deprecation for dfs.namenode.checkpoint.edits.dir 924 DEBUG - Handling deprecation for yarn.resourcemanager.keytab 925 DEBUG - Handling deprecation for hadoop.rpc.socket.factory.class.default 926 DEBUG - Handling deprecation for yarn.resourcemanager.fail-fast 927 DEBUG - Handling deprecation for dfs.datanode.http.address 928 DEBUG - Handling deprecation for mapreduce.task.profile 929 DEBUG - Handling deprecation for mapreduce.jobhistory.move.interval-ms 930 DEBUG - Handling deprecation for dfs.namenode.edits.dir 931 DEBUG - Handling deprecation for dfs.storage.policy.enabled 932 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.size 933 DEBUG - Handling deprecation for hadoop.fuse.timer.period 934 DEBUG - Handling deprecation for mapreduce.jobhistory.http.policy 935 DEBUG - Handling deprecation for mapreduce.jobhistory.intermediate-done-dir 936 DEBUG - Handling deprecation for mapreduce.map.skip.proc.count.autoincr 937 DEBUG - Handling deprecation for fs.AbstractFileSystem.viewfs.impl 938 DEBUG - Handling deprecation for mapreduce.job.speculative.slowtaskthreshold 939 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled 940 DEBUG - Handling deprecation for s3native.stream-buffer-size 941 DEBUG - Handling deprecation for yarn.nodemanager.delete.debug-delay-sec 942 DEBUG - Handling deprecation for dfs.secondary.namenode.kerberos.internal.spnego.principal 943 DEBUG - Handling deprecation for dfs.datanode.available-space-volume-choosing-policy.balanced-space-threshold 944 DEBUG - Handling deprecation for fs.s3n.multipart.uploads.block.size 945 DEBUG - Handling deprecation for dfs.namenode.safemode.threshold-pct 946 DEBUG - Handling deprecation for mapreduce.ifile.readahead.bytes 947 DEBUG - Handling deprecation for yarn.scheduler.maximum-allocation-mb 948 DEBUG - Handling deprecation for ipc.client.fallback-to-simple-auth-allowed 949 DEBUG - Handling deprecation for fs.har.impl.disable.cache 950 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.read-cache-size 951 DEBUG - Handling deprecation for yarn.timeline-service.hostname 952 DEBUG - Handling deprecation for s3native.bytes-per-checksum 953 DEBUG - Handling deprecation for mapreduce.job.committer.setup.cleanup.needed 954 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms 955 DEBUG - Handling deprecation for fs.s3a.paging.maximum 956 DEBUG - Handling deprecation for yarn.client.nodemanager-connect.retry-interval-ms 957 DEBUG - Handling deprecation for yarn.nodemanager.log-aggregation.compression-type 958 DEBUG - Handling deprecation for yarn.app.mapreduce.am.job.committer.commit-window 959 DEBUG - Handling deprecation for hadoop.http.authentication.type 960 DEBUG - Handling deprecation for dfs.client.failover.sleep.base.millis 961 DEBUG - Handling deprecation for mapreduce.job.submithostaddress 962 DEBUG - Handling deprecation for yarn.nodemanager.vmem-check-enabled 963 DEBUG - Handling deprecation for hadoop.jetty.logs.serve.aliases 964 DEBUG - Handling deprecation for ha.failover-controller.graceful-fence.rpc-timeout.ms 965 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.input.buffer.percent 966 DEBUG - Handling deprecation for dfs.datanode.max.transfer.threads 967 DEBUG - Handling deprecation for mapreduce.reduce.merge.inmem.threshold 968 DEBUG - Handling deprecation for mapreduce.task.io.sort.mb 969 DEBUG - Handling deprecation for dfs.namenode.acls.enabled 970 DEBUG - Handling deprecation for yarn.client.application-client-protocol.poll-interval-ms 971 DEBUG - Handling deprecation for hadoop.security.kms.client.authentication.retry-count 972 DEBUG - Handling deprecation for dfs.namenode.handler.count 973 DEBUG - Handling deprecation for yarn.resourcemanager.connect.max-wait.ms 974 DEBUG - Handling deprecation for dfs.namenode.retrycache.heap.percent 975 DEBUG - Handling deprecation for yarn.timeline-service.enabled 976 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users 977 DEBUG - Handling deprecation for hadoop.ssl.client.conf 978 DEBUG - Handling deprecation for yarn.resourcemanager.container.liveness-monitor.interval-ms 979 DEBUG - Handling deprecation for yarn.nodemanager.vmem-pmem-ratio 980 DEBUG - Handling deprecation for mapreduce.client.completion.pollinterval 981 DEBUG - Handling deprecation for yarn.app.mapreduce.client.max-retries 982 DEBUG - Handling deprecation for hadoop.ssl.enabled 983 DEBUG - Handling deprecation for fs.client.resolve.remote.symlinks 984 DEBUG - Handling deprecation for fs.AbstractFileSystem.hdfs.impl 985 DEBUG - Handling deprecation for mapreduce.tasktracker.reduce.tasks.maximum 986 DEBUG - Handling deprecation for yarn.nodemanager.hostname 987 DEBUG - Handling deprecation for mapreduce.reduce.input.buffer.percent 988 DEBUG - Handling deprecation for fs.s3a.multipart.purge 989 DEBUG - Handling deprecation for yarn.app.mapreduce.am.command-opts 990 DEBUG - Handling deprecation for dfs.namenode.invalidate.work.pct.per.iteration 991 DEBUG - Handling deprecation for dfs.bytes-per-checksum 992 DEBUG - Handling deprecation for yarn.resourcemanager.webapp.https.address 993 DEBUG - Handling deprecation for dfs.replication 994 DEBUG - Handling deprecation for dfs.datanode.block.id.layout.upgrade.threads 995 DEBUG - Handling deprecation for mapreduce.shuffle.ssl.file.buffer.size 996 DEBUG - Handling deprecation for dfs.namenode.list.cache.directives.num.responses 997 DEBUG - Handling deprecation for dfs.permissions.enabled 998 DEBUG - Handling deprecation for mapreduce.jobtracker.maxtasks.perjob 999 DEBUG - Handling deprecation for dfs.datanode.use.datanode.hostname 1000 DEBUG - Handling deprecation for mapreduce.task.userlog.limit.kb 1001 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-directory-items 1002 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.low-watermark 1003 DEBUG - Handling deprecation for fs.s3a.buffer.dir 1004 DEBUG - Handling deprecation for s3.client-write-packet-size 1005 DEBUG - Handling deprecation for hadoop.user.group.static.mapping.overrides 1006 DEBUG - Handling deprecation for mapreduce.shuffle.max.threads 1007 DEBUG - Handling deprecation for hadoop.security.kms.client.encrypted.key.cache.expiry 1008 DEBUG - Handling deprecation for dfs.client.failover.sleep.max.millis 1009 DEBUG - Handling deprecation for mapreduce.job.maps 1010 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-component-length 1011 DEBUG - Handling deprecation for yarn.fail-fast 1012 DEBUG - Handling deprecation for hadoop.ssl.enabled.protocols 1013 DEBUG - Handling deprecation for s3.blocksize 1014 DEBUG - Handling deprecation for mapreduce.map.output.compress 1015 DEBUG - Handling deprecation for dfs.namenode.edits.journal-plugin.qjournal 1016 DEBUG - Handling deprecation for dfs.namenode.datanode.registration.ip-hostname-check 1017 DEBUG - Handling deprecation for yarn.nodemanager.pmem-check-enabled 1018 DEBUG - Handling deprecation for dfs.client.short.circuit.replica.stale.threshold.ms 1019 DEBUG - Handling deprecation for dfs.client.https.need-auth 1020 DEBUG - Handling deprecation for yarn.scheduler.minimum-allocation-mb 1021 DEBUG - Handling deprecation for mapreduce.jobhistory.max-age-ms 1022 DEBUG - Handling deprecation for yarn.timeline-service.http-authentication.type 1023 DEBUG - Handling deprecation for ftp.replication 1024 DEBUG - Handling deprecation for dfs.namenode.secondary.https-address 1025 DEBUG - Handling deprecation for dfs.blockreport.split.threshold 1026 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.split.minsize 1027 DEBUG - Handling deprecation for fs.s3n.block.size 1028 DEBUG - Handling deprecation for mapreduce.job.token.tracking.ids.enabled 1029 DEBUG - Handling deprecation for yarn.ipc.rpc.class 1030 DEBUG - Handling deprecation for dfs.namenode.num.extra.edits.retained 1031 DEBUG - Handling deprecation for yarn.nodemanager.localizer.cache.cleanup.interval-ms 1032 DEBUG - Handling deprecation for hadoop.http.staticuser.user 1033 DEBUG - Handling deprecation for mapreduce.jobhistory.move.thread-count 1034 DEBUG - Handling deprecation for fs.s3a.multipart.size 1035 DEBUG - Handling deprecation for mapreduce.job.jvm.numtasks 1036 DEBUG - Handling deprecation for mapreduce.task.profile.maps 1037 DEBUG - Handling deprecation for dfs.datanode.max.locked.memory 1038 DEBUG - Handling deprecation for dfs.cachereport.intervalMsec 1039 DEBUG - Handling deprecation for mapreduce.shuffle.port 1040 DEBUG - Handling deprecation for yarn.resourcemanager.nodemanager.minimum.version 1041 DEBUG - Handling deprecation for mapreduce.shuffle.connection-keep-alive.timeout 1042 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.merge.percent 1043 DEBUG - Handling deprecation for mapreduce.jobtracker.http.address 1044 DEBUG - Handling deprecation for mapreduce.task.skip.start.attempts 1045 DEBUG - Handling deprecation for yarn.resourcemanager.connect.retry-interval.ms 1046 DEBUG - Handling deprecation for yarn.scheduler.minimum-allocation-vcores 1047 DEBUG - Handling deprecation for mapreduce.task.io.sort.factor 1048 DEBUG - Handling deprecation for dfs.namenode.checkpoint.dir 1049 DEBUG - Handling deprecation for nfs.exports.allowed.hosts 1050 DEBUG - Handling deprecation for tfile.fs.input.buffer.size 1051 DEBUG - Handling deprecation for fs.s3.block.size 1052 DEBUG - Handling deprecation for tfile.io.chunk.size 1053 DEBUG - Handling deprecation for fs.s3n.multipart.copy.block.size 1054 DEBUG - Handling deprecation for io.serializations 1055 DEBUG - Handling deprecation for yarn.resourcemanager.max-completed-applications 1056 DEBUG - Handling deprecation for mapreduce.jobhistory.principal 1057 DEBUG - Handling deprecation for mapreduce.output.fileoutputformat.outputdir 1058 DEBUG - Handling deprecation for yarn.resourcemanager.ha.automatic-failover.zk-base-path 1059 DEBUG - Handling deprecation for mapreduce.reduce.shuffle.fetch.retry.interval-ms 1060 DEBUG - Handling deprecation for mapreduce.job.end-notification.retry.interval 1061 DEBUG - Handling deprecation for dfs.namenode.backup.address 1062 DEBUG - Handling deprecation for fs.s3n.multipart.uploads.enabled 1063 DEBUG - Handling deprecation for io.seqfile.sorter.recordlimit 1064 DEBUG - Handling deprecation for dfs.block.access.token.enable 1065 DEBUG - Handling deprecation for s3native.client-write-packet-size 1066 DEBUG - Handling deprecation for dfs.namenode.fs-limits.max-xattr-size 1067 DEBUG - Handling deprecation for ftp.bytes-per-checksum 1068 DEBUG - Handling deprecation for hadoop.security.group.mapping 1069 DEBUG - Handling deprecation for dfs.client.domain.socket.data.traffic 1070 DEBUG - Handling deprecation for dfs.client.read.shortcircuit.streams.cache.size 1071 DEBUG - Handling deprecation for fs.s3a.connection.timeout 1072 DEBUG - Handling deprecation for mapreduce.job.end-notification.max.retry.interval 1073 DEBUG - Handling deprecation for yarn.acl.enable 1074 DEBUG - Handling deprecation for yarn.nm.liveness-monitor.expiry-interval-ms 1075 DEBUG - Handling deprecation for mapreduce.input.fileinputformat.list-status.num-threads 1076 DEBUG - Handling deprecation for dfs.client.mmap.cache.size 1077 DEBUG - Handling deprecation for mapreduce.tasktracker.map.tasks.maximum 1078 DEBUG - Handling deprecation for yarn.timeline-service.ttl-enable 1079 DEBUG - Handling deprecation for yarn.nodemanager.linux-container-executor.resources-handler.class 1080 DEBUG - Handling deprecation for dfs.namenode.max.objects 1081 DEBUG - Handling deprecation for yarn.resourcemanager.state-store.max-completed-applications 1082 DEBUG - Handling deprecation for dfs.namenode.delegation.token.max-lifetime 1083 DEBUG - Handling deprecation for mapreduce.job.classloader 1084 DEBUG - Handling deprecation for yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size 1085 DEBUG - Handling deprecation for mapreduce.job.hdfs-servers 1086 DEBUG - Handling deprecation for dfs.datanode.hdfs-blocks-metadata.enabled 1087 DEBUG - Handling deprecation for mapreduce.tasktracker.dns.nameserver 1088 DEBUG - Handling deprecation for dfs.datanode.readahead.bytes 1089 DEBUG - Handling deprecation for dfs.datanode.bp-ready.timeout 1090 DEBUG - Handling deprecation for mapreduce.job.ubertask.maxreduces 1091 DEBUG - Handling deprecation for dfs.image.compress 1092 DEBUG - Handling deprecation for mapreduce.shuffle.ssl.enabled 1093 DEBUG - Handling deprecation for yarn.log-aggregation-enable 1094 DEBUG - Handling deprecation for mapreduce.tasktracker.report.address 1095 DEBUG - Handling deprecation for mapreduce.tasktracker.http.threads 1096 DEBUG - Handling deprecation for dfs.stream-buffer-size 1097 DEBUG - Handling deprecation for tfile.fs.output.buffer.size 1098 DEBUG - Handling deprecation for fs.permissions.umask-mode 1099 DEBUG - Handling deprecation for dfs.client.datanode-restart.timeout 1100 DEBUG - Handling deprecation for dfs.namenode.resource.du.reserved 1101 DEBUG - Handling deprecation for yarn.resourcemanager.am.max-attempts 1102 DEBUG - Handling deprecation for yarn.nodemanager.resource.percentage-physical-cpu-limit 1103 DEBUG - Handling deprecation for ha.failover-controller.graceful-fence.connection.retries 1104 DEBUG - Handling deprecation for dfs.datanode.drop.cache.behind.writes 1105 DEBUG - Handling deprecation for mapreduce.map.output.value.class 1106 DEBUG - Handling deprecation for hadoop.common.configuration.version 1107 DEBUG - Handling deprecation for mapreduce.job.ubertask.enable 1108 DEBUG - Handling deprecation for yarn.app.mapreduce.am.resource.cpu-vcores 1109 DEBUG - Handling deprecation for dfs.namenode.replication.work.multiplier.per.iteration 1110 DEBUG - Handling deprecation for mapreduce.job.acl-modify-job 1111 DEBUG - Handling deprecation for io.seqfile.local.dir 1112 DEBUG - Handling deprecation for yarn.resourcemanager.system-metrics-publisher.enabled 1113 DEBUG - Handling deprecation for fs.s3.sleepTimeSeconds 1114 DEBUG - Handling deprecation for mapreduce.client.output.filter 1115 INFO - The url to track the job: http://localhost:8080/ 1116 INFO - Running job: job_local1553403857_0001 1117 INFO - OutputCommitter set in config null 1118 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323) 1119 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323) 1120 INFO - OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter 1121 DEBUG - Starting mapper thread pool executor. 1122 DEBUG - Max local threads: 1 1123 DEBUG - Map tasks to process: 1 1124 INFO - Waiting for map tasks 1125 INFO - Starting task: attempt_local1553403857_0001_m_000000_0 1126 DEBUG - currentIndex 0 0:0 1127 DEBUG - mapreduce.cluster.local.dir for child : /tmp/hadoop-SYJ/mapred/local/localRunner//SYJ/jobcache/job_local1553403857_0001/attempt_local1553403857_0001_m_000000_0 1128 DEBUG - using new api for output committer 1129 INFO - ProcfsBasedProcessTree currently is supported only on Linux. 1130 INFO - Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@5e2c17f7 1131 INFO - Processing split: file:/c:/wordcount/input/words.txt:0+83 1132 DEBUG - Trying map output collector class: org.apache.hadoop.mapred.MapTask$MapOutputBuffer 1133 INFO - (EQUATOR) 0 kvi 26214396(104857584) 1134 INFO - mapreduce.task.io.sort.mb: 100 1135 INFO - soft limit at 83886080 1136 INFO - bufstart = 0; bufvoid = 104857600 1137 INFO - kvstart = 26214396; length = 6553600 1138 INFO - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer 1139 INFO - 1140 INFO - Starting flush of map output 1141 INFO - Spilling map output 1142 INFO - bufstart = 0; bufend = 129; bufvoid = 104857600 1143 INFO - kvstart = 26214396(104857584); kvend = 26214352(104857408); length = 45/6553600 1144 INFO - Finished spill 0 1145 INFO - Task:attempt_local1553403857_0001_m_000000_0 is done. And is in the process of committing 1146 INFO - map 1147 INFO - Task ‘attempt_local1553403857_0001_m_000000_0‘ done. 1148 INFO - Finishing task: attempt_local1553403857_0001_m_000000_0 1149 INFO - map task executor complete. 1150 DEBUG - Starting reduce thread pool executor. 1151 DEBUG - Max local threads: 1 1152 DEBUG - Reduce tasks to process: 1 1153 INFO - Waiting for reduce tasks 1154 INFO - Starting task: attempt_local1553403857_0001_r_000000_0 1155 DEBUG - currentIndex 0 0:0 1156 DEBUG - mapreduce.cluster.local.dir for child : /tmp/hadoop-SYJ/mapred/local/localRunner//SYJ/jobcache/job_local1553403857_0001/attempt_local1553403857_0001_r_000000_0 1157 DEBUG - using new api for output committer 1158 INFO - ProcfsBasedProcessTree currently is supported only on Linux. 1159 INFO - Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@41217e67 1160 INFO - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@25071521 1161 INFO - MergerManager: memoryLimit=1287946240, maxSingleShuffleLimit=321986560, mergeThreshold=850044544, ioSortFactor=10, memToMemMergeOutputsThreshold=10 1162 INFO - attempt_local1553403857_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events 1163 DEBUG - Got 0 map completion events from 0 1164 DEBUG - GetMapEventsThread about to sleep for 1000 1165 DEBUG - LocalFetcher 1 going to fetch: attempt_local1553403857_0001_m_000000_0 1166 DEBUG - attempt_local1553403857_0001_m_000000_0: Proceeding with shuffle since usedMemory (0) is lesser than memoryLimit (1287946240).CommitMemory is (0) 1167 INFO - localfetcher#1 about to shuffle output of map attempt_local1553403857_0001_m_000000_0 decomp: 155 len: 159 to MEMORY 1168 INFO - Read 155 bytes from map-output for attempt_local1553403857_0001_m_000000_0 1169 INFO - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155 1170 DEBUG - map attempt_local1553403857_0001_m_000000_0 done 1 / 1 copied. 1171 INFO - EventFetcher is interrupted.. Returning 1172 INFO - 1 / 1 copied. 1173 INFO - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs 1174 INFO - Merging 1 sorted segments 1175 INFO - Down to the last merge-pass, with 1 segments left of total size: 145 bytes 1176 INFO - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit 1177 DEBUG - Disk file: /tmp/hadoop-SYJ/mapred/local/localRunner/SYJ/jobcache/job_local1553403857_0001/attempt_local1553403857_0001_r_000000_0/output/map_0.out.merged Length is 159 1178 INFO - Merging 1 files, 159 bytes from disk 1179 INFO - Merging 0 segments, 0 bytes from memory into reduce 1180 INFO - Merging 1 sorted segments 1181 INFO - Down to the last merge-pass, with 1 segments left of total size: 145 bytes 1182 INFO - 1 / 1 copied. 1183 INFO - mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords 1184 INFO - Task:attempt_local1553403857_0001_r_000000_0 is done. And is in the process of committing 1185 INFO - 1 / 1 copied. 1186 INFO - Task attempt_local1553403857_0001_r_000000_0 is allowed to commit now 1187 INFO - Saved output of task ‘attempt_local1553403857_0001_r_000000_0‘ to file:/c:/wordcount/output/_temporary/0/task_local1553403857_0001_r_000000 1188 INFO - reduce > reduce 1189 INFO - Task ‘attempt_local1553403857_0001_r_000000_0‘ done. 1190 INFO - Finishing task: attempt_local1553403857_0001_r_000000_0 1191 INFO - reduce task executor complete. 1192 DEBUG - Merging data from DeprecatedRawLocalFileStatus{path=file:/c:/wordcount/output/_temporary/0/task_local1553403857_0001_r_000000; isDirectory=true; modification_time=1484061974082; access_time=0; owner=; group=; permission=rwxrwxrwx; isSymlink=false} to file:/c:/wordcount/output 1193 DEBUG - Merging data from DeprecatedRawLocalFileStatus{path=file:/c:/wordcount/output/_temporary/0/task_local1553403857_0001_r_000000/part-r-00000; isDirectory=false; length=62; replication=1; blocksize=33554432; modification_time=1484061974091; access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false} to file:/c:/wordcount/output/part-r-00000 1194 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331) 1195 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323) 1196 INFO - Job job_local1553403857_0001 running in uber mode : false 1197 INFO - map 100% reduce 100% 1198 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.getTaskCompletionEvents(Job.java:677) 1199 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323) 1200 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323) 1201 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.getTaskCompletionEvents(Job.java:677) 1202 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323) 1203 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323) 1204 INFO - Job job_local1553403857_0001 completed successfully 1205 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.getCounters(Job.java:765) 1206 INFO - Counters: 33 1207 File System Counters 1208 FILE: Number of bytes read=822 1209 FILE: Number of bytes written=505185 1210 FILE: Number of read operations=0 1211 FILE: Number of large read operations=0 1212 FILE: Number of write operations=0 1213 Map-Reduce Framework 1214 Map input records=4 1215 Map output records=12 1216 Map output bytes=129 1217 Map output materialized bytes=159 1218 Input split bytes=99 1219 Combine input records=0 1220 Combine output records=0 1221 Reduce input groups=7 1222 Reduce shuffle bytes=159 1223 Reduce input records=12 1224 Reduce output records=7 1225 Spilled Records=24 1226 Shuffled Maps =1 1227 Failed Shuffles=0 1228 Merged Map outputs=1 1229 GC time elapsed (ms)=0 1230 CPU time spent (ms)=0 1231 Physical memory (bytes) snapshot=0 1232 Virtual memory (bytes) snapshot=0 1233 Total committed heap usage (bytes)=457703424 1234 Shuffle Errors 1235 BAD_ID=0 1236 CONNECTION=0 1237 IO_ERROR=0 1238 WRONG_LENGTH=0 1239 WRONG_MAP=0 1240 WRONG_REDUCE=0 1241 File Input Format Counters 1242 Bytes Read=83 1243 File Output Format Counters 1244 Bytes Written=74 1245 DEBUG - PrivilegedAction as:SYJ (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)
19.到"c:/wordcount"目录下面可以看到生成的output目录，下面有一些输出结果文件，其中"part-r-00000"文件就是程序运算的结果：
如何在Windows下面运行hadoop的MapReduce程序
原文：http://www.cnblogs.com/jun1019/p/6271303.html
