开发环境设置：
1.首先去Apach页面上下载最新版本的MRUnit，根据Hadoop版本来选择对应的版本。将jar包添加到IDE的classpath中。
2.下载最新版本mokito和JUnit jar包，同样将其添加到classpath中。
通过Maven添加依赖：
<dependency>
<groupId>org.apache.mrunit</groupId>
<artifactId>mrunit</artifactId>
<version>0.9.0-incubating</version>
<classifier>hadoop1</classifier>
</dependency>
Use Classifier as hadoop2 if you are using Hadoop 2 version
MRUnit测试实例：
MRUnit测试框架基于JUnit。下面是使用MRUnit对MapReduce程序进行单元测试的实例。
数据记录如下：
MapReduce 程序对这些记录进行分析, 找到所有wiCDRType为1的记录, 并记录下对应的SMS Status码. 例如Mapper的输出结果如下：
6, 10, 1
Reducer将它们作为输入，然后输出记录中特定的状态码出现的次数。相应的Mapper、Reducer代码如下：
public class SMSCDRMapper extends Mapper<LongWritable, Text, Text, IntWritable> { private Text status = new Text(); private final static IntWritable addOne = new IntWritable(1); /** * Returns the SMS status code and its count */ protected void map(LongWritable key, Text value, Context context) throws java.io.IOException, InterruptedException { //655209;1;796764372490213;804422938115889;6 is the Sample record format String[] line = value.toString().split(";"); // If record is of SMS CDR if (Integer.parseInt(line[1]) == 1) { status.set(line[4]); context.write(status, addOne); } } }
相应的Reducer代码如下：
public class SMSCDRReducer extends Reducer<Text, IntWritable, Text, IntWritable> { protected void reduce(Text key, Iterable<IntWritable> values, Context context) throws java.io.IOException, InterruptedException { int sum = 0; for (IntWritable value : values) { sum += value.get(); } context.write(key, new IntWritable(sum)); } }
The MRUnit test class for the Mapper, Reduce and full MapReduce is
完整的MRUnit测试类如下：
import java.util.ArrayList; import java.util.List; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mrunit.mapreduce.MapDriver; import org.apache.hadoop.mrunit.mapreduce.MapReduceDriver; import org.apache.hadoop.mrunit.mapreduce.ReduceDriver; import org.junit.Before; import org.junit.Test; public class SMSCDRMapperReducerTest { MapDriver<LongWritable, Text, Text, IntWritable> mapDriver; ReduceDriver<Text, IntWritable, Text, IntWritable> reduceDriver; MapReduceDriver<LongWritable, Text, Text, IntWritable, Text, IntWritable> mapReduceDriver; @Before public void setUp() { SMSCDRMapper mapper = new SMSCDRMapper(); SMSCDRReducer reducer = new SMSCDRReducer(); mapDriver = MapDriver.newMapDriver(mapper); reduceDriver = ReduceDriver.newReduceDriver(reducer); mapReduceDriver = MapReduceDriver.newMapReduceDriver(mapper, reducer); } @Test public void testMapper() { mapDriver.withInput(new LongWritable(), new Text( "655209;1;796764372490213;804422938115889;6")); mapDriver.withOutput(new Text("6"), new IntWritable(1)); mapDriver.runTest(); } @Test public void testReducer() { List<IntWritable> values = new ArrayList<IntWritable>(); values.add(new IntWritable(1)); values.add(new IntWritable(1)); reduceDriver.withInput(new Text("6"), values); reduceDriver.withOutput(new Text("6"), new IntWritable(2)); reduceDriver.runTest(); } @Test public void testMapReduce() { mapReduceDriver.withInput(new LongWritable(), new Text( "655209;1;796764372490213;804422938115889;6")); List<IntWritable> values = new ArrayList<IntWritable>(); values.add(new IntWritable(1)); values.add(new IntWritable(1)); mapReduceDriver.withOutput(new Text("6"), new IntWritable(2)); mapReduceDriver.runTest(); } }
运行测试类，如果Mapper代码正确则运行通过，否则失败。
测试计数器：
计数器的通常用处是跟踪数据中格式不正确的记录。例如当输入CDR记录部位SMS类型，Mapper则忽略该数据、同时计数器递增。
增加计数器后的Mapper如下：
public class SMSCDRMapper extends Mapper<LongWritable, Text, Text, IntWritable> { private Text status = new Text(); private final static IntWritable addOne = new IntWritable(1); static enum CDRCounter { NonSMSCDR; }; /** * Returns the SMS status code and its count */ protected void map(LongWritable key, Text value, Context context) throws java.io.IOException, InterruptedException { String[] line = value.toString().split(";"); // If record is of SMS CDR if (Integer.parseInt(line[1]) == 1) { status.set(line[4]); context.write(status, addOne); } else {// CDR record is not of type SMS so increment the counter context.getCounter(CDRCounter.NonSMSCDR).increment(1); } } }
修改后的test Mapper如下：
public void testMapper() { mapDriver.withInput(new LongWritable(), new Text( "655209;0;796764372490213;804422938115889;6")); //mapDriver.withOutput(new Text("6"), new IntWritable(1)); mapDriver.runTest(); assertEquals("Expected 1 counter increment", 1, mapDriver.getCounters() .findCounter(CDRCounter.NonSMSCDR).getValue()); }
当CDR不为SMS类型时，计数器递增。这里通过断言来进行检查判断。当然可以通过同样的方式为Reducer添加计数器。
测试参数传递：
下面通过Configuration 类来给Testing传递参数。
在Mapper和Reducer类中通过Configuration.get()获取配置参数。在测试类中创建新配置对象，
Configuration conf = new Configuration();
在SetUp方法中添加一下代码：
mapDriver.setConfiguration(conf); conf.set("myParameter1", "20"); conf.set("myParameter2", "23");
测试类需要将这些参数传递给mapper.
参考：
https://cwiki.apache.org/confluence/display/MRUNIT/MRUnit+Tutorial
原文：http://www.cnblogs.com/wmymartin/p/4893377.html