Eclipse集成hadoop插件开发环境

?6.在ecplise里面window->preferences里找到 Hadoop Map/Reduce 把本地安装的hadoop路径指定到这。
?
7.先确认hadoop集群是否启动，然后

在第二步的Map/Reduce Locations里新建


然后点完成，就能看到ecplise连接hadoop了

如果看不到点击localhadoop右键reconnection 重新连
8.导入包
eclipse里面 windows-》preferences->Java ->bulid path-》user libraries

然后 ecplise 创建一个工程: File -> new -> project - >java -> java project


打jar包的时候不用把hadoopLib jar打进去只用打程序
点到工程，把JUnit4包引进去。然后创建一个conf文件夹在工程里，在conf目录下再创建一个HA目录

把hadoop集群中的core-site.xml 和 hdfs-site.xml 添加到HA目录中
点中HA文件夹

测试代码:
package com.test.hadoop.hdfs;

import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.BlockLocation;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class TestHDFS {

Configuration conf;
FileSystem fs;

@Before
public void conn() throws IOException {
    conf = new Configuration(true);
    fs = FileSystem.get(conf);
}

@After
public void close() throws IOException {
    fs.close();
}

//创建目录
@Test
public void mkDir() throws IOException {
    Path ifile = new Path("/ecpliseMkdir");
    if(fs.exists(ifile)) {
        fs.delete(ifile, true);
    }
    fs.mkdirs(ifile);
}

//上传文件
@Test
public void upload() throws IOException {
    Path ifile = new Path("/ecpliseMkdir/hello.txt");
    FSDataOutputStream output = fs.create(ifile);
    InputStream input = new BufferedInputStream(new FileInputStream(new File("d:\\ywcj_chnl_risk_map_estimate_model.sql")));
    IOUtils.copyBytes(input, output, conf, true);
}

//下载
@Test
public void downLocal() throws IOException {
    Path ifile = new Path("/ecpliseMkdir/hello.txt");
    FSDataInputStream open = fs.open(ifile);
    File newFile = new File("d:\\test.txt");
    if(!newFile.exists()) {
        newFile.createNewFile();
    }
    BufferedOutputStream output = new BufferedOutputStream(new FileOutputStream(newFile));
    IOUtils.copyBytes(open, output, conf, true);

}

//获取block 块信息
@Test
public void blockInfo() throws IOException {
    Path ifile = new Path("/ecpliseMkdir/hello.txt");
    FileStatus fsu = fs.getFileStatus(ifile);
    BlockLocation[] fileBlockLocations = fs.getFileBlockLocations(ifile, 0, fsu.getLen());
    for(BlockLocation b : fileBlockLocations) {
        System.out.println(b);
    }
}

//删除文件
@Test
public void deleteFile() throws IOException {
    Path ifile = new Path("/ecpliseMkdir/hello.txt");
    boolean delete = fs.delete(ifile, true);
    if(delete) {
        System.out.println("删除成功---------");
    }
}

}

Eclipse集成hadoop插件开发环境

原文：https://blog.51cto.com/14159501/2451626