下载：http://mirrors.hust.edu.cn/apache/
安装： tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
配置：
① 修改 sqoop-env.sh mv sqoop-env-template.sh sqoop-env.sh
export HADOOP_COMMON_HOME=/home/hadoop/apps/hadoop-2.7.5 #Set path to where hadoop-*-core.jar is available export HADOOP_MAPRED_HOME=/home/hadoop/apps/hadoop-2.7.5 #set the path to where bin/hbase is available export HBASE_HOME=/home/hadoop/apps/hbase-1.2.6 #Set the path to where bin/hive is available export HIVE_HOME=/home/hadoop/apps/apache-hive-2.3.3-bin #Set the path for where zookeper config dir is export ZOOCFGDIR=/home/hadoop/apps/zookeeper-3.4.10/conf
②加入 mysql 驱动包到 sqoop1.4.6/lib 目录下
cp mysql-connector-java-5.1.41-bin.jar /root/sqoop-1.4.7/lib/
③配置系统环境变量 vim /etc/profile
export SQOOP_HOME=/root/sqoop-1.4.7
export PATH=$PATH:$SQOOP_HOME/bin
④使环境变量生效
⑤验证安装是否成功 sqoop-version
使用：
①列出有哪些库:
sqoop list-databases
--connect jdbc:mysql://localhost:3306/
--username root
--password root
②列出有哪些表:
sqoop list-tables
--connect jdbc:mysql://localhost:3306/
--username root
--password root
③Mysql导入HDFS:
sqoop import
--connect jdbc:mysql://localhost:3306/myhive
--username root
--password root
--table depart_name -m 1
④把MySQL数据库中的表数据导入到HBase:
sqoop import
--connect jdbc:mysql://localhost:3305/myhive
--username root
--password root
--table depart_name
--hbase-table depart_name
--column-family person
--hbase-row-key depart_id
⑤把HDFS导出到Mysql
sqoop export --connect jdbc:mysql://localhost:3360/myhive --username root --password root --table depart_name --export-dir /user/myhive
⑥Sqoop Job
创建增量导入：
sqoop job
--create myjob
--import
--connect jdbc:mysql://localhost:3306/myhive
--username root
--password root
--table depart_name
-m 1
--incremental append
--check-column id
--last_value 1
--verbose
查看：
sqoop job --list
sqoop job --show myjob #myjob任务名称
sqoop job --exec myjob #执行myjob
⑦Sqoop定时作业
Linux Crontab调度：
/root/sqoop/bin/sqoop job --exec myjob >myjob.log 2>&1 &
Sqoop 错误：
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils at org.apache.sqoop.manager.MySQLManager.initOptionDefaults(MySQLManager.java:73) at org.apache.sqoop.manager.SqlManager.<init>(SqlManager.java:89) at com.cloudera.sqoop.manager.SqlManager.<init>(SqlManager.java:33) at org.apache.sqoop.manager.GenericJdbcManager.<init>(GenericJdbcManager.java:51) at com.cloudera.sqoop.manager.GenericJdbcManager.<init>(GenericJdbcManager.java:30) at org.apache.sqoop.manager.CatalogQueryManager.<init>(CatalogQueryManager.java:46) at com.cloudera.sqoop.manager.CatalogQueryManager.<init>(CatalogQueryManager.java:31) at org.apache.sqoop.manager.InformationSchemaManager.<init>(InformationSchemaManager.java:38) at com.cloudera.sqoop.manager.InformationSchemaManager.<init>(InformationSchemaManager.java:31) at org.apache.sqoop.manager.MySQLManager.<init>(MySQLManager.java:65) at org.apache.sqoop.manager.DefaultManagerFactory.accept(DefaultManagerFactory.java:67) at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:184) at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:272) at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:44) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.StringUtils at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 20 more
在apache 上下载commons-lang-2.6.jar包并上传到lib下即可，下载地址：
http://mirrors.tuna.tsinghua.edu.cn/apache//commons/lang/binaries/commons-lang-2.6-bin.zip
原文：https://www.cnblogs.com/wwbz/p/12594861.html