首页 > 其他 > 详细

sqoop使用

时间:2017-09-07 00:39:47      阅读:494      评论:0      收藏:0      [点我收藏+]

sqoop:000> show connector
+------------------------+---------+------------------------------------------------------------+----------------------+
| Name | Version | Class | Supported Directions |
+------------------------+---------+------------------------------------------------------------+----------------------+
| generic-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
| kite-connector | 1.99.7 | org.apache.sqoop.connector.kite.KiteConnector | FROM/TO |
| oracle-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO |
| ftp-connector | 1.99.7 | org.apache.sqoop.connector.ftp.FtpConnector | TO |
| hdfs-connector | 1.99.7 | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
| kafka-connector | 1.99.7 | org.apache.sqoop.connector.kafka.KafkaConnector | TO |
| sftp-connector | 1.99.7 | org.apache.sqoop.connector.sftp.SftpConnector | TO |
+------------------------+---------+------------------------------------------------------------+----------------------+
sqoop:000> create link -c generic-jdbc-connector
Creating link for connector with name generic-jdbc-connector
Please fill following values to create new link object
Name: mysql

Database connection

Driver class: com.mysql.jdbc.Driver
Connection String: jdbc:mysql://localhost:3306/lte
Username: root
Password: ****
Fetch Size: 100
Connection Properties:
There are currently 0 values in the map:
entry#

SQL Dialect

Identifier enclose:
Wed Sep 06 23:38:48 CST 2017 WARN: Establishing SSL connection without server‘s identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn‘t set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to ‘false‘. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
New link was successfully created with validation status OK and name mysql
sqoop:000>
sqoop:000>
sqoop:000> create link -c hdfs-connector
Creating link for connector with name hdfs-connector
Please fill following values to create new link object
Name: hdfs

HDFS cluster

URI: hdfs://localhost:9000
Conf directory: /Users/admaster/workspace/hadoop-2.8.1/etc/hadoop
Additional configs::
There are currently 0 values in the map:
entry#
New link was successfully created with validation status OK and name hdfs

 

 

sqoop:000> create job -f mysql -t hdfs
Creating job for links with from name mysql and to name hdfs
Please fill following values to create new job object
Name: city

Database source

Schema name: lte
Table name: city
SQL statement:
Column names:
There are currently 0 values in the list:
element#
Partition column: id
Partition column nullable:
Boundary query:

Incremental read

Check column:
Last value:

Target configuration

Override null value:
Null value:
File format:
0 : TEXT_FILE
1 : SEQUENCE_FILE
2 : PARQUET_FILE
Choose: 0
Compression codec:
0 : NONE
1 : DEFAULT
2 : DEFLATE
3 : GZIP
4 : BZIP2
5 : LZO
6 : LZ4
7 : SNAPPY
8 : CUSTOM
Choose:
Custom codec:
Output directory: /user/sqoop
Append mode:

Throttling resources

Extractors: 2
Loaders: 1

Classpath configuration

Extra mapper jars:
There are currently 0 values in the list:
element#
New job was successfully created with validation status OK and name city
sqoop:000>
sqoop:000>
sqoop:000> show job
+----+------+--------------------------------+-----------------------+---------+
| Id | Name | From Connector | To Connector | Enabled |
+----+------+--------------------------------+-----------------------+---------+
| 1 | city | mysql (generic-jdbc-connector) | hdfs (hdfs-connector) | true |
+----+------+--------------------------------+-----------------------+---------+
sqoop:000> start job -n city

 

sqoop使用

原文:http://www.cnblogs.com/loveItLoveFaimly/p/7487682.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!