首页 > 其他 > 详细

Greenplum+Hadoop学习笔记-13-客户端接口和程序

时间:2015-03-26 23:23:10      阅读:389      评论:0      收藏:0      [点我收藏+]

5.客户端接口和程序

5.1.配置客户端认证

5.1.1.允许连接到GPDB,在Master节点上编辑pg_hba.conf 文件

1)        依次查看Master节点和Slave节点中该文件:

[gpadmin@master gpseg-1]$ pwd

/data/master/gpseg-1

[gpadmin@master gpseg-1]$ cat pg_hba.conf

# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD

# "local" is for Unix domain socket connections only

# IPv4 local connections:

# IPv6 local connections:

local    all         gpadmin         ident

host     all         gpadmin         127.0.0.1/28    trust

host     all         gpadmin         192.168.80.200/24       trust

host     all         gpadmin         ::1/128       trust

host     all         gpadmin         fe80::20c:29ff:fe9d:3bd6/128       trust

[gpadmin@slave1 gpseg0]$ pwd

/data/primary/gpseg0

[gpadmin@slave1 gpseg0]$ cat pg_hba.conf

# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD

# "local" is for Unix domain socket connections only

local   all         all                               trust

# IPv4 local connections:

host    all         all         127.0.0.1/24          trust

# IPv6 local connections:

host    all         all         ::1/128               trust

host all all ::1/128 trust

host all all 192.168.80.200/32 trust

host all all fe80::20c:29ff:fe9d:3bd6/128 trust

host all gpadmin 192.168.80.201/32 trust

host all gpadmin ::1/128 trust

host all gpadmin fe80::250:56ff:fe24:a00/128 trust

2)        编辑文件,示例文件如下所示:

# allow the gpadmin user local access to all databases     允许gpadmin用户连接本机所有数据库

# using ident authentication                      

local all gpadmin ident sameuser                    

host all gpadmin 127.0.0.1/32 ident

host all gpadmin ::1/128 ident

host all dba 192.168.0.0/32 md5

host all all 192.168.0.0/32

ldap ldapserver=usldap1     

ldapport=1389 ldapprefix="cn="                   

ldapsuffix=",ou=People,dc=company,dc=com”

3)        保存并关闭文件

4)        重新加载pg_hba.conf使修改生效

-u (reload pg_hba.conf and postgresql.conf files only)

 

 This option reloads the pg_hba.conf files of the master and

 segments and the runtime parameters of the postgresql.conf files

 but does not shutdown the Greenplum Database array. Use this

 option to make new configuration settings active after editing

 postgresql.conf or pg_hba.conf. Note that this only applies to

 configuration parameters that are designated as runtime parameters.

[gpadmin@master gpseg-1]$ gpstop -u

20150324:22:44:04:008787 gpstop:master:gpadmin-[INFO]:-Starting gpstop with args: -u

20150324:22:44:04:008787 gpstop:master:gpadmin-[INFO]:-Gathering information and validating the environment...

20150324:22:44:04:008787 gpstop:master:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information

20150324:22:44:04:008787 gpstop:master:gpadmin-[INFO]:-Obtaining Segment details from master...

20150324:22:44:04:008787 gpstop:master:gpadmin-[INFO]:-Greenplum Version: ‘postgres (Greenplum Database) 4.2.8.0 build 1‘

20150324:22:44:04:008787 gpstop:master:gpadmin-[INFO]:-Signalling all postmaster processes to reload

 

5.1.2.限制并发连接

1)        配置Server参数max_connections来实现

2)        Segment的值不能小于Master,官方建议5~10倍,实际使用经验是2~3倍;因为在Segment需要执行多个进程,所以需要更多的连接;

3)        设置依赖的参数max_prepared_transactions,至少要和Master上的max_connections值一样大

4)        修改最大连接数

a)        停止数据库

b)        分别在Master和Segment上编辑postgresql.conf文件
修改max_connections和max_prepared_transactions参数

c)        重启数据库

Master节点核心信息:

port=5432

 

max_connections = 250           

 

shared_buffers = 125MB                  

max_prepared_transactions = 250         

 

 

max_fsm_pages = 200000               

 

 

gp_autostats_mode=on_no_stats          # none, on_no_stats, on_change. see documentation for semantics.

gp_autostats_on_change_threshold=2147483647 # [0..INT_MAX]. see documentation for semantics.

log_autostats=off    # print additional autostats information

 

 

 

datestyle = ‘iso, mdy‘                          # inserted by initdb

 

lc_messages = ‘en_US.utf8‘              # inserted by initdb

 

lc_monetary = ‘en_US.utf8‘              # inserted by initdb

 

lc_numeric = ‘en_US.utf8‘                 # inserted by initdb

 

lc_time = ‘en_US.utf8‘                        # inserted by initdb

 

gp_resqueue_memory_policy = ‘eager_free‘ # memory request based queueing.

                                                                                    # eager_free, auto or none

s

max_appendonly_tables = 10000 # Maximum number of append only tables that can

                                                                   # participate in writing data concurrently.

 

gp_interconnect_type=udpifc

 

# - Worker Process Creation -

 

gp_connections_per_thread = 64

gp_segment_connect_timeout = 600s

 

gp_vmem_protect_limit = 8192  #Virtual memory limit (in MB).

 

gp_backup_directIO = off                                     # enable\disable dump with direct IO

gp_backup_directIO_read_chunk_mb = 20    # Size of read Chunk buffer in directIO dump (in MB)

 

gp_crash_handler_async = on

#Greenplum specific configuration parameters for Master instance database

 

log_statement=all

checkpoint_segments=8

 

Segment节点核心信息:

 

# - Connection Settings -

 

listen_addresses=‘*‘ ##listen_addresses = ‘*‘                  # what IP address(es) to listen on;

                                               # comma-separated list of addresses;

                                               # defaults to ‘*‘, ‘*‘ = all

                                               # (change requires restart)

 

port=40000 ##port = 5432                                   # sets the database listener port for

                                                # a Greenplum instance. The master and

                                               # each segment has its own port number.

 

 

max_connections = 750                     # inserted by initdb默认是主节点的三倍

 

 

# - Memory -

 

shared_buffers = 125MB                            # inserted by initdb

#shared_buffers = 128MB                          # min 128kB or max_connections*16kB

                                               # (change requires restart)

#temp_buffers = 8MB                        # min 800kB

max_prepared_transactions = 250          # can be 0 or more

                                               # (change requires restart)

 

# - Free Space Map -

 

max_fsm_pages = 200000                         # inserted by initdb

 

 

 

gp_autostats_mode=on_no_stats          # none, on_no_stats, on_change. see documentation for semantics.

gp_autostats_on_change_threshold=2147483647 # [0..INT_MAX]. see documentation for semantics.

log_autostats=off    # print additional autostats information

 

0

 

# - Locale and Formatting -

 

datestyle = ‘iso, mdy‘                          # inserted by initdb

 

# These settings are initialized by initdb, but they can be changed.

lc_messages = ‘en_US.utf8‘              # inserted by initdb

 

lc_monetary = ‘en_US.utf8‘              # inserted by initdb

 

lc_numeric = ‘en_US.utf8‘                 # inserted by initdb

 

lc_time = ‘en_US.utf8‘                        # inserted by initdb

 

gp_resqueue_memory_policy = ‘eager_free‘ # memory request based queueing.

                                                                                    # eager_free, auto or none

 

max_appendonly_tables = 10000 # Maximum number of append only tables that can

                                                                   # participate in writing data concurrently.

 

 

#gp_max_packet_size = 8192

gp_interconnect_type=udpifc

 

# - Worker Process Creation -

 

gp_connections_per_thread = 64

gp_segment_connect_timeout = 600s

#gp_enable_delete_as_truncate = off

 

# - Resource limits -

gp_vmem_protect_limit = 8192  #Virtual memory limit (in MB).

 

#custom_variable_classes = ‘‘                            # list of custom variable class names

gp_backup_directIO = off                                     # enable\disable dump with direct IO

gp_backup_directIO_read_chunk_mb = 20    # Size of read Chunk buffer in directIO dump (in MB)

 

gp_crash_handler_async = on

#MPP Specific parameters

#----------------------

checkpoint_segments=8

 

 

配置过程中出现如下错误:

Command was: ‘env GPSESSID=0000000000 GPERA=None $GPHOME/bin/pg_ctl -D /data/master/gpseg-1 -l /data/master/gpseg-1/pg_log/startup.log -w -t 600 -o " -p 5432 -b 1 -z 0 --silent-mode=true -i -M master -C -1 -x 0 -c gp_role=utility " start‘

rc=1, stdout=‘waiting for server to start......could not start server

‘, stderr=‘pg_ctl: PID file "/data/master/gpseg-1/postmaster.pid" does not exist

解决方案:

 在GP_master上重启服务即可。

5.2.客户端接口和程序以及主要客户端工具

5.2.1.pgAdmin3 for GPDB

5.2.1.1.说明

图形化管理和SQL执行、分析与监控工具,下载地址:http://www.pgadmin.org/download/windows.php

5.2.1.2.安装步骤概要
5.2.1.2.1.点击“运行”

 技术分享

5.2.1.2.2.显示欢迎界面

 

技术分享

 

5.2.1.2.3.选择接收协议

技术分享

5.2.1.2.4.选择安装目录

技术分享技术分享

5.2.1.2.5.点击“Install”

 技术分享技术分享

5.2.1.2.6.点击“Finish”

 技术分享

5.2.1.2.7.运行后显示界面

技术分享

修改gpadmin用户密码为’gpadmin’

template1=# alter role gpadmin with password ‘gpadmin‘;

ALTER ROLE

技术分享

如上图所示,勾选保存密码选项,但不需要恢复环境,因为每次都恢复环境会对服务器产生相应的压力。

登陆成功后的界面如下所示:


技术分享

5.2.2.PSQL

5.2.2.1.说明

必须通过Master节点连接的行命令操作和管理工具

5.2.2.2.连接信息

[gpadmin@master ~]$ psql -h

psql: option requires an argument -- h

Try "psql --help" for more information.

[gpadmin@master ~]$ psql --help

This is psql 8.2.15, the PostgreSQL interactive terminal (Greenplum version).

 

Usage:

  psql [OPTION]... [DBNAME [USERNAME]]

 

General options:

  -c, --command=COMMAND    run only single command (SQL or internal) and exit

  -d, --dbname=DBNAME      database name to connect to (default: "gpadmin")  连接的数据库名称,首次连接时默认是template1

  -f, --file=FILENAME      execute commands from file, then exit

  -l, --list               list available databases, then exit

  -v, --set=, --variable=NAME=VALUE

                           set psql variable NAME to VALUE

  -X, --no-psqlrc          do not read startup file (~/.psqlrc)

  -1 ("one"), --single-transaction

                           execute command file as a single transaction

  --help                   show this help, then exit

  --version                output version information, then exit

 

Input and output options:

  -a, --echo-all           echo all input from script

  -e, --echo-queries       echo commands sent to server

  -E, --echo-hidden        display queries that internal commands generate

  -L, --log-file=FILENAME  send session log to file

  -n, --no-readline        disable enhanced command line editing (readline)

  -o, --output=FILENAME    send query results to file (or |pipe)

  -q, --quiet              run quietly (no messages, only query output)

  -s, --single-step        single-step mode (confirm each query)

  -S, --single-line        single-line mode (end of line terminates SQL command)

 

Output format options:

  -A, --no-align           unaligned table output mode

  -F, --field-separator=STRING

                           set field separator (default: "|")

  -H, --html               HTML table output mode

  -P, --pset=VAR[=ARG]     set printing option VAR to ARG (see \pset command)

  -R, --record-separator=STRING

                           set record separator (default: newline)

  -t, --tuples-only        print rows only

  -T, --table-attr=TEXT    set HTML table tag attributes (e.g., width, border)

  -x, --expanded           turn on expanded table output

 

Connection options:

  -h, --host=HOSTNAME      database server host or socket directory (default: "local socket")   Master主机名

  -p, --port=PORT          database server port (default: "5432")                           Master端口,默认是5432

  -U, --username=USERNAME  database user name (default: "gpadmin")                      登陆用户,默认是gpadmin,且gpadmin是超级用户

  -w, --no-password        never prompt for password         

  -W, --password           force password prompt (should happen automatically)

 

For more information, type "\?" (for internal commands) or "\help" (for SQL

commands) from within psql, or consult the psql section in the PostgreSQL

documentation.

 

Report bugs to <pgsql-bugs@postgresql.org>.

 

 

Greenplum+Hadoop学习笔记-13-客户端接口和程序

原文:http://blog.csdn.net/mavs41/article/details/44658487

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!