Greenplum docker 安装测试环境

1. 下载docker 拉去docker 镜像，创建镜像
   yum -y install docker
   systemctl start dokcer
   docker pull centos：6
   docker run --privileged -dti -p 65000:6500 --name gptest1 centos:6 bash
   docker run --privileged -dti --name gptest2 centos:6 bash
   docker run --privileged -dti --name gptest3 centos:6 bash
   docker run --privileged -dti --name gptest4 centos:6 bash
   --privileges docker 有root权限

2、下载依赖包并启动ssh,并配置时间同步
docker中默认没有启动ssh，为了方便各节点之间的互连，启动docker的每个节点里面的ssh，并创建相关的认证key
yum install -y net-tools which openssh-clients openssh-server less zip
unzip iproute.x86_64 vim ntp ed
yum install update 更新系统内核
ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key
ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key
/usr/sbin/sshd
ntpdate ntp1.aliyun.com

3. 配置主机域名映射关系
   172.17.0.2 dw-greenplum-1 mdw
   172.17.0.3 dw-greenplum-2 sdw1
   172.17.0.4 dw-greenplum-3 sdw2
   172.17.0.4 dw-greenplum-4 sdw3
   同时，修改所有节点里面的 /etc/sysconfig/network 文件，保持主机名一致
   修改/etc/hostname 的名字
   [root@mdw /]# cat /etc/hostname
   mdw
   [root@mdw /]# cat /etc/sysconfig/network
   NETWORKING=yes
   HOSTNAME=mdw

4. 修改每个节点上面的文件打开数量的限制
   cat /etc/security/limits.conf
   • soft nofile 65536
   • hard nofile 65536
   • soft nproc 131072
   • hard nproc 131072
     [root@mdw /]# cat /etc/security/limits.d/90-nproc.conf
   • soft nproc 131072
     root soft nproc unlimited

5. 修改内核参数
   sysctl -w kernel.sem="500 64000 50 150"
   cat /proc/sys/kernel/sem

6. 关闭所有节点的防火墙和selinux
   service iptables stop
   chkconfig iptables off
   setenforce 0

7. 下载greenplum安装包
   需要注册账号。也可以在公司的机器上面拉取现有的二进制包
   https://network.pivotal.io/products/pivotal-gpdb

7.所有节点创建 greenplum 的用户和用户组
groupadd -g 530 gpadmin
useradd -g 530 -u 530 -m -d /home/gpadmin -s /bin/bash gpadmin
chown -R gpadmin:gpadmin /home/gpadmin
echo 123456 |passwd gpadmin --stdin

8. 在master上面安装greenplum
   su - gpadmin
   unzip greenplum-db-4.3.14.1-rhel5-x86_64.zip
   sh /greenplum-db-4.3.14.1-rhel5-x86_64.bin
   安装期间需要修改默认安装目录，输入/home/gpadmin/greenplum-db
   为了方便安装集群，greenplum提供了批量操作的指令，通过创建配置文件，以使用批处理命令
   创建文件
   [gpadmin@mdw ~]$ cat conf/hostlist
   mdw
   sdw1
   sdw2
   sdw3
   [gpadmin@mdw ~]$ cat conf/seg_hosts
   sdw1
   sdw2

9. 设置环境变量打通所有的节点
   source greenplum-db/greenplum_path.sh
   [gpadmin@mdw ~]$ gpssh-exkeys -f /home/gpadmin/conf/hostlist //传输秘钥
   [STEP 1 of 5] create local ID and authorize on local host
   ... /home/gpadmin/.ssh/id_rsa file exists ... key generation skipped

[STEP 2 of 5] keyscan all hosts and update known_hosts file

[STEP 3 of 5] authorize current user on remote hosts
... send to sdw1
... send to sdw2

[STEP 4 of 5] determine common authentication file content

[STEP 5 of 5] copy authentication files to all remote hosts
... finished key exchange with sdw1
... finished key exchange with sdw2
... finished key exchange with sdw3
[INFO] completed successfully
使用gpssh-exkeys命令时一定要使用gpadmin身份，因为会在/home/gpadmin/.ssh
中生成ssh的免密码登录秘钥，
[gpadmin@mdw ~]$ gpssh -f /home/gpadmin/conf/hostlist
Note: command history unsupported on this machine ...
=> pwd
[ mdw] /home/gpadmin
[sdw2] /home/gpadmin
[sdw1] /home/gpadmin
[sdw3] /home/gpadmin

10. 分发所有的安装包到每个值节点
    tar cf greenplum.tar greenplum-db/
    将文件拷贝到每个机器上面
    gpscp -f /home/gpadmin/conf/hostlist greenplum.tar =:/home/gpadmin/
    批量解压
    tar -xf greenplum.tar
    这样所有的节点就完成的安装

11. 初始化创建数据目录
    [gpadmin@mdw conf]$ gpssh -f hostlist
    => mkdir gpdata
    [ mdw]
    [sdw2]
    [sdw1]
    [sdw3]
    => cd gpdata
    [ mdw]
    [sdw2]
    [sdw1]
    [sdw3]
    => mkdir gpmaster gpdatap1 gpdatap2 gpdatam1 gpdatam2
    [ mdw]
    [sdw2]
    [sdw1]
    [sdw3]
    => exit

每个节点上面配置环境变量
cat .bash_profile
source /home/gpadmin/greenplum-db/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/home/gpadmin/gpdata/gpmaster/gpseg-1
export PGPORT=6500
export PGDATABASE=postgres
source .bash_profile

配置初始化文件
[gpadmin@mdw conf]$ cat gpinitsystem_config
ARRAY_NAME="Greenplum"
MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts

Segment 的名称前缀

SEG_PREFIX=gpseg

Primary Segment 起始的端口号

PORT_BASE=33000

指定 Primary Segment 的数据目录

declare -a DATA_DIRECTORY=(/home/gpadmin/gpdata/gpdatap1 /home/gpadmin/gpdata/gpdatap2)

Master 所在机器的 Hostname

MASTER_HOSTNAME=mdw

指定 Master 的数据目录

MASTER_DIRECTORY=/home/gpadmin/gpdata/gpmaster

Master 的端口

MASTER_PORT=6500

指定Bash的版本

TRUSTED_SHELL=ssh

Mirror Segment起始的端口号

CHECK_POINT_SEGMENTS=256
MIRROR_PORT_BASE=43000

Primary Segment 主备同步的起始端口号

REPLICATION_PORT_BASE=34000

Mirror Segment 主备同步的起始端口号

MIRROR_REPLICATION_PORT_BASE=44000

Mirror Segment 的数据目录

declare -a MIRROR_DATA_DIRECTORY=(/home/gpadmin/gpdata/gpdatam1 /home/gpadmin/gpdata/gpdatam2)

初始化数据库
gpinitsystem -c gpinitsystem_config -h hostlist -s sdw3 -S
//如无报错就是初始化完成，如果有报错，就需要根据日志提示修改配置，并重新初始化

[gpadmin@mdw conf]$ psql
psql (8.2.15)
Type "help" for help.

postgres=# select version();
version

PostgreSQL 8.2.15 (Greenplum Database 4.3.10.0 build commit: f413ff3b006655f14b6b9aa217495ec94da5c96c) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compi
led on Oct 21 2016 19:36:26
(1 row)

错误截图，日志和截图
节点有部分起来，一部分没有起来。主机资源不够。

2019-10-28 21:20:25.520556 CST,,,p6788,th-1469757664,,,,0,,,seg-1,,,,,"FATAL","XX000","could not create semaphores: No space left on device (pg_sema.c:132)","Failed system call was semget(43001004, 17, 03600).","This error does not mean that you have run out of disk space.
It occurs when either the system limit for the maximum number of semaphore sets (SEMMNI), or the system wide maximum number of semaphores (SEMMNS), would be exceeded. You need to raise the respective kernel parameter. Alternatively, reduce PostgreSQL's consumption of semaphores by reducing its max_connections parameter (currently 750).
The PostgreSQL documentation contains more information about configuring your system for PostgreSQL.",,,,,,"InternalIpcSemaphoreCreate","pg_sema.c",132,1 0xb0a80e postgres errstart + 0x4de

调整配置文件
vim /home/gpadmin/gpdata/gpdatam1/gpseg6/postgresql.conf
//每台机器4个segments 都要修改
max_connections = 200 //原值750 ，调整到200
shared_buffers = 500MB //原值275 调整到500
gpstart -a 重启启动集群
gpstop -u 重新加载配置文件
#常用操作命令
gpstate:查看数据库集群状态，示例:gpstate -a
gpstart:启动数据库集群,示例:gpstart -a
gpstop:关闭数据库集群,示例:gpstop -a -M fast
gpssh:远程执行shell命令，示例:gpssh -f hosts -e 'date'

部署参考博文
https://www.cnblogs.com/dap570/archive/2015/03/21/greenplum_4node_install.html
https://my.oschina.net/u/876354/blog/1606419