oracle 19c rac install 之grid install

一 前置检查：
安装最低要求：
Red Hat Enterprise Linux 7.4: 3.10.0-693.5.2.0.1.el7.x86_64 or later
At least 8 GB RAM for Oracle Grid Infrastructure installations. （其实4G也可以安装完成）
SWAP：4G
安装目录：grid 10G,介质安装完成6.4G
参考:grid install guide
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwlin/server-hardware-checklist-for-oracle-grid-infrastructure.html#GUID-D964EF96-9C9C-4173-A919-F3CAA72180B1
这里环境概述：
IP：和11G准备一样，5个公网，2个私网；
共享存储：5GB；
操作系统：RHEL7.6 x64
介质准备：V981624-01.zip，V981627-01.zip，V981623-01.zip三个压缩文件，比18C小一些；只有for exadata版 本，在edelivery.oracle.com下载，公版没出来，不过装在linux应当是没有问题；介质变化：
1 软件解压目录即安装目录
2 自带JDK 1.8/TFA
3 logfile默认由50调到了200M

二 网络配置
OS采用最小安装，安装过程略，安装完成后先启用临时网络：

nmcli con modify "System eth0" ipv4.addresses 192.168.56.102/24 
nmcli con up "System eth0" 
ip addr show

设置主机名：
hostnamectl set-hostname rhel6
由于rhel7的网卡名称命名变化,如需显示eth0,eth2,需在RHEL 7修改网卡参数：
1、修改/etc/sysconfig/grub或者/etc/default/grub，它们是一个文件，改谁都可以，需要添加参数，给GRUB_CMDLINE_LINUX参数中增加
net.ifnames=0 biosdevname=0
然后：
grub2-mkconfig -o /boot/grub2/grub.cfg
2、配置网卡

mv /etc/sysconfig/network-scripts/ifcfg-enp0s3 /etc/sysconfig/network-scripts/ifcfg-eth0
vi /etc/sysconfig/network-scripts/ifcfg-eth0
修改：
device=eth0
name=eth0

*3、非必须，更改/etc/udev/ruled.d/90-eno-fix.rules，将其中的NAME="enp0s3"改为eth0
最后重启主机生效：
#禁用dhcp 169.254路由表

echo "NOZEROCONF=yes">>/etc/sysconfig/network
/etc/init.d/network restart

#RHEL 7禁用防火墙

 systemctl stop firewalld
systemctl mask firewalld

三 配置共享磁盘属性

vi /etc/rc.d/rc.local 
/usr/bin/raw /dev/raw/raw1 /dev/sdb
/usr/bin/chown grid:asmadmin /dev/raw/raw1
/usr/bin/chmod 660 /dev/raw/raw1
chmod +x /etc/rc.d/rc.local

*四 建立用户
如果先安装oracle的yum包，会自动建立

$ id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba), 
54323(oper),54324(backupdba),54325(dgdba),54326(kmdba),54327(asmdba),54330(racdba)
$ id grid
uid=54331(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),
54327(asmdba),54328(asmoper),54329(asmadmin),54330(racdba)

这种用户

groupadd -g 501 oinstall
groupadd -g 502 dba
groupadd -g 504 asmadmin
groupadd -g 506 asmdba
groupadd -g 507 asmoper
useradd -u 501 -g oinstall -G asmadmin,asmdba,asmoper grid
useradd -u 502 -g oinstall -G dba,asmdba oracle
echo 'oracle123' |passwd oracle --stdin;
echo 'oracle123' |passwd grid --stdin;

--配置ssh信任关系
./oui/prov/resources/scripts/sshUserSetup.sh -user grid -hosts "rhel6 rhel7" -noPromptPassphrase
或者
ssh-keygen生成用户密钥,配置互信；

五 配置yum 软件包
cd /etc/yum.repos.d/
mkdir bak
mv Centos ./bak
直接：
wget http://yum.oracle.com/public-yum-ol7.repo
修改其中的ol7_latest
或者新建一个repo文件：

[ol7_latest]
name=Oracle Linux $releasever Latest ($basearch)
baseurl=https://yum.oracle.com/repo/OracleLinux/OL7/latest/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=0
enabled=1

由于最小安装，需补充包:

yum install oracle-database-preinstall-18c
yum -y install gcc*
yun -y install xhost+

当然也可以用传统的挂载镜像来做yum源

mount -o loop /tmp/rhel-server-6.6-x86_64-dvd.iso /media
vi /etc/yum.respos.d/dvd.repo
[c6-media]
name=CentOS-$releasever - Media
baseurl=file:///media
gpgcheck=0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo
rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' bc \
binutils \
compat-libcap1 \
compat-libstdc++ \
dtrace-modules \
dtrace-modules-headers \
dtrace-modules-provider-headers \
dtrace-utils \
elfutils-libelf \
elfutils-libelf-devel \
fontconfig-devel \
glibc \
glibc-devel \
ksh \
libaio \
libaio-devel \
libdtrace-ctf-devel \
libX11 \
libXau \
libXi \
libXtst \
libXrender \
libXrender-devel \
libgcc \
librdmacm-devel \
libstdc++ \
libstdc++-devel \
libxcb 

rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel make sysstat unixODBC unixODBC-devel|grep -i "not installed"|awk '{print $2}'|xargs yum install -y

如果为RHEL6得升级glibc,libstdc++库到2.14/4.5以下版本；
检查版本:```
ldd --version
[grid@rhel6 grid]$ strings /usr/lib64/libstdc++.so.6|grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_FORCE_NEW
GLIBCXX_DEBUG_MESSAGE_LENGTH

**六 配置hosts**

vi /etc/hosts
127.0.0.1 localhost
#eth0 - PUBLIC
192.168.56.101 rhel7
192.168.56.102 rhel6
#VIP
192.168.56.103 rhel7-vip
192.168.56.104 rhel6-vip
#eth2 - PRIVATE
10.0.2.15 rhel7-pvt
10.0.2.16 rhel6-pvt
192.168.56.105 rac-scan

**七 配置 参数**

RHEL7 yum install后可以忽略

vi /etc/sysctl.conf
kernel.shmall = 4294967296
kernel.shmmni =4096
kernel.sem = 250 32000 100 128
fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65000
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
kernel.panic_on_oops=1

#sysctl -p

#调整shm:

tmpfs /dev/shm tmpfs defaults,size=3000M 0 0
mount -o remount /dev/shm

修改limit参数：
/etc/security/limits.conf

grid soft nproc 4096
grid hard nproc 16384
grid soft nofile 4096
grid hard nofile 65536
oracle soft nproc 4096
oracle hard nproc 16384
oracle soft nofile 4096
oracle hard nofile 65536

RHEL6注意可能不生效的情况需修改：
`/etc/security/limits.d/90-nproc.conf`
来源详细参见：
http://blog.yufeng.info/archives/2568

vi /etc/pam.d/login
session required pam_limits.so

vi /etc/profile
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi

#关闭selinux
setenforce 0
vi /etc/sysconfig/selinux，修改target参数为disabled

**8 配置环境变量**

export ORACLE_HOME=/data/grid
export ORACLE_BASE=/u01/grid
export ORACLE_SID=+ASM2
export DISPLAY=192.168.56.1:0.0

PATH=$PATH:$HOME/.local/bin:$HOME/bin:$ORACLE_HOME/bin

**9 安装**
执行安装前检查
` ./runcluvfy.sh stage -pre crsinst -n rhel6,rhel7 -verbose`

>  Verifying DNS/NIS name service ...FAILED
> PRVG-2058 : The 'hosts' entries in the existing "/etc/nsswitch.conf" files are
> inconsistent.
> "hosts" entry was found as "hosts: files dns myhostname" on nodes: rhel7
> "hosts" entry was found as "hosts: files dns" on nodes: rhel6
>  
> Verifying Daemon "avahi-daemon" not configured and running ...FAILED
> Verifying RPM Package Manager database ...INFORMATION
> PRVG-11250 : The check "RPM Package Manager database" was not performed because it needs 'root' user privileges.

一个是DNS解析文件的顺序不一致，一个是avahi-daemon没配置。
查看avahi解释：
Avahi允许程序在不需要进行手动网络配置的情况 下，在一个本地网络中发布和获知各种服务和主机。例如，当某用户把他的计算机接入到某个局域网时，如果他的机器运行有Avahi服务，则Avahi程式自 动广播，从而发现网络中可用的打印机、共享文件和可相互聊天的其他用户，这有点象他正在接收局域网中的各种网络广告一样。
如此看来，可以先忽略；
 运行：  
`  ./gridSetup.sh `
注意：先择 网络时选择asm&private,ausize默认为4MB了,其他安装和以前基本相同；
查看安装日志:
` tail -30f /data/grid/install/root_rhel7_2019-02-23_13-05-05-377719388.log`
 查看ohasd日志:

tail -30f $ORACLE_BASE/diag/crs/rhel7/crs/trace/ohasd.trc
tail -30f $ORACLE_BASE/diag/crs/rhel7/crs/trace/ohasd_orarootagent_root.trc

**10 查看资源状态**
> [grid@rhel6 bin]$ crsctl stat res -t
> 
> Name           Target  State        Server                   State details       
> 
> Local Resources
> 
> ora.LISTENER.lsnr
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> ora.chad
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> ora.net1.network
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> ora.ons
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> 
> Cluster Resources
> 
> ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    STABLE
>       2        ONLINE  ONLINE       rhel6                    STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.DATA.dg(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    STABLE
>       2        ONLINE  ONLINE       rhel6                    STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.LISTENER_SCAN1.lsnr
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.asm(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    Started,STABLE
>       2        ONLINE  ONLINE       rhel6                    Started,STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.asmnet1.asmnetwork(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    STABLE
>       2        ONLINE  ONLINE       rhel6                    STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.cvu
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.qosmserver
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.rhel6.vip
>       1        ONLINE  ONLINE       rhel6                    STABLE
> ora.rhel7.vip
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.scan1.vip
>       1        ONLINE  ONLINE       rhel7                    STABLE
> 
在安装过程中检查内存使用情况，其实4GB是可以 安装完成grid的。

[grid@rhel6 bin]$ free -m
total used free shared buff/cache available
Mem: 3789 1339 386 653 2063 1579
Swap: 2047 0 2047

检查OCR使用量：

ASMCMD> lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 512 4096 4194304 5120 4784 0 4784 0 Y DATA/