在VMware Workstation中一步一步搭建Oracle 21c rac集群环境

一、rac安装规划
- 1.1、所需软件介绍
- 1.2、IP地址规划
- 1.3、操作系统本地磁盘分区规划
- 1.4、共享存储与ASM磁盘组规划
二、操作系统配置
- 2.1、准备OS
- 2.2、添加网卡
- 2.3、修改主机名
- 2.4、配置静态IP地址
  - 2.4.1、修改mac地址
  - 2.4.2、配置静态IP地址
- 2.5、关闭防火墙
- 2.6、禁用selinux
- 2.7、修改/etc/hosts文件
- 2.8、添加组和用户
- 2.9、创建安装目录
  - 2.9.1、挂载/u01目录
  - 2.9.2、创建目录
- 2.10、配置grid和oracle用户的环境变量文件
- 2.11、配置root用户的环境变量
- 2.12、安装软件依赖包
- 2.13、配置内核参数
  - 2.13.1、修改/etc/sysctl.conf文件
  - 2.13.2、修改/etc/security/limits.conf文件
  - 2.13.3、修改/etc/pam.d/login文件
  - 2.13.4、修改/etc/profile文件
- 2.14、关闭NTP和chrony服务
- 2.15、关闭avahi-daemon
- 2.16、将NOZEROCONF=yes添加到/etc/sysconfig/network文件中
- 2.17、禁用透明大页
- 2.18、停止不用的服务
- 2.19、提升开关机速度
三、配置共享存储
- 3.1、服务端配置
  - 3.1.1、创建LVM
  - 3.1.2、使用yum安装targetd和targetcli
  - 3.1.3、使用targetcli创建设备
  - 3.1.4、使用targetcli创建iqn和LUN
  - 3.1.5、使用targetcli创建acls
  - 3.1.6、配置target监听IP和端口
  - 3.1.7、保存配置
- 3.2、客户端配置
  - 3.2.1、安装服务并启动
  - 3.2.2、使用iscsiadm发现可用存储设备并登陆连接
  - 3.2.3、创建并配置udev rules文件
四、数据库软件包准备
- 4.1、上传安装软件
- 4.2、解压软件
五、安装前预检查
- 5.1、安装补丁包(cvuqdisk)
- 5.2、配SSH互信，建立ssh等效性
- 5.3、cluster硬件检测--安装前预检查配置信息
六、图形界面安装集群和db
- 6.1、安装grid
- 6.2、安装db
- 6.3、创建磁盘组
- 6.4、创建数据库
七、静默安装集群和db
- 7.1、静默安装grid
- 7.2、静默安装db软件
- 7.3、创建DATA和FRA磁盘组
- 7.4、静默创建数据库
- 7.5、创建PDB数据库
八、修改基本配置
- 8.1、禁用crs和db的自启动
- 8.2、修改SQL提示符
参考

参考：https://www.xmmup.com/zaicentos7-8shanganzhuangoracle-jiqunrac-21cxiangxiwanzhengban.html

一、rac安装规划

官网安装过程：https://docs.oracle.com/en/database/oracle/oracle-database/21/rilin/index.html

1.1、所需软件介绍

Oracle RAC不支持异构平台。在同一个集群中，可以支持具有速度和规模不同的机器，但所有节点必须运行在相同的操作系统。Oracle RAC不支持具有不同的芯片架构的机器。

序号类型内容MD51数据库LINUX.X64_213000_db_home.zip8ac915a800800ddf16a382506d3953db2集群软件LINUX.X64_213000_grid_home.zipb3fbdb7621ad82cbd4f40943effdd1be3操作系统CentOS-7.8-x86_64-DVD-2003.iso16673979023254EA09CC0B57853A7BBD4虚拟机软件VMware Workstation Pro 16.0.0 build-168942995Xmanager Enterprise 4Xmanager Enterprise 4，用于打开图形界面6SecureCRTPortable.exeVersion 6.6.1 (build 289) ，带有SecureCRT和SecureFX，用于SSH连接和FTP上传

备注：

1、OS内存至少8G

2、安装OS和数据库前需要对下载好的安装包进行MD5值校验，检查其完整性

21c数据库软件下载可以参考：https://www.xmmup.com/dbbao76zaidockerzhongzhixu2bujikeyongyouoracle-21chuanjing.html

1.2、IP地址规划

从Oracle 11g开始，一共至少7个IP地址，2块网卡，其中public、vip和scan都在同一个网段，private在另一个网段，主机名不要包含下横线，如：RAC_01是不允许的；通过执行ifconfig -a检查2个节点的网卡名称必须一致。另外，在安装之前，公网、私网共4个IP可以ping通，其它3个不能ping通才是正常的。

从18c开始，scan建议至少3个。

节点/主机名IP AddressInterface Name地址类型注册位置虚拟网卡适配器操作系统网卡raclhr-21c-n1192.168.59.62raclhr-21c-n1Public/etc/hostsVMnet8(nat模式)ens33raclhr-21c-n1192.168.59.64raclhr-21c-n1-vipVirtual/etc/hostsVMnet8(nat模式)ens33raclhr-21c-n1192.168.2.62raclhr-21c-n1-privPrivate/etc/hostsVMnet2(仅主机模式)ens34raclhr-21c-n2192.168.59.63raclhr-21c-n2Public/etc/hostsVMnet8(nat模式)ens33raclhr-21c-n2192.168.59.65raclhr-21c-n2-vipVirtual/etc/hostsVMnet8(nat模式)ens33raclhr-21c-n2192.168.2.63raclhr-21c-n2-privPrivate/etc/hostsVMnet2(仅主机模式)ens34192.168.59.66192.168.59.67192.168.59.68raclhr-21c-scanSCAN/etc/hostsVMnet8(nat模式)ens33 1.3、操作系统本地磁盘分区规划

除了/boot分区外，其它分区均采用逻辑卷的方式，这样有利于文件系统的扩展。

序号分区名称大小逻辑卷用途说明1/u0150G/dev/mapper/vg_orasoft-lv_orasoft_u01oracle和grid的安装目录2/soft20G/dev/mapper/VG_OS-lv_VG_OS_soft存储软件，至少10G 1.4、共享存储与ASM磁盘组规划序号ASM磁盘名称磁盘组名称冗余方式大小用途备注1/dev/asm-diskd/dev/asm-diske/dev/asm-diskfOCRNormal3GOCR+VOTINGDISK最少1G2/dev/asm-diskg/dev/asm-diskh/dev/asm-diskiMGMTExternal30GMGMT最少30G，可以选择不安装GIMR组件3/dev/asm-diskj/dev/asm-diskk/dev/asm-dislDATAExternal45G存储数据库数据文件4/dev/asm-diskm/dev/asm-diskn/dev/asm-diskoFRAExternal30G快速恢复区二、操作系统配置

若无特殊说明，那么以下操作在2个节点均需要执行。

2.1、准备OS

安装步骤略，也可以下载小麦苗已经安装好的虚拟机环境：https://www.xmmup.com/ziliaofenxiang.html

安装一台虚拟机，然后复制改名，如下：

用虚拟机软件打开：

2.2、添加网卡

首先，需要添加网卡适配器：

VMnet2如下：

VMnet8如下：

网关为：192.168.59.2，如下所示：

先删除再添加，添加2块网卡，VMnet8为公有网卡，VMnet2为私有网卡，如下所示：

打开后，选择我已复制该虚拟机。

2.3、修改主机名

修改2个节点的主机名为raclhr-21c-n1和raclhr-21c-n2：

hostnamectl set-hostname raclhr-21c-n1
hostnamectl set-hostname raclhr-21c-n2

2.4、配置静态IP地址

在2个节点上分别配置静态IP地址。

在节点2上配置IP的时候注意将IP地址(IPADDR)修改掉。需要确保2个节点上的网卡MAC地址不一样，否则节点间不能通信。

2.4.1、修改mac地址

使用ifconfig或ip link show查看mac地址，修改文件/etc/udev/rules.d/70-persistent-ipoib.rules

cat > /etc/udev/rules.d/70-persistent-ipoib.rules  /home/oracle/.bash_profile  /home/grid/.bash_profile  /etc/profile  /etc/sysctl.conf  /etc/security/limits.conf

2.13.3、修改/etc/pam.d/login文件

echo "session    required     pam_limits.so" >> /etc/pam.d/login

2.13.4、修改/etc/profile文件

cat >> /etc/profile > /etc/sysconfig/network

2.17、禁用透明大页

参考：https://www.xmmup.com/linux-biaozhundayehetoumingdaye.html

cat >> /etc/rc.local  /sys/kernel/mm/transparent_hugepage/defrag
fi
 
EOF

chmod +x /etc/rc.d/rc.local
sh /etc/rc.local

cat /sys/kernel/mm/transparent_hugepage/defrag
cat /sys/kernel/mm/transparent_hugepage/enabled

建议配置大页功能，参考：https://www.xmmup.com/oracleshujukupeizhidaye.html

2.18、停止不用的服务

systemctl list-unit-files | grep enable

systemctl stop autofs
systemctl stop nfslock
systemctl stop rpcidmapd
systemctl stop rpcgssd
systemctl stop ntpd
systemctl stop bluetooth.service
systemctl stop cups.path 
systemctl stop cups.socket
systemctl stop postfix.service
systemctl stop rpcbind.service
systemctl stop rpcbind.socket
systemctl stop NetworkManager-dispatcher.service
systemctl stop dbus-org.freedesktop.NetworkManager.service
systemctl stop abrt-ccpp.service
systemctl stop cups.service
systemctl stop libvirtd

systemctl disable autofs
systemctl disable nfslock
systemctl disable rpcidmapd
systemctl disable rpcgssd
systemctl disable ntpd
systemctl disable bluetooth.service
systemctl disable cups.path 
systemctl disable cups.socket
systemctl disable postfix.service
systemctl disable rpcbind.service
systemctl disable rpcbind.socket
systemctl disable NetworkManager-dispatcher.service
systemctl disable dbus-org.freedesktop.NetworkManager.service
systemctl disable abrt-ccpp.service
systemctl disable cups.service
systemctl disable  libvirtd

2.19、提升开关机速度

在centos在关机或开机时一直等待，等待时间久，卡住时间久，提示A stop job is running for ……

解决：

cat >> /etc/systemd/system.conf > /etc/systemd/logind.conf
systemctl daemon-reload
systemctl restart systemd-logind

报错三：

Network Time Protocol (NTP) ...FAILED
raclhr-21c-n2: PRVG-1017 : NTP configuration file "/etc/ntp.conf" is present on
               nodes "raclhr-21c-n2,raclhr-21c-n1" on which NTP daemon or
               service was not running

raclhr-21c-n1: PRVG-1017 : NTP configuration file "/etc/ntp.conf" is present on
               nodes "raclhr-21c-n2,raclhr-21c-n1" on which NTP daemon or
               service was not running

我们使用ctssd来同步集群的时间，所以ntp必须关闭。

六、图形界面安装集群和db

安装之前重启一次OS，并检查网络和共享盘是否正确。

首先，打开Xmanager - Passive，如下：

6.1、安装grid

注意：以grid用户登录，然后运行下面这个脚本，和之前版本的grid安装有所不同：

[grid@raclhr-21c-n1 ~]$ export DISPLAY=192.168.59.1:0.0
[grid@raclhr-21c-n1 ~]$ /u01/app/21.3.0/grid/gridSetup.sh
Launching Oracle Grid Infrastructure Setup Wizard...

点击add添加节点2后，点击next

默认选中了不安装GIMR，我们也暂时不安装MIMR库：

修改磁盘路径为“/dev/asm*”就可以找到ASM磁盘了。

忽悠所有错误继续安装

分别在节点1和节点2上执行：

/u01/app/oraInventory/orainstRoot.sh
/u01/app/21.3.0/grid/root.sh

节点1运行：

[root@raclhr-21c-n1 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@raclhr-21c-n1 ~]# /u01/app/21.3.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/21.3.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/21.3.0/grid/crs/install/crsconfig_params
2021-08-20 08:38:46: Got permissions of file /u01/app/grid/crsdata/raclhr-21c-n1/crsconfig: 0775
2021-08-20 08:38:46: Got permissions of file /u01/app/grid/crsdata: 0775
2021-08-20 08:38:46: Got permissions of file /u01/app/grid/crsdata/raclhr-21c-n1: 0775
The log of current session can be found at:
  /u01/app/grid/crsdata/raclhr-21c-n1/crsconfig/rootcrs_raclhr-21c-n1_2021-08-20_08-38-46AM.log
2021/08/20 08:38:56 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2021/08/20 08:38:56 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2021/08/20 08:38:56 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2021/08/20 08:38:58 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2021/08/20 08:39:00 CLSRSC-594: Executing installation step 5 of 19: 'SetupOSD'.
Redirecting to /bin/systemctl restart rsyslog.service
2021/08/20 08:39:00 CLSRSC-594: Executing installation step 6 of 19: 'CheckCRSConfig'.
2021/08/20 08:39:00 CLSRSC-594: Executing installation step 7 of 19: 'SetupLocalGPNP'.
2021/08/20 08:39:13 CLSRSC-594: Executing installation step 8 of 19: 'CreateRootCert'.
2021/08/20 08:39:17 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2021/08/20 08:39:35 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2021/08/20 08:39:35 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2021/08/20 08:39:40 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2021/08/20 08:39:41 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2021/08/20 08:39:58 CLSRSC-4002: Successfully installed Oracle Autonomous Health Framework (AHF).
2021/08/20 08:40:14 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2021/08/20 08:40:14 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2021/08/20 08:40:19 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2021/08/20 08:40:24 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
2021/08/20 08:41:52 CLSRSC-482: Running command: '/u01/app/21.3.0/grid/bin/ocrconfig -upgrade grid oinstall'
CRS-4256: Updating the profile
Successful addition of voting disk e4f4713a678d4f04bf0cce6c0c14692d.
Successful addition of voting disk 7f85f487c7e34f42bff66f6c7cfc6032.
Successful addition of voting disk 408b301b47c84f97bfc109558258b289.
Successfully replaced voting disk group with +OCR.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   e4f4713a678d4f04bf0cce6c0c14692d (/dev/asm-diskd) [OCR]
 2. ONLINE   7f85f487c7e34f42bff66f6c7cfc6032 (/dev/asm-diske) [OCR]
 3. ONLINE   408b301b47c84f97bfc109558258b289 (/dev/asm-diskf) [OCR]
Located 3 voting disk(s).
2021/08/20 08:43:30 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2021/08/20 08:44:43 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/08/20 08:44:43 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2021/08/20 08:47:14 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2021/08/20 08:47:36 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

最后输出Configure Oracle Grid Infrastructure for a Cluster ... succeeded表示成功，跑完后，检查一下：

[root@raclhr-21c-n1 ~]# crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.chad
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.net1.network
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.ons
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        OFFLINE OFFLINE                               STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.OCR.dg(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        OFFLINE OFFLINE                               STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        OFFLINE OFFLINE                               STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        OFFLINE OFFLINE                               STABLE
ora.cdp1.cdp
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.cdp2.cdp
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.cdp3.cdp
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.cvu
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.qosmserver
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.raclhr-21c-n1.vip
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
--------------------------------------------------------------------------------

节点2：

[root@raclhr-21c-n2 ~]# /u01/app/21.3.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/21.3.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/21.3.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/grid/crsdata/raclhr-21c-n2/crsconfig/rootcrs_raclhr-21c-n2_2021-08-20_08-51-42AM.log
2021/08/20 08:51:48 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2021/08/20 08:51:48 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2021/08/20 08:51:48 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2021/08/20 08:51:50 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2021/08/20 08:51:50 CLSRSC-594: Executing installation step 5 of 19: 'SetupOSD'.
2021/08/20 08:51:50 CLSRSC-594: Executing installation step 6 of 19: 'CheckCRSConfig'.
2021/08/20 08:51:51 CLSRSC-594: Executing installation step 7 of 19: 'SetupLocalGPNP'.
2021/08/20 08:51:52 CLSRSC-594: Executing installation step 8 of 19: 'CreateRootCert'.
2021/08/20 08:51:52 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2021/08/20 08:51:52 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2021/08/20 08:52:23 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2021/08/20 08:52:24 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2021/08/20 08:52:25 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2021/08/20 08:52:55 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2021/08/20 08:52:55 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2021/08/20 08:52:57 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2021/08/20 08:52:58 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
2021/08/20 08:53:04 CLSRSC-4002: Successfully installed Oracle Autonomous Health Framework (AHF).
2021/08/20 08:53:08 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2021/08/20 08:54:09 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/08/20 08:54:09 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2021/08/20 08:54:28 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2021/08/20 08:54:38 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

最后输出Configure Oracle Grid Infrastructure for a Cluster ... succeeded表示成功，查看：

[root@raclhr-21c-n1 ~]# crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
               ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.chad
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
               ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.net1.network
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
               ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.ons
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
               ONLINE  ONLINE       raclhr-21c-n2            STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.OCR.dg(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.cdp1.cdp
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.cdp2.cdp
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.cdp3.cdp
      1        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.cvu
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.qosmserver
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.raclhr-21c-n1.vip
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.raclhr-21c-n2.vip
      1        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       raclhr-21c-n2            STABLE
--------------------------------------------------------------------------------

此时回到界面点击OK，继续运行：

最后集群校验报错，看了一下是dns和ntp的问题，可以忽略，继续安装：

6.2、安装db

[root@raclhr-21c-n1 ~]# su - oracle
Last login: Thu Aug 19 15:26:38 CST 2021 on pts/0
[oracle@raclhr-21c-n1 ~]$ cd $ORACLE_HOME
[oracle@raclhr-21c-n1 dbhome_1]$ export DISPLAY=192.168.59.1:0.0
[oracle@raclhr-21c-n1 dbhome_1]$ ./runInstaller

节点1和节点2分别执行：

[root@raclhr-21c-n1 ~]# /u01/app/oracle/product/21.3.0/dbhome_1/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/oracle/product/21.3.0/dbhome_1

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.

执行完后，点击OK：

6.3、创建磁盘组

以 grid 用户执行 asmca 命令，创建DATA和FRA两个磁盘组。：

[root@raclhr-21c-n1 ~]# su - grid
Last login: Fri Aug 20 09:59:53 CST 2021
[grid@raclhr-21c-n1 ~]$ export DISPLAY=192.168.59.1:0.0
[grid@raclhr-21c-n1 ~]$ asmca

查看：

[grid@raclhr-21c-n1 ~]$ $ORACLE_HOME/bin/kfod disks=all st=true ds=true
--------------------------------------------------------------------------------
 Disk          Size Header    Path                                    Disk Group    User     Group   
================================================================================
   1:       1024 MB MEMBER    /dev/asm-diskd                           OCR          grid     asmadmin
   2:       1024 MB MEMBER    /dev/asm-diske                           OCR          grid     asmadmin
   3:       1024 MB MEMBER    /dev/asm-diskf                           OCR          grid     asmadmin
   4:      10240 MB CANDIDATE /dev/asm-diskg                           #            grid     asmadmin
   5:      10240 MB CANDIDATE /dev/asm-diskh                           #            grid     asmadmin
   6:      10240 MB CANDIDATE /dev/asm-diski                           #            grid     asmadmin
   7:      15360 MB MEMBER    /dev/asm-diskj                           DATA         grid     asmadmin
   8:      15360 MB MEMBER    /dev/asm-diskk                           DATA         grid     asmadmin
   9:      15360 MB MEMBER    /dev/asm-diskl                           DATA         grid     asmadmin
  10:      10240 MB MEMBER    /dev/asm-diskm                           FRA          grid     asmadmin
  11:      10240 MB MEMBER    /dev/asm-diskn                           FRA          grid     asmadmin
  12:      10240 MB MEMBER    /dev/asm-disko                           FRA          grid     asmadmin
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME                                                          
================================================================================

6.4、创建数据库

[oracle@raclhr-21c-n1 ~]$ export DISPLAY=192.168.59.1:0.0
[oracle@raclhr-21c-n1 ~]$ dbca

七、静默安装集群和db

安装之前重启一次OS，并检查网络和共享盘是否正确。

参考：https://www.xmmup.com/zaicentos7-8shanganzhuangoracle-jiqunrac-21cxiangxiwanzhengban.html/7

7.1、静默安装grid

参考：https://www.xmmup.com/zaicentos7-8shanganzhuangoracle-jiqunrac-21cxiangxiwanzhengban.html/7

7.2、静默安装db软件

参考：https://www.xmmup.com/zaicentos7-8shanganzhuangoracle-jiqunrac-21cxiangxiwanzhengban.html/7

7.3、创建DATA和FRA磁盘组

参考：https://www.xmmup.com/zaicentos7-8shanganzhuangoracle-jiqunrac-21cxiangxiwanzhengban.html/7

7.4、静默创建数据库

-- rac数据库
dbca -silent -ignorePreReqs  -ignorePrereqFailure  -createDatabase -templateName General_Purpose.dbc -responseFile NO_VALUE \
-gdbname rac21c  -sid rac21c \
-createAsContainerDatabase TRUE \
-sysPassword lhr -systemPassword lhr -dbsnmpPassword lhr \
-datafileDestination '+DATA' -recoveryAreaDestination '+FRA' \
-storageType ASM \
-characterset AL32UTF8 \
-totalMemory 1024 \
-databaseType OLTP \
-emConfiguration none \
-nodeinfo raclhr-21c-n1,raclhr-21c-n2

-- 单实例
dbca -silent -createDatabase -templateName General_Purpose.dbc -responseFile NO_VALUE \
-gdbname LHRCDB  -sid LHRCDB \
-createAsContainerDatabase TRUE \
-numberOfPDBs 1 \
-pdbName pdbxmm \
-pdbAdminPassword lhr \
-sysPassword lhr -systemPassword lhr \
-datafileDestination '/u01/app/oracle/oradata' \
-recoveryAreaDestination '/u01/app/oracle/flash_recovery_area' \
-redoLogFileSize 50 \
-storageType FS \
-characterset AL32UTF8 -nationalCharacterSet AL16UTF16 \
-totalMemory 1024 \
-databaseType OLTP  \
-emConfiguration NONE

创建过程：

[oracle@raclhr-21c-n1 ~]$ dbca -silent -ignorePreReqs  -ignorePrereqFailure  -createDatabase -templateName General_Purpose.dbc -responseFile NO_VALUE \
> -gdbname rac21c  -sid rac21c \
> -createAsContainerDatabase TRUE \
> -sysPassword lhr -systemPassword lhr -dbsnmpPassword lhr \
> -datafileDestination '+DATA' -recoveryAreaDestination '+FRA' \
> -storageType ASM \
> -characterset AL32UTF8 \
> -totalMemory 1024 \
> -databaseType OLTP \
> -emConfiguration none \
> -nodeinfo raclhr-21c-n1,raclhr-21c-n2
[WARNING] [DBT-06208] The 'SYS' password entered does not conform to the Oracle recommended standards.
   CAUSE: 
a. Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9].
b.The password entered is a keyword that Oracle does not recommend to be used as password
   ACTION: Specify a strong password. If required refer Oracle documentation for guidelines.
[WARNING] [DBT-06208] The 'SYSTEM' password entered does not conform to the Oracle recommended standards.
   CAUSE: 
a. Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9].
b.The password entered is a keyword that Oracle does not recommend to be used as password
   ACTION: Specify a strong password. If required refer Oracle documentation for guidelines.
[WARNING] [DBT-06208] The 'PDBADMIN' password entered does not conform to the Oracle recommended standards.
   CAUSE: 
a. Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9].
b.The password entered is a keyword that Oracle does not recommend to be used as password
   ACTION: Specify a strong password. If required refer Oracle documentation for guidelines.
Prepare for db operation
7% complete
Copying database files
27% complete
Creating and starting Oracle instance
28% complete
31% complete
35% complete
37% complete
40% complete
Creating cluster database views
41% complete
53% complete
Completing Database Creation
57% complete
59% complete
60% complete
80% complete
Executing Post Configuration Actions
100% complete
Database creation complete. For details check the logfiles at:
 /u01/app/oracle/cfgtoollogs/dbca/rac21c.
Database Information:
Global Database Name:rac21c
System Identifier(SID) Prefix:rac21c
Look at the log file "/u01/app/oracle/cfgtoollogs/dbca/rac21c/rac21c.log" for further details.

[oracle@raclhr-21c-n1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
               ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.chad
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
               ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.net1.network
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
               ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.ons
               ONLINE  ONLINE       raclhr-21c-n1            STABLE
               ONLINE  ONLINE       raclhr-21c-n2            STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.DATA.dg(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.FRA.dg(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.OCR.dg(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            Started,STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.cdp1.cdp
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.cdp2.cdp
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.cdp3.cdp
      1        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.cvu
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.qosmserver
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.rac21c.db
      1        ONLINE  ONLINE       raclhr-21c-n1            Open,HOME=/u01/app/o
                                                             racle/product/21.3.0
                                                             /dbhome_1,STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            Open,HOME=/u01/app/o
                                                             racle/product/21.3.0
                                                             /dbhome_1,STABLE
ora.raclhr-21c-n1.vip
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.raclhr-21c-n2.vip
      1        ONLINE  ONLINE       raclhr-21c-n2            STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       raclhr-21c-n2            STABLE
--------------------------------------------------------------------------------

[oracle@raclhr-21c-n1 ~]$ sas

SQL*Plus: Release 21.0.0.0.0 - Production on Mon Aug 23 10:32:50 2021
Version 21.3.0.0.0

Copyright (c) 1982, 2021, Oracle.  All rights reserved.


Connected to:
Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
Version 21.3.0.0.0

SYS@rac21c1> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO

7.5、创建PDB数据库

SYS@rac21c1> show parameter db_create_file_dest

NAME                                 TYPE                   VALUE
------------------------------------ ---------------------- ------------------------------
db_create_file_dest                  string                 +DATA
SYS@rac21c1> create pluggable database PDBLHR1 admin user lhr identified by lhr;

Pluggable database created.

SYS@rac21c1> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDBLHR1                        MOUNTED
SYS@rac21c1> alter pluggable database pdblhr1 open;

Pluggable database altered.

SYS@rac21c1> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDBLHR1                        READ WRITE NO
SYS@rac21c1>  alter pluggable database all save state;

Pluggable database altered.

[oracle@raclhr-21c-n1 ~]$ export ORACLE_PDB_SID=PDBLHR1
[oracle@raclhr-21c-n1 ~]$ sas

SQL*Plus: Release 21.0.0.0.0 - Production on Mon Aug 23 10:45:11 2021
Version 21.3.0.0.0

Copyright (c) 1982, 2021, Oracle.  All rights reserved.


Connected to:
Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
Version 21.3.0.0.0

SYS@rac21c1> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         3 PDBLHR1                        READ WRITE NO
SYS@rac21c1> show con_name

CON_NAME
------------------------------
PDBLHR1


[root@raclhr-21c-n1 ~]# crsctl status resource  -w "TYPE = ora.database.type" -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.rac21c.db
      1        ONLINE  ONLINE       raclhr-21c-n1            Open,HOME=/u01/app/o
                                                             racle/product/21.3.0
                                                             /dbhome_1,STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            Open,HOME=/u01/app/o
                                                             racle/product/21.3.0
                                                             /dbhome_1,STABLE
--------------------------------------------------------------------------------
[root@raclhr-21c-n1 ~]# crsctl status resource  -w "TYPE = ora.pdb.type" -t        
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.rac21c.pdblhr1.pdb
      1        ONLINE  ONLINE       raclhr-21c-n1            STABLE
      2        ONLINE  ONLINE       raclhr-21c-n2            STABLE
--------------------------------------------------------------------------------

八、修改基本配置 8.1、禁用crs和db的自启动

-- 禁用crs自启动
crsctl disable has

-- 禁用db自启动
crsctl modify resource ora.rac21c.db -attr AUTO_START=never -unsupported
crsctl stat res ora.rac21c.db -p | grep AUTO_START

8.2、修改SQL提示符

oracle和grid均修改：

cat >> $ORACLE_HOME/sqlplus/admin/glogin.sql

在VMware Workstation中一步一步搭建Oracle 21c rac集群环境

[ 申请 ]友情链接：