Oracle Database Appliance (ODA) Bundle Patching

Oracle released Oracle Exadata, the first engineering database system, in 2008. in 2011, the company announced the Oracle Database Appliance as a smaller, less powerful alternative to Oracle Exadata at a lower price point. According to industry analysts, Oracle expected Oracle Database Appliance to fill the gap in its product line under Oracle Exadata by targeting midsize customers. The platform has proven very popular among database administrators.

Oracle Database Appliance supports more than just database systems. Starting with versions 2.5 (ODA V1) and 2.5.5 (ODA V2 X3-2) Oracle Database Appliance can be deployed using Oracle VM, which allows an administrator to install the application layer along with the database.

Oracle Database Appliance also offers a pay-as-you-grow model for Oracle licenses. This allows a customer to license only the number of CPUs they need, not the entire capacity of the device. When virtualized, this is supported for both the database and application layer.

When the Oracle Database Appliance is connected to a ZS3 storage array, the DBA can leverage Hybrid Columnar Compression for data stored in the array to enable tiered storage and compression rates in excess of 200%.

Oracle Cloud integration safeguard customer data, Oracle Cloud Infrastructure (OCI) integration enables IT staff to back up and archive data in the cloud, while easily moving workloads to the cloud when needed.

Anyway, back to the topic and let’s get started 🙂

We can in order to upgrade the ODA version, Dataguard was switched over first, and the Standby server was converted to Prod.

Let’s follow the steps below for the switchover steps.



-> primary

SELECT SWITCHOVER_STATUS FROM V$DATABASE;

ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY WITH SESSION SHUTDOWN;

STARTUP MOUNT;

-> standby

SELECT SWITCHOVER_STATUS FROM V$DATABASE;

ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY WITH SESSION SHUTDOWN;

ALTER DATABASE OPEN;

-> old primary

ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT FROM SESSION;



Patch Bundle 2.10.0.0.0 files downloaded from Mysupport ( p18284007_210000_Linux-x86-64_1of2.zip, p18284007_210000_Linux-x86-64_2of2.zip ) were moved to ODA via USB at node 1.

-> /usb mount point created.
-> On the second node, the ‘udb’ directory was created with mkdir and the zip files were moved here with scp.



[root@hogwarts1 usb]# scp p18284007_210000_Linux-x86-64_* root@hogwarts2:/usb

root@hogwarts2’s password:

p18284007_210000_Linux-x86-64_1of2.zip                 100% 2846MB  55.8MB/s   00:51

p18284007_210000_Linux-x86-64_2of2.zip                 100% 1738MB  52.7MB/s   00:33

[root@hogwarts1 usb]#

 –   You can check current version

[root@hogwarts ~]# /opt/oracle/oak/bin/oakcli show version

Version

——-

2.7.0.0.0

1. Step: Unpack Patch bundle zip documents on both nodes of ODA.
   We UNPACK both zip files on both nodes of ODA.



 [root@hogwarts ~]# /opt/oracle/oak/bin/oakcli unpack –package /usb/p18284007_210000_Linux-x86-64_1of2.zip

Unpacking takes a while,  pls wait….

WARNING: 2020-05-15 13:22:24: You are trying to unpack 2.10.0.0.0 Patchbundle

INFO: 2020-05-15 13:22:24: Current system is at 2.7.0.0.0

WARNING: 2020-05-15 13:22:24: Unpacking this bundle may overwrite some files which may cause patching/deployment failures

Press Yes to continue or No to quit…. :

yes

Successfully unpacked the files to repository.



[root@hogwarts ~]# /opt/oracle/oak/bin/oakcli unpack -package /usb/p18284007_210000_Linux-x86-64_2of2.zip

Unpacking takes a while,  pls wait….

WARNING: 2020-05-15 13:26:59: You are trying to unpack 2.10.0.0.0 Patchbundle

INFO: 2020-05-15 13:26:59: Current system is at 2.7.0.0.0

WARNING: 2020-05-15 13:26:59: Unpacking this bundle may overwrite some files which may cause patching/deployment failures

Press Yes to continue or No to quit…. :

yes

Successfully unpacked the files to repository.

-> We unpack on the second node with the same commands.

Note: Unpacked patch documents are created under /tmp.

2. Step:

As a result of switchover, since this is in Dataguard status, we first stop the dataguard.

[root@hogwarts ~]# su – oracle

[oracle@hogwarts ~]$ db

[oracle@hogwarts ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Sat Feb 7 13:38:50 2020

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 – 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Data Mining and Real Application Testing options

SQL> alter database recover managed standby database cancel;

Database altered.

3. Step: verify the Components to be Patched

[root@hogwarts ~]# /opt/oracle/oak/bin/oakcli update -patch 2.10.0.0.0 –verify

INFO: 2020-05-15 13:30:02: Reading metadata . It takes a while…

  Component Name            Installed Version         Proposed Patch Version

    —————                               ——————        —————–

              Controller_INT                            11.05.02.00               11.05.03.00

               Controller_EXT                          11.05.02.00               11.05.03.00

                Expander                                    0010                            0018

                SSD_SHARED                              9432                            9440

                HDD_LOCAL                               A31A                           A606

                HDD_SHARED {

                [ c2d2 ]                                         A600                      Up-to-date

                [ c2d15 ]                                       A690                      A600

                [ c2d0,c2d1,c2d3,c2d                 A31A                      A600

                4,c2d5,c2d6,c2d7,c2d

                8,c2d9,c2d10,c2d11,c

                2d12,c2d13,c2d14,c2d

                16,c2d17,c2d18,c2d19

                 ]

                             }

                ILOM                                   3.1.2.10.b r77700         3.1.2.10.d r83372

                BIOS                                    17030300                       17050100

                IPMI                                     1.8.10.5                         Up-to-date

                HMP                                      2.2.6.2                            2.2.6.5

                OAK                                    2.7.0.0.0                            2.10.0.0.0

                OEL                                              5.9                            Up-to-date

                GI_HOME                   11.2.0.3.7(16619892, 16742216)          11.2.0.4.2(18031740,

                DB_HOME                11.2.0.3.7(16619892, 16742216)     11.2.0.3.10(18031683,17592127)

4. Step : We Apply the Patch for the infra.

INFRA includes:  Disk firmware, Controller Firmware, Operating System, ILOM,BIOS, Hardware Management Pack, IPMI, OAK, ASR.For virtualized environment, it also includes the Dom0 updates.         

[root@hogwarts ~]# /opt/oracle/oak/bin/oakcli update -patch 2.10.0.0.0 –infra

INFO: DB, ASM, Clusterware may be stopped during the patch if required

INFO: Both nodes may get rebooted automatically during the patch if required

Do you want to continue: [Y/N]?: Y

INFO: User has confirmed the reboot

INFO: Patch bundle must be unpacked on the second node also before applying this patch

Did you unpack the patch bundle on the second node?: [Y/N]?: Y

Please enter the ‘root’ user password:

Please re-enter the ‘root’ user password:

INFO: Setting up the SSH

……….done

INFO: Running pre-install scripts

……….done

INFO: 2020-05-15 13:44:00: Running pre patch script for 2.10.0.0.0

INFO: 2020-05-15 13:44:14: Completed pre patch script for 2.10.0.0.0

INFO: 2020-05-15 13:44:30: ——————Patching HMP————————-

SUCCESS: 2020-05-15 13:45:06: Successfully upgraded the HMP

INFO: 2020-05-15 13:45:06: ———————-Patching OAK———————

SUCCESS: 2020-05-15 13:45:30: Successfully upgraded OAK

INFO: 2020-05-15 13:45:31: ———————-Patching JDK———————

SUCCESS: 2020-05-15 13:45:42: Successfully upgraded JDK

INFO: 2020-05-15 13:45:42: ——————Patching OS————————-

INFO: 2020-05-15 13:46:06: Clusterware is running on one or more nodes of the cluster

INFO: 2020-05-15 13:46:06: Attempting to stop clusterware and its resources across the cluster

SUCCESS: 2020-05-15 13:48:11: Successfully stopped the clusterware

SUCCESS: 2020-05-15 13:49:26: Successfully upgraded the OS

INFO: 2020-05-15 13:49:32: ———————-Patching IPMI———————

INFO: 2020-05-15 13:49:32: IPMI is already upgraded or running with latest version

INFO: 2020-05-15 13:49:39: —————-Patching the Storage——————-

INFO: 2020-05-15 13:49:39: ………………..Patching SSDs……………

INFO: 2020-05-15 13:49:39: Updating the  Disk : d20 with the firmware : Z16IZF4EUSUN200G 9440

SUCCESS: 2020-05-15 13:50:05: Successfully updated the firmware on  Disk : d20 to Z16IZF4EUSUN200G 9440

INFO: 2020-05-15 13:50:05: Updating the  Disk : d21 with the firmware : Z16IZF4EUSUN200G 9440

SUCCESS: 2020-05-15 13:50:32: Successfully updated the firmware on  Disk : d21 to Z16IZF4EUSUN200G 9440

INFO: 2020-05-15 13:50:32: Updating the  Disk : d22 with the firmware : Z16IZF4EUSUN200G 9440

SUCCESS: 2020-05-15 13:51:11: Successfully updated the firmware on  Disk : d22 to Z16IZF4EUSUN200G 9440

INFO: 2020-05-15 13:51:11: Updating the  Disk : d23 with the firmware : Z16IZF4EUSUN200G 9440

SUCCESS: 2020-05-15 13:51:49: Successfully updated the firmware on  Disk : d23 to Z16IZF4EUSUN200G 9440

INFO: 2020-05-15 13:51:49: ………………..Patching shared HDDs……………

INFO: 2020-05-15 13:51:49: Updating the  Disk : d0 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:52:20: Successfully updated the firmware on  Disk : d0 to H109090SESUN900G A600

INFO: 2020-05-15 13:52:20: Updating the  Disk : d1 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:52:50: Successfully updated the firmware on  Disk : d1 to H109090SESUN900G A600

INFO: 2020-05-15 13:52:50: Disk : d2  is already running with : H109090SESUN900G A600

INFO: 2020-05-15 13:52:50: Updating the  Disk : d3 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:53:20: Successfully updated the firmware on  Disk : d3 to H109090SESUN900G A600

INFO: 2020-05-15 13:53:20: Updating the  Disk : d4 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:53:48: Successfully updated the firmware on  Disk : d4 to H109090SESUN900G A600

INFO: 2020-05-15 13:53:48: Updating the  Disk : d5 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:54:16: Successfully updated the firmware on  Disk : d5 to H109090SESUN900G A600

INFO: 2020-05-15 13:54:17: Updating the  Disk : d6 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:54:49: Successfully updated the firmware on  Disk : d6 to H109090SESUN900G A600

INFO: 2020-05-15 13:54:49: Updating the  Disk : d7 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:55:20: Successfully updated the firmware on  Disk : d7 to H109090SESUN900G A600

INFO: 2020-05-15 13:55:20: Updating the  Disk : d8 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:55:51: Successfully updated the firmware on  Disk : d8 to H109090SESUN900G A600

INFO: 2020-05-15 13:55:51: Updating the  Disk : d9 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:56:38: Successfully updated the firmware on  Disk : d9 to H109090SESUN900G A600

INFO: 2020-05-15 13:56:38: Updating the  Disk : d10 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:57:20: Successfully updated the firmware on  Disk : d10 to H109090SESUN900G A600

INFO: 2020-05-15 13:57:20: Updating the  Disk : d11 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:57:55: Successfully updated the firmware on  Disk : d11 to H109090SESUN900G A600

INFO: 2020-05-15 13:57:55: Updating the  Disk : d12 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:58:29: Successfully updated the firmware on  Disk : d12 to H109090SESUN900G A600

INFO: 2020-05-15 13:58:30: Updating the  Disk : d13 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:59:02: Successfully updated the firmware on  Disk : d13 to H109090SESUN900G A600

INFO: 2020-05-15 13:59:02: Updating the  Disk : d14 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 13:59:35: Successfully updated the firmware on  Disk : d14 to H109090SESUN900G A600

INFO: 2020-05-15 13:59:35: Disk : d15 is running with latest firmware version : H109090SESUN900G A690

INFO: 2020-05-15 13:59:35: Not attempting to patch Disk : d15

INFO: 2020-05-15 13:59:35: Updating the  Disk : d16 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 14:00:07: Successfully updated the firmware on  Disk : d16 to H109090SESUN900G A600

INFO: 2020-05-15 14:00:08: Updating the  Disk : d17 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 14:00:40: Successfully updated the firmware on  Disk : d17 to H109090SESUN900G A600

INFO: 2020-05-15 14:00:41: Updating the  Disk : d18 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 14:01:21: Successfully updated the firmware on  Disk : d18 to H109090SESUN900G A600

INFO: 2020-05-15 14:01:21: Updating the  Disk : d19 with the firmware : H109090SESUN900G A600

SUCCESS: 2020-05-15 14:02:07: Successfully updated the firmware on  Disk : d19 to H109090SESUN900G A600

INFO: 2020-05-15 14:02:08: ………………..Patching local HDDs……………

INFO: 2020-05-15 14:02:08: ………………..Patching Expanders……………

INFO: 2020-05-15 14:02:08: Updating the  Expander : c1x0 with the firmware : DE2-24P 0018

SUCCESS: 2020-05-15 14:04:21: Successfully updated the firmware on  Expander : c1x0 to DE2-24P 0018

INFO: 2020-05-15 14:04:22: Updating the  Expander : c2x0 with the firmware : DE2-24P 0018

SUCCESS: 2020-05-15 14:06:41: Successfully updated the firmware on  Expander : c2x0 to DE2-24P 0018

INFO: 2020-05-15 14:06:41: ………………..Patching Controllers……………

INFO: 2020-05-15 14:06:41: Updating the  Controller : c0 with the firmware : 0x0072 11.05.03.00

SUCCESS: 2020-05-15 14:08:37: Successfully updated the firmware on  Controller : c0 to 0x0072 11.05.03.00

INFO: 2020-05-15 14:08:37: Updating the  Controller : c1 with the firmware : 0x0072 11.05.03.00

SUCCESS: 2020-05-15 14:10:50: Successfully updated the firmware on  Controller : c1 to 0x0072 11.05.03.00

INFO: 2020-05-15 14:10:50: Updating the  Controller : c2 with the firmware : 0x0072 11.05.03.00

SUCCESS: 2020-05-15 14:13:09: Successfully updated the firmware on  Controller : c2 to 0x0072 11.05.03.00

INFO: 2020-05-15 14:13:09: ————Finished the storage Patching————

INFO: 2020-05-15 14:13:12: —————–Patching Ilom & Bios—————–

INFO: 2020-05-15 14:13:12: Updating the Ilom using LAN over USB

INFO: 2020-05-15 14:13:13: Enabling Interconnect

INFO: 2020-05-15 14:13:35: Getting the SP IP address

INFO: 2020-05-15 14:13:48: Updating the ILOM. It takes a while

INFO: 2020-05-15 14:29:49: Verifying the updated Ilom Version, it may take a while if ServiceProcessor is booting

INFO: 2020-05-15 14:29:51: Waiting for the service processor to be up

INFO: 2020-05-15 14:34:37: Disabling Interconnect

SUCCESS: 2020-05-15 14:34:50: Successfully updated the ILOM with the firmware 3.1.2.10.d r83372

INFO: Patching the infrastructure on node: hogwarts2 , it may take upto 30 minutes. Please wait

…

…………done

INFO: Infrastructure patching summary on node: 192.168.16.24

SUCCESS: 2020-05-15 15:06:37:  Successfully upgraded the HMP

SUCCESS: 2020-05-15 15:06:37:  Successfully updated the OAK

SUCCESS: 2020-05-15 15:06:37:  Successfully updated the JDK

SUCCESS: 2020-05-15 15:06:37:  Successfully upgraded the OS

INFO: 2020-05-15 15:06:37:  IPMI is already upgraded

INFO: 2020-05-15 15:06:37:  Storage patching summary

SUCCESS: 2020-05-15 15:06:37:  No failures during storage upgrade

SUCCESS: 2020-05-15 15:06:37:  Successfully updated the ILOM & Bios

INFO: Infrastructure patching summary on node: 192.168.16.25

SUCCESS: 2020-05-15 15:06:37:  Successfully upgraded the HMP

SUCCESS: 2020-05-15 15:06:37:  Successfully updated the OAK

SUCCESS: 2020-05-15 15:06:37:  Successfully updated the JDK

SUCCESS: 2020-05-15 15:06:37:  Successfully upgraded the OS

INFO: 2020-05-15 15:06:37:  IPMI is already upgraded

INFO: 2020-05-15 15:06:37:  Storage patching summary

SUCCESS: 2020-05-15 15:06:37:  No failures during storage upgrade

SUCCESS: 2020-05-15 15:06:37:  Successfully updated the ILOM & Bios

INFO: Running post-install scripts

…………done

INFO: OS is upgraded to UEK.

INFO: current GI version is 11.2.0.3.7(16619892,16742216)

INFO: You must upgrade the GI also

INFO: Some of the patched components require node reboot. Rebooting the nodes

INFO: Setting up the SSH

…………done

Broadcast message from root (Sat Feb  7 15:11:54 2020):

The system is going down for system halt NOW!             

5. Step: We apply Patch for GI ( Grid Infrastructure ). First node, then the second node applies it automatically. We don’t need to run it on the second node.             

Oracle Grid Infrastructure (GI) Stack including ASM, and TFA.

[root@hogwarts ~]# /opt/oracle/oak/bin/oakcli update -patch 2.10.0.0.0 –gi

Please enter the ‘root’ user password:

Please re-enter the ‘root’ user password:

Please enter the ‘grid’ user password:

Please re-enter the ‘grid’ user password:

INFO: Setting up the SSH

……….done

INFO: Running pre-install scripts

……….done

INFO: 2020-05-15 16:01:36: Running pre patch script for 2.10.0.0.0

INFO: 2020-05-15 16:01:39: Completed pre patch script for 2.10.0.0.0

INFO: Stopped Oakd

…

…

INFO: 2020-05-15 16:02:16: Upgrading GI, it may take up to 45 mins

…………done

SUCCESS: Ran /usr/bin/rsync -tarqvz /opt/oracle/oak/onecmd/ [email protected]:/opt/oracle/oak/onecmd –exclude=*zip –exclude=*gz –exclude=*log –exclude=*trc –exclude=*rpm and it returned: RC=0

……….done

……….done

…

SUCCESS: All nodes in /opt/oracle/oak/onecmd/tmp/db_nodes are pingable and alive.

INFO: 2020-05-15 16:03:25: Installing GI clone

INFO: 2020-05-15 16:09:05: Running root scripts

SUCCESS: 2020-05-15 16:26:38: Successfully upgraded the GI

…

……….done

INFO: GI patching summary on node: hogwarts

SUCCESS: 2020-05-15 16:27:01:  Successfully upgraded GI

INFO: GI patching summary on node: hogwarts2

INFO: Running post-install scripts

……….done

…

…

INFO: Started Oakd

INFO: Setting up the SSH

……….done     

6. Step: Checking the Components that are Update

[root@hogwarts ~]# /opt/oracle/oak/bin/oakcli show version –detail

Reading the metadata. It takes a while…

System Version  Component Name            Installed Version         Supported Version

————–  —————           ——————        —————–

2.10.0.0.0

                Controller_INT            11.05.03.00               Up-to-date

                Controller_EXT            11.05.03.00               Up-to-date

                Expander                          0018                      Up-to-date

                SSD_SHARED                         9440                      Up-to-date

                HDD_LOCAL                          A31A                      A606

                HDD_SHARED {

                [ c2d0,c2d1,c2d2,c2d          A600                      Up-to-date

                3,c2d4,c2d5,c2d6,c2d

                7,c2d8,c2d9,c2d10,c2

                d11,c2d12,c2d13,c2d1

                4,c2d16,c2d17,c2d18,

                c2d19 ]

                [ c2d15 ]                               A690                             A600

                             }

                ILOM                                   3.1.2.10.d r83372         Up-to-date

                BIOS                                    17050100                        Up-to-date

                IPMI                                    1.8.10.5                           Up-to-date

                HMP                                   2.2.6.5                              Up-to-date

                OAK                                    2.10.0.0.0                          Up-to-date

                OEL                                    5.9                                       Up-to-date

                GI_HOME                         11.2.0.4.2(18031740,      Up-to-date

                                                                             18031668)

                DB_HOME                   11.2.0.3.7(16619892, 16742216)           11.2.0.3.10(18031683,17592127)

[root@hogwarts ~]#

7. Step

[oracle@hogwarts ~]$ db

[oracle@hogwarts ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Sat Feb 7 16:40:25 2020

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 – 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Data Mining and Real Application Testing options

SQL> alter database recover managed standby database using current logfile disconnect;

Database altered.

Step: Final checks are made and the Prod environment will be taken over the ODA again with the Switchover steps.