Issue:
While applying the OJVM patch on Exadata, the patch application fails on relinking.
Error:
OPatch failed: ApplySession failed in system modification phase... 'ApplySession::apply failed: Re-link fails on target "jox_refresh_knlopt ioracle".
OJVM Patch Installation:
Go to the patch directory.
[oracle@exadb01~]$ cd /u01/exadata_patch_apr2016/22738416/Database/11.2.0.4.0/11.2.0.4.160419OJVMPSU/22674697
OPatch Conflict Check
Determine whether any currently installed interim patches conflict with this patch.
[oracle@exadb01 22674697]$ opatch prereq CheckConflictAgainstOHWithDetail -ph ./
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/oracle/product/11.2.0.4/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11.2.0.4/dbhome_1/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.4.0
Log file location : /u01/app/oracle/product/11.2.0.4/dbhome_1/cfgtoollogs/opatch/opatch2016-11-04_13-34-29PM_1.log
Invoking prereq "checkconflictagainstohwithdetail"
Prereq "checkConflictAgainstOHWithDetail" passed.
OPatch succeeded.
Ensure that you shut down all the services running from the Oracle home.
For a Non Oracle RAC environment, shut down all databases and listeners associated with the Oracle home that you are updating.
For an Oracle RAC environment, shut down all the services (database) running from the Oracle home on all the nodes you want to patch. After all nodes are patched, start all services. OPatch is used on only one node at a time.
Applying Patch
Set your current directory to the directory where the patch is located and then run the OPatch utility.
[oracle@exadb01 22674697]$ opatch apply
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/oracle/product/11.2.0.4/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11.2.0.4/dbhome_1/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.4.0
Log file location : /u01/app/oracle/product/11.2.0.4/dbhome_1/cfgtoollogs/opatch/22674697_Nov_04_2016_13_38_09/apply2016-11-04_13-38-09PM_1.log
Applying interim patch '22674697' to OH '/u01/app/oracle/product/11.2.0.4/dbhome_1'
Verifying environment and performing prerequisite checks...
Patch 22674697: Optional component(s) missing : [ oracle.sqlj, 11.2.0.4.0 ] , [ oracle.sqlj.companion, 11.2.0.4.0 ]
All checks passed.
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:
You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: y
This node is part of an Oracle Real Application Cluster.
Remote nodes: 'exadb02'
Local node: 'exadb01'
Please shut down Oracle instances running out of this ORACLE_HOME on all the nodes.
(Oracle Home = '/u01/app/oracle/product/11.2.0.4/dbhome_1')
Are all the nodes ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Patching component oracle.javavm.server, 11.2.0.4.0...
Patching component oracle.rdbms, 11.2.0.4.0...
Patching component oracle.rdbms.dbscripts, 11.2.0.4.0...
Patching component oracle.dbjava.jdbc, 11.2.0.4.0...
Patching component oracle.dbjava.ic, 11.2.0.4.0...
Patching component oracle.javavm.client, 11.2.0.4.0...
Make failed to invoke "/usr/bin/make -f ins_rdbms.mk jox_refresh_knlopt ioracle ORACLE_HOME=/u01/app/oracle/product/11.2.0.4/dbhome_1"....'make: *** No rule to make target `jox_refresh_knlopt'. Stop.
'
The following make actions have failed :
Re-link fails on target "jox_refresh_knlopt ioracle".
Do you want to proceed? [y|n]
n
User Responded with: N
OPatch failed: ApplySession failed in system modification phase... 'ApplySession::apply failed: Re-link fails on target "jox_refresh_knlopt ioracle".
'
OPatch will attempt to restore the system...
Restoring the Oracle Home...
Checking if OPatch needs to invoke 'make' to restore some binaries...
Make failed to invoke "/usr/bin/make -f ins_rdbms.mk jox_refresh_knlopt ioracle ORACLE_HOME=/u01/app/oracle/product/11.2.0.4/dbhome_1"....'make: *** No rule to make target `jox_refresh_knlopt'. Stop.
'
--------------------------------------------------------------------------------
Failed to run make commands. They are stored in file '/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/make.txt'
Invoke these commands manually to restore the binaries in the Oracle Home.
OPatch failed to restore OH '/u01/app/oracle/product/11.2.0.4/dbhome_1'. Consult OPatch document to restore the home manually before proceeding.
Log file location: /u01/app/oracle/product/11.2.0.4/dbhome_1/cfgtoollogs/opatch/22674697_Nov_04_2016_13_38_09/apply2016-11-04_13-38-09PM_1.log
OPatch failed with error code 102
Solution:
Move the jox.o file from the $ORACLE_HOME/javavm/admin directory.
[oracle@exadb01 22674697]$ ls -ltr $ORACLE_HOME/javavm/admin/jox.o
-rwxrwxrwx 1 oracle oinstall 1564176 Mar 17 2016 /u01/app/oracle/product/11.2.0.4/dbhome_1/javavm/admin/jox.o
[oracle@exadb01 admin]$ mv $ORACLE_HOME/javavm/admin/jox.o $ORACLE_HOME/javavm/admin/jox.o_BKP
Applying Patch
[oracle@exadb01 22674697]$ opatch apply
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/oracle/product/11.2.0.4/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11.2.0.4/dbhome_1/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.4.0
Log file location : /u01/app/oracle/product/11.2.0.4/dbhome_1/cfgtoollogs/opatch/22674697_Nov_04_2016_16_06_10/apply2016-11-04_16-06-10PM_1.log
Applying interim patch '22674697' to OH '/u01/app/oracle/product/11.2.0.4/dbhome_1'
Verifying environment and performing prerequisite checks...
Patch 22674697: Optional component(s) missing : [ oracle.sqlj, 11.2.0.4.0 ] , [ oracle.sqlj.companion, 11.2.0.4.0 ]
All checks passed.
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:
You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y
This node is part of an Oracle Real Application Cluster.
Remote nodes: 'exadb02'
Local node: 'exadb01'
Please shut down Oracle instances running out of this ORACLE_HOME on all the nodes.
(Oracle Home = '/u01/app/oracle/product/11.2.0.4/dbhome_1')
Are all the nodes ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Patching component oracle.javavm.server, 11.2.0.4.0...
Patching component oracle.rdbms, 11.2.0.4.0...
Patching component oracle.rdbms.dbscripts, 11.2.0.4.0...
Patching component oracle.dbjava.jdbc, 11.2.0.4.0...
Patching component oracle.dbjava.ic, 11.2.0.4.0...
Patching component oracle.javavm.client, 11.2.0.4.0...
Verifying the update...
--------------------------------------------------------------------------------
*******************************************************************************************************
*******************************************************************************************************
** ATTENTION **
** **
** It is critical and essential to complete the post installation (if you are applying the patch) or **
** post deinstallation (if you are doing rollback of the patch) as per Readme to have a working **
** and functional Oracle JavaVM Component. **
** Failure to follow the instructions could lead to inconsistencies in the database. **
** **
*******************************************************************************************************
*******************************************************************************************************
--------------------------------------------------------------------------------
Patching in all-node mode.
Updating nodes 'exadb02'
Apply-related files are:
FP = "/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/rac/copy_files.txt"
DP = "/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/rac/copy_dirs.txt"
MP = "/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/rac/make_cmds.txt"
RC = "/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/rac/remote_cmds.txt"
Instantiating the file "/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/rac/copy_files.txt.instantiated" by replacing $ORACLE_HOME in "/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/rac/copy_files.txt" with actual path.
Propagating files to remote nodes...
Instantiating the file "/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/rac/copy_dirs.txt.instantiated" by replacing $ORACLE_HOME in "/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/rac/copy_dirs.txt" with actual path.
Propagating directories to remote nodes...
Instantiating the file "/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/rac/make_cmds.txt.instantiated" by replacing $ORACLE_HOME in "/u01/app/oracle/product/11.2.0.4/dbhome_1/.patch_storage/22674697_Mar_17_2016_07_51_04/rac/make_cmds.txt" with actual path.
Running command on remote node 'exadb02':
cd /u01/app/oracle/product/11.2.0.4/dbhome_1/rdbms/lib; /usr/bin/make -f ins_rdbms.mk jox_refresh_knlopt ioracle ORACLE_HOME=/u01/app/oracle/product/11.2.0.4/dbhome_1 || echo REMOTE_MAKE_FAILED::>&2
All nodes have been patched. You may start Oracle instances on the local system and nodes 'exadb02'
There were relinks on remote nodes. Remember to check the binary size and timestamp on the nodes 'exadb02' .
The following make commands were invoked on remote nodes:
'cd /u01/app/oracle/product/11.2.0.4/dbhome_1/rdbms/lib; /usr/bin/make -f ins_rdbms.mk jox_refresh_knlopt ioracle ORACLE_HOME=/u01/app/oracle/product/11.2.0.4/dbhome_1
'
Patch 22674697 successfully applied
Log file location: /u01/app/oracle/product/11.2.0.4/dbhome_1/cfgtoollogs/opatch/22674697_Nov_04_2016_16_06_10/apply2016-11-04_16-06-10PM_1.log
OPatch succeeded.
[oracle@exadb01 22674697]$
Verify whether the patch has been successfully installed.
[oracle@exadb01 22674697]$ opatch lsinventory|grep 22674697
Patch 22674697 : applied on Fri Nov 04 16:06:42 AST 2016
[oracle@exadb02 22674697]$ opatch lsinventory|grep 22674697
Patch 22674697 : applied on Fri Nov 04 16:06:42 AST 2016
Finally, run post install steps on all databases in the patched home.