Over the past few weeks, I've been working on some new (and older) installations of Exadata, and came across a few items that piqued my interest. Each of these things had been on my mind for a while, but it's nice to see them finally resolved.
The first is a small change to the installation tree of the Oracle homes on Exadata. With the release of 11.2.0.2, Oracle created a new "best practice" of performing all patch sets out of place into a new home. While this makes it really easy to roll back a patch, the default naming convention for Oracle homes on Exadata became a bit of a sticky situation. If your 11.2.0.2 Grid Infrastructure home was at /u01/app/11.2.0/grid, where would you put your 11.2.0.3 home when it's ready to come out? This was the topic of more than a few discussions around the Enkitec office. Do you extend the version out another digit to 11.2.0.3, or version the home (/u01/app/11.2.0/grid_11.2.0.2, etc). Well, Oracle has put this discussion to rest....Your new Oracle home directories on Exadata are:
Grid Infrastructure - /u01/app/11.2.0.2/grid
Database - /u01/app/oracle/product/11.2.0.2/dbhome_1
read on for another change (it has to do with bundle patches)
The second change affects everyone, as it has to do with patches. One of the more frequent questions I get from Exadata clients is "how can I easily tell what my patch levels are on Exadata?" The answer was always to run an "opatch lsinventory" and "imageinfo" and then cross-reference these with the supported versions note for Exadata (888828.1). It was a little tricky sometimes with the bundle patches, as there were several patch numbers listed in the inventory, even for one bundle patch.
For example, here's the output from a system patched with bundle patch 8:
[enkdb03:oracle:DEMO1] /home/oracle
> opatch lsinventory
Invoking OPatch 11.2.0.1.6
Oracle Interim Patch Installer version 11.2.0.1.6
Copyright (c) 2011, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/oracle/product/11.2.0.2/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /etc/oraInst.loc
OPatch version : 11.2.0.1.6
OUI version : 11.2.0.2.0
Log file location : /u01/app/oracle/product/11.2.0.2/dbhome_1/cfgtoollogs/opatch/opatch2011-09-28_03-23-36AM.log
Lsinventory Output file location : /u01/app/oracle/product/11.2.0.2/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2011-09-28_03-23-36AM.txt
--------------------------------------------------------------------------------
Installed Top-level Products (1):
Oracle Database 11g 11.2.0.2.0
There are 1 products installed in this Oracle Home.
Interim patches (3) :
Patch 12332686 : applied on Wed Jul 20 10:33:34 CDT 2011
Unique Patch ID: 13789775
Created on 26 May 2011, 04:53:25 hrs PST8PDT
Bugs fixed:
12332686, 10626132
Patch 12419090 : applied on Wed Jul 20 10:33:20 CDT 2011
Unique Patch ID: 13789775
Created on 3 May 2011, 23:04:18 hrs PST8PDT
Bugs fixed:
9959110, 10314123, 10014392, 10157622, 10089120, 10057296, 10385838
10053985, 9864003, 10425674, 10044622, 9812970, 10083789, 10073372
9876201, 9963327, 11077756, 10375649, 9336825, 10062301, 10018215
10105195, 10007185, 10376847, 10071992, 10634513, 10038791, 11655840
10048487, 9926027, 10260251, 10052721, 10028235, 10027079, 10028343
11866171, 10231906, 10065216, 10045436, 9907089, 9897335, 10175855
9891341, 10284828, 10072474, 10036834, 10028637, 10029900, 9974223
9979706, 10016083, 10015460, 9918485, 9971646, 11782423, 10040647
9978765, 10069541, 9915329, 10107380, 10110969, 10029119, 10305361
10233159, 10083009, 9812956, 10008467, 10036193, 10048027, 10040109
10015210, 9944978, 10033106, 9978195, 10042143, 10284693, 11877079
9679401, 10111010, 10075643, 10057680, 10205290, 10124517, 10078086
9944948, 10146768, 10052529, 10011084, 10073075, 10299006, 10248739
10236074, 10128191, 10253630, 10178670, 9975837, 10168006, 9949676
Patch 12576172 : applied on Wed Jul 20 10:32:30 CDT 2011
Unique Patch ID: 13789775
Created on 17 Jun 2011, 05:57:18 hrs PST8PDT
Bugs fixed:
9940990, 10249791, 10031806, 11935601, 9744252, 9956713, 10635701
9591812, 10127360, 10207551, 11795047, 11723722, 10396874, 9443361
11772687, 9707965, 10245086, 9845644, 12327579, 10147185, 11658679
10039731, 10077191, 10195991, 10035521, 12417897, 10184634, 10258337
10264680, 10209232, 9819413, 10240581, 10284570, 9770718, 10106828
9881076, 10621169, 10048701, 9078442, 11057263, 10126219, 10073205
10251084, 10092153, 10264696, 8223165, 10142909, 10052956, 10332589
10329146, 11870249, 11677855, 10639365, 9869401, 10218578, 10229719
11724984, 10411618, 11927619, 9724970, 10296898, 10084145, 10285394
10367188, 10208386, 10142776, 10245259, 11656163, 10190642, 10129643
10113633, 9715581, 10302581, 11889177, 10121589, 10126094, 10396041
11826679, 11785390, 10375930, 9795214, 10142788, 9905049, 11674898
6892311, 11936885, 10358019, 10134407, 11864352, 10207092, 9965655
11651810, 11067567, 9869287, 9375300, 10155605, 10126680, 10356782
10326338, 10051315, 10218814, 10261389, 10033603, 11684307, 11795819
9929660, 9889683, 9761434, 11707302, 10230571, 10043801, 10205715
10227133, 10353880, 10005127, 10061015, 10260808, 10094949, 9572787
9309735, 12387009, 10632425, 10113803, 10073683, 10269026, 9748749
10022980, 10355493, 11674485, 10006008, 10187168, 9041435, 10158965
10044114, 10010252, 10204358, 10356513, 10378005, 10026193, 10170431
10222719, 10403641, 10035737, 10384285, 10630260, 11069199, 11867363
10205230, 10052141, 11818335, 10040921, 10228393, 10368754, 10199627
9465079, 10219576, 11707699, 9951423, 11695333, 10138589, 8752691
10270888, 9500046, 10220118, 10106107, 10278372, 11719062, 10149223
11737421, 10122077, 10157249, 9735237, 10317487, 10219583, 10636231
10310299, 11074393, 11065646, 12433293, 10055063, 10368698, 9709292
10079168, 10098253, 10035825, 10233732, 10353054, 10314582, 9953542
10033071, 10080579, 11699057, 10110863, 9919654, 10363186, 10004943
10417716, 12341386, 11729347, 10019218, 10239480, 10373381, 9539440
10261072, 10061490, 10124841, 10190421, 11800170, 11695285, 10157402
9941757, 9651350, 10299224, 10151017, 10010792, 11724916, 10228635
10018789, 10248523, 9770451, 10094635, 10425676, 9824198, 9683047
10180307, 10071193, 9924349, 10221016, 10132870, 10094201, 10171273
10145612, 10193846, 11664046, 10324294, 10046912, 9414040, 10086980
11830776, 10364073, 11830777, 11830778, 10200404, 10147588, 11073768
10102506, 10640142, 10051966, 10040531, 10082277, 10222405, 9788588
10026967, 10260870, 10054513, 12400621, 10105926, 10040035, 11781487
10013431, 10228151, 9172465, 10324526, 10217802, 10238786, 9791810
10026972, 10332111, 10352368, 10227288, 10261680, 10623249, 11887380
9671271, 11664719, 10094732, 9735282, 11800854, 9853076, 10097711
6523037, 10053725, 8685446
Rac system comprising of multiple nodes
Local node = enkdb03
Remote node = enkdb04
--------------------------------------------------------------------------------
OPatch succeeded.
How did we determine that we were running bundle patch 8? Well, we looked for the last patch listed, and grabbed that patch number, then looked it up on the supported versions note. On a system that has more than a few patches installed (I'm looking in your direction, EBS), it's not always as easy as looking at the last patch listed. Also, wouldn't you expect the "Unique Patch ID" to be the bundle patch number, since it's common across the 3 patches included in the bundle? Not so.
Back to the new feature. Take a look at the output of "opatch lsinventory" from this system and see if you can spot anything different:
[oracle@co01db01 config]$ $ORACLE_HOME/OPatch/opatch lsinventory
Invoking OPatch 11.2.0.1.6
Oracle Interim Patch Installer version 11.2.0.1.6
Copyright (c) 2011, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/oracle/product/11.2.0/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /etc/oraInst.loc
OPatch version : 11.2.0.1.6
OUI version : 11.2.0.2.0
Log file location : /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2011-08-30_10-25-12AM.log
Lsinventory Output file location : /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2011-08-30_10-25-12AM.txt
--------------------------------------------------------------------------------
Installed Top-level Products (1):
Oracle Database 11g 11.2.0.2.0
There are 1 products installed in this Oracle Home.
Interim patches (3) :
Patch 12332686 : applied on Mon Aug 29 22:14:45 CDT 2011
Unique Patch ID: 13971811
Patch Description: "Exadata Database Recommended Patch (Diskmon) : 11.2.0.2.10 (12332686)"
Created on 17 Jul 2011, 21:22:04 hrs PST8PDT
Bugs fixed:
12332686, 10626132
Patch 12421404 : applied on Mon Aug 29 22:14:31 CDT 2011
Unique Patch ID: 13971811
Patch Description: "Exadata Database Recommended Patch (CRS) : 11.2.0.2.10 (12421404)"
Created on 3 Jul 2011, 00:38:21 hrs PST8PDT
Bugs fixed:
9959110, 10272615, 10314123, 10014392, 10089120, 10057296, 9864003
11775080, 9916145, 10044622, 12399977, 12421404, 12340700, 10056713
10637741, 9939306, 9902536, 10007185, 10376847, 10038791, 11741224
11655840, 10048487, 10322157, 10260251, 10052721, 10028235, 10027079
10357258, 10045436, 10231906, 10622973, 9891341, 10072474, 10036834
10029900, 9974223, 10016083, 9918485, 11781515, 10040647, 10069541
10029119, 10233159, 12332919, 9812956, 10036193, 10015210, 12340501
10621175, 11877079, 8906163, 10111010, 10115514, 10104377, 10057680
10280665, 10078086, 9944948, 10146768, 10052529, 10011084, 10012319
10073075, 10233811, 10299006, 10248739, 10236074, 10128191, 11071429
10019726, 10178670, 9975837, 10253630, 9949676, 11936945, 10637483
10157622, 11698552, 10385838, 10053985, 10425674, 9812970, 11828633
11899801, 10083789, 9876201, 10073372, 9963327, 11077756, 10375649
9336825, 11682409, 10062301, 10018215, 10105195, 10419987, 10071992
10634513, 9926027, 10103954, 10028343, 11866171, 10065216, 9907089
9897335, 10190153, 11744313, 10175855, 10284828, 10028637, 10361177
9979706, 10324594, 10015460, 9971646, 11782423, 11654726, 9978765
10398810, 11904778, 10397652, 9915329, 10107380, 10110969, 10305361
10331452, 10083009, 10631693, 10008467, 10048027, 10040109, 9944978
10033106, 9978195, 11840629, 10042143, 10284693, 10638381, 9679401
11663339, 10075643, 10205290, 10124517, 11069614, 9593552, 10168006
12677816, 11807012
Patch 12773458 : applied on Mon Aug 29 22:13:16 CDT 2011
Unique Patch ID: 13971811
Patch Description: "Exadata Database Recommended Patch (Database) : 11.2.0.2.10 (12773458)"
Created on 26 Jul 2011, 22:25:05 hrs PST8PDT
Bugs fixed:
9940990, 10249791, 10031806, 11935601, 9744252, 9956713, 12314102
10635701, 9591812, 10127360, 10207551, 11795047, 11723722, 10396874
9443361, 11772687, 12337910, 9707965, 10259565, 10245086, 9845644
9401552, 12327579, 10147185, 11658679, 10039731, 12366627, 10077191
10195991, 9842238, 10035521, 12417897, 10184634, 10258337, 7293136
10264680, 10094823, 10209232, 9819413, 10240581, 11907498, 10284570
9770718, 10106828, 9672816, 9881076, 11906405, 10621169, 10048701
12569482, 9078442, 11057263, 10126219, 10073205, 12584509, 10251084
12340939, 9835605, 9380245, 10092153, 8223165, 10264696, 10142909
12730844, 10052956, 10285022, 10332589, 10329146, 11870249, 11677855
10639365, 9869401, 10218578, 10229719, 11724984, 10411618, 11927619
9724970, 10296898, 10084145, 10367188, 10285394, 10208386, 10142776
10245259, 11656163, 10190642, 12586486, 12586487, 10129643, 12586488
10113633, 12586489, 9715581, 10302581, 10121589, 11889177, 10126094
12341758, 10396041, 11826679, 12400751, 9436324, 11785390, 10375930
12586490, 12586491, 12586492, 9795214, 12586493, 10142788, 12586494
12586495, 9905049, 12586496, 11674898, 6892311, 11936885, 10264285
10358019, 12431716, 10422126, 10134407, 11864352, 10207092, 11781904
11879563, 9965655, 11890804, 11651810, 11067567, 11716621, 9869287
9375300, 10155605, 10126680, 10356782, 10326338, 10051315, 10420027
10218814, 10261389, 12340464, 10425616, 10033603, 12569737, 11076894
11684307, 11795819, 9929660, 9889683, 9761434, 11707302, 10230571
10043801, 10205715, 9966609, 12419321, 10227133, 10353880, 10005127
12647754, 10061015, 10260808, 10094949, 9572787, 9309735, 12387009
10632425, 10113803, 10073683, 9578670, 10269026, 9748749, 10022980
12579947, 12419331, 10355493, 11674485, 10006008, 10187168, 10648873
9041435, 10158965, 10044114, 12635537, 12698611, 10010252, 10204358
10356513, 10378005, 10026193, 10170431, 10222719, 10403641, 10092858
10630260, 10035737, 10384285, 11883472, 11069199, 11867363, 10205230
10052141, 11818335, 12371955, 11665247, 10040921, 10228393, 10368754
12554483, 10199627, 11827088, 9465079, 10219576, 12408350, 11707699
9951423, 11695333, 10138589, 8752691, 10270888, 9500046, 10220118
11719062, 10106107, 10278372, 10149223, 11737421, 10122077, 11820118
10157249, 12378147, 9735237, 10317487, 10219583, 11074393, 10636231
10310299, 11065646, 10055063, 12433293, 10368698, 9709292, 10348372
10079168, 10098253, 10035825, 10233732, 10353054, 10314582, 9953542
10033071, 10620808, 11699057, 12620422, 10080579, 10110863, 11663661
9919654, 10363186, 10004943, 10417716, 12341386, 11064851, 11729347
10019218, 9539440, 10239480, 10373381, 10261072, 10061490, 10124841
10190421, 11775474, 11695285, 11800170, 10157402, 9941757, 9651350
10299224, 10151017, 11724916, 10010792, 10228635, 10018789, 10408903
10248523, 9770451, 10094635, 10425676, 9824198, 9683047, 10180307
12544893, 10071193, 9924349, 10221016, 10132870, 10171273, 10094201
10145612, 10193846, 11664046, 10324294, 11696169, 9414040, 10046912
10086980, 11830776, 11830777, 10364073, 11830778, 10200404, 10147588
11073768, 10102506, 10640142, 11733179, 10040531, 10051966, 10082277
12382708, 10222405, 9788588, 10026967, 10260870, 12326246, 12400621
10054513, 10040035, 10105926, 11781487, 9771278, 9172465, 10013431
10228151, 12404330, 10324526, 10217802, 10238786, 9791810, 12363485
12341996, 10026972, 10332111, 10352368, 10227288, 10261680, 10623249
11887380, 9671271, 11664719, 10094732, 9735282, 11800854, 9853076
9900417, 10097711, 6523037, 10053725, 8685446, 10363801
Rac system comprising of multiple nodes
Local node = co01db01
Remote node = co01db02
--------------------------------------------------------------------------------
OPatch succeeded.
Did you catch the difference? Starting with bundle patch 10 for 11.2.0.2, we have a new line in the patch titled a "Patch Description" that includes a version number - in this case, it's 11.2.0.2.10, meaning that we have bundle patch 10 installed. Also, you get what component is patched in each patch included in the bundle, so you can see that bundle patch 10 has separate fixes for diskmon, CRS, and database.
Where does this information come from? It's simply an added line in the $ORACLE_HOME/inventory/oneoffs//etc/config/inventory.xml file:
[enkdb01:oracle:SANDBOX1] /u01/app/oracle/product/11.2.0/dbhome_11202/inventory/oneoffs/12332686/etc/config
> cat inventory.xml
Exadata Database Recommended Patch (Diskmon) : 11.2.0.2.10 (12332686)
13971811
11.2.0.1.5
true
true
false
false
false
As I said, these items may not interest everyone...just a few notes about changes to the newer installations that I found interesting.
Re the home locations, remeber these are just suggestions. Customers are free to modify the paths to any valid location (remembering that /u01 is in it’s own filesystem/volume)
Philip…