My apologies for being away. I've been enjoying the Christmas season, traveling, working and such things and being sick. I'm healing up, So, now it's time to get caught up and post more regularly.I've also got to get my 12c OCP book done for Sybex. I'm so far behind on that one.
I have some additional errata to post on the 12c New Features book soon. Thanks to everyone who has provided feedback. Writing a book is time consuming and it's easy to miss a dot in a product version as one person has pointed out to me....
January QFSDP Is Out
It's time and the January QFSDP is out now. I've not had the opportunity to look at the patch set in it's totality yet. I'll be doing so in the next few days and I'll post a blog entry on that. I did notice that the 12.x version of the storage cell software does not appear to be in the release. It is available in a separate download. I hope to have time to experiment with both of these in the new future but resources and time are thin. I'll see what I can do. All of this is listed in MOS note 888828.1.
A checklist
If you find upgrading Exadata to be a difficult task, with lots of steps and the like, I think you are probably correct. I've had several people tell me how much harder upgrading Oracle was for them now with Exadata. I remind them that they are doing a great deal more than they did before, and the fact that it goes smoothly more often than not is really a testament to the folks doing the Exadata development.
As I've done several Exadata patches now, I've developed a checklist that I use and keep up to date (as much as I can). It pulls in the various tasks from the read me's and adds additional tasks that I think are important. I'm going to be updating this spreadsheet for the January QFSDP and then testing it. I may release some version of the checklist to the public domain, but I need to make sure our friends at Oracle Legal are fine with that. So, keep an eye out for that if you would find it helpful.
Exadata Critical Issues
On another front, you probably want to make a trip to MOS note 1270094.1 which is titled "Exadata Critical Issues". There you will find some new Exadata issues you will want to be aware of. These have to do with some infiniband patching issues which are related to Exalogic, Issues with rolling upgrades, and serious issues with backups.
Exadata Infiniband Issues
\If you are running Exadata and Exalogic together, then you will want to be aware of a issue revolving around patching Infiniband.This is critical bug IB4 and is related to bug 16833031. In short, before you go applying patches to Exalogic and Exadata you will want to review this note. I can not verify if Exacheck will pick up on this issue or not. If you get some Exacheck output that does raise this issue, I'd be happy if you could send me a screen dump of the relevant Exacheck message. Just mail me at robertgfreeman@yahoo.com.
Note that this issue also appears to impact Super cluster according to note 1452277.1./
EX13 - Rolling Upgrades
The second issue to be aware of is EX13. This impacts Exadata when you are doing rolling upgrades. The basic text of the issue reads as follows:
One
or more cells is using version 11.2.3.3.0 combined with 11.2.3.2.x or
11.2.3.1.x, running database version 11.2.0.2 or 11.2.0.3. Bug 17854520
- Running cell version 11.2.3.3.0 combined with 11.2.3.2.x or
11.2.3.1.x (e.g. during rolling cell patching or rack expansion) using
Oracle Database 11.2.0.3 or 11.2.0.2 without patch 17854520 installed
can cause database hang or crash
Fixed in Oracle Database 11.2.0.4 and 12.1.0.1.
Before creating the mixed cell version configuration, install the fix for bug 17854520 in 11.2.0.3 and 11.2.0.2 database homes.
I do know that this issue will surface on Exacheck reports as I just saw it do so last week at a customer location.
At this time, I do not see that this is an issue for Supercluster, but please double and tripple check before you start you next upgrade.
DB23 - Incremental backups not recoveable
Did that one make you shiver? DB23 is a nasty sounding bug. Here is the description:
=============================================================
RMAN incremental backups created with one of the following database patch sets:
- 12.1.0.1 GIPSU1 or earlier
- 11.2.0.3 BP21 or earlier
- any 11.2.0.2
- any 11.2.0.1
Bug 16057129
- Exadata cell optimized incremental backup can miss some blocks if a
database file grows larger while the file is being backed up. A missed
block can lead to stuck recovery and ORA-600[3020] errors if the
incremental backup is used for media recovery.
See Document 16057129.8 for details.
Existing RMAN incremental backups taken without the bug fix in place
should be considered invalid and not usable for database recovery,
incrementally updating level 0 backups, or standby database creation.
RMAN full backups, level 0 backups that are not part of an
incrementally updated backup strategy, and database recovery using
archived redo logs are not affected by this issue.
Step 1.Set the following parameter in all databases that use Exadata storage:
_disable_cell_optimized_backups=TRUE
SQL> alter system set "_disable_cell_optimized_backups"=TRUE scope=both;
The parameter specified above may be removed after the fix for bug
16057129 is installed by upgrade or by applying an interim patch. See
below for fix availability.
Step 2.
Create new RMAN backups. Minimally a new RMAN cumulative incremental
backup must be taken. In addition, level 0 backups that are part of an
incrementally updated backup strategy must be recreated.
Fix availability
Fixed in 12.1.0.1 GIPSU2
Fixed in Patch 16057129 for 12.1.0.1 GIPSU1
Fixed in 11.2.0.4.0
Fixed in 11.2.0.3 BP22
Fixed in Patch 16057129 for 11.2.0.3 BP21
Fixed in Patch 17599908 for 11.2.0.2 BP22
This problem also exists on Supercluster. So you will want to address it theere.
=============================================================
In my mind, this is concerning but not fatal. First, I find it unlikely that data files will extend during an incremental backup that often. It can happen, but it's going to be rare in many cases. Still, this bug is bad enough that I'd suggest you address it immediately if you are using an incremental backup strategy. Once everything is fixed, if you did the cell workaround, don't forget to reverse it out. So often when I come on customer sites,m I find parameter files full of old legacy parametrer settings that never got removed. There was one white paper I saw from Oracle on upgrades. On one upgrade they left legacy parameters in place. In the other upgrade they took them out. The upgrade with the parameters removed performed much faster! Moral of the story, take advantage of upgrades to update /modernize your parameter files.
More later! Thanks for reading! Comments always welcome.
