A recent BlackBerry outage is the nightmare for both RIM and its users. Nature and mass popularity of Blackberry service  make this particular outage highly visible. There are many other in-house outages we never hear about. Companies prefer to keep silent about them, if they can. The only insight we get into the poor state of DR procedures is in some public cases like this one, or from limited personal experience. In my Oracle database infrastructure consulting career I have seen several serious production outages and was part of a few data recovery efforts.

Backups and DR procedures are just not as high on IT priority list as development, production support and new projects are. Backups and DR are frequently considered a chore, mechanical stuff, uninteresting work. Oracle database works fine most of the time, it is very reliable and robust product. When disaster strikes, be it human mistake or external cause (hardware failure), it is often difficult to recover from it.

Fairly large percentage of Oracle database backups in average enterprise fail every day and nobody even notices. Recovery of failed Oracle production database is not a simple task, most of database administrators can not do it properly. There are many possible variations and cases during Oracle database recovery. Smart, experienced, collected and well trained DBA will perhaps be able to recover the database, if all elements are properly aligned and available. Loss of data, incomplete recovery or even ad-hoc rebuild of production environments is very real possibility.