On UNIFI Network, WDM incremental backups fail repeatedly with ORA-600 errors in the alert log - WKB49296

SYMPTOMS

• Incremental backups fail repeatedly with no errors in Network server log
• When reviewing the alert log, the following is recorded:
  ORA-00600: internal error code, arguments: [krbb1cf_badblk]referencing the WatSteamblob datafile

ENVIRONMENT

• UNIFI 1.8.x Network
• UNIFI 1.9.x Network

CAUSE

Oracle bug

ORA-600:[krbb1cf_badblk] during RMAN backup (Doc ID 2088954.1)

US1#218145

CRI-388

Fixed on Oracle 19c which is used in waters_connect 1.9.12 and later

FIX or WORKAROUND

• Disable Incremental Backups and rely on one Full Backup (Sunday) and archive logs. In this case please ensure daily archive logs are also copied to the offline backup copy location as they will also be needed in case of disaster recovery. They are located in the ARCHIVELOG folder in the FRA
• Optionally, a daily backup task can be set for just the archivelogs. These backups will also reside in the BACKUPSET folder in the FRA with the Full backup.
• Another option is to disable the block change tracking as follows BUT this will result in much larger incremental backups and may exhaust the FRA disk space so only an option for smaller databases (discuss with GSS if considering).
  • login to SQLPLUS then run
  • SQL> ALTER DATABASE DISABLE BLOCK CHANGE TRACKING;
  • Reboot the server

ADDITIONAL INFORMATION