Internal Error Code Arguments 3020
Contents |
Jackson | 0 Comments The Problem We are running Windows Server 2003, Sp2 on both the primary and standby
Ora-10567 Redo Is Inconsistent With Data Block Standby
DB servers, using Oracle 11.2.0.2. We are using a data guard physical oracle support standby DB, also. We have a large tablespace due to a migration that we had completed as a one-off to load the data into the new database. I attempted to shrink some of the datafiles to make them more manageable and not use up unnecessary space. After making the changes I checked that the physical standby database had been updated properly and found some errors in the alert log: ORA-00600: internal error code, arguments: [3020], [9], [490111], [38238847], [], [], [], [], [], [], [], [] ORA-10567: Redo is inconsistent with data block (file# 9, block# 490111, file offset is 4014989312 bytes) ORA-10564: tablespace CDC_PUBLISHER_X1M ORA-01110: data file 9: ‘D:\ORADATA\CTS\CDC_PUBLISHER_X1M02.DBF' ORA-10561: block type ‘TRANSACTION MANAGED DATA BLOCK', data object# 1367456 Incident details in: D:\diag\rdbms\dg\incident\incdir_66377\db_pr02_4048_i66377.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Slave exiting with ORA-600 exception The Cause Oracle support say that this is a bug which is fixed in 11.2.0.2 patchset 14 (which was not out for Windows at the time of writing). The Solution The first thing to do would be to apply the patch which fixes the issue. Failing that, or perhaps in addition to that if it's required, you can take the following steps to resolve the issue: 1. On the standby database stop apply ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL; 2. Shutdown the standby database SHUTDOWN IMMEDIATE; 3. Shutdown the primary database SHUTDOWN IMMEDIATE; 4. Copy the affected files from the primary to the standby database Copy File1 File2 5. Restart the primary DB STARTUP 6. Start the standby DB and restart apply services STARTUP MOUNT ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION; Hopefully this has helped
[], [], [], [], [], [], [] - Oracle 11gr2 RAC - Active data-guard Let me brief the background, we have banking database with 2 node RAC in production and we have configured Active data-guard and it is fully utilized for reports database for every minute and we cannot effort down time for even for a minute. The steps I http://www.ora00600.com/wordpress/scripts/ora600/ora-00600-internal-error-code-arguments-3020-9/ followed were Inspired by "The Arup Nanda Blog: Resolving Gaps in Data Guard Apply Using Incremental RMAN Backup " : http://arup.blogspot.com/2009/12/resolving-gaps-in-data-guard-apply.htmlA big thanks to Arup (I never interacted with him)I will narrate the chain of events as they happened: We use the following query to check for http://bbalijepalli.blogspot.com/2011/02/work-around-ora-600-internal-error.html the sync:Query-1SQL> SELECT ARCH.THREAD# "Thread", ARCH.SEQUENCE# "Last Sequence Received", APPL.SEQUENCE# "Last Sequence Applied", (ARCH.SEQUENCE# - APPL.SEQUENCE#) "Difference" FROM (SELECT THREAD# ,SEQUENCE# FROM V$ARCHIVED_LOG WHERE (THREAD#,FIRST_TIME ) IN (SELECT THREAD#,MAX(FIRST_TIME) FROM V$ARCHIVED_LOG GROUP BY THREAD#))ARCH, (SELECT THREAD# ,SEQUENCE# FROM V$LOG_HISTORY WHERE (THREAD#,FIRST_TIME ) IN (SELECT THREAD#,MAX(FIRST_TIME) FROM V$LOG_HISTORY GROUP BY THREAD#)) APPL WHERE ARCH.THREAD# = APPL.THREAD# ORDER BY 1; Thread Last Sequence Received Last Sequence Applied Difference---------- ---------------------- --------------------- ---------- 1 44878 44878 0 2 15739 15739 0My team mate noticed the difference has risen to 7 & 8 in the respective threads I started investigating and look at the alert log and we had the ora-600 error and also looked at the following query:Query-2SQL> SELECT PROCESS, STATUS, THREAD#, SEQUENCE#,BLOCK#, BLOCKS FROM V$MANAGED_STANDBY;(Sample query output not actual result)PROCESS STATUS THREAD# SEQUENCE# BLOCK# BLOCKS--------- ------------ ---------- ---------- ---------- ----------A
- redo is inconsistent with data block [message #161503] Sun, 05 March 2006 20:36 skodman Messages: 10Registered: March 2005 Location: Auckland Junior http://www.orafaq.com/forum/t/59863/0/ Member Hi all Here is the situation. I have primary db in https://asksundar.wordpress.com/2015/09/16/corruption-3020/ town A and standby in town b. Standby runs in managed recovery mode. Database version is 10GR2. couple a days ago sysadmin restarted the server where stdby db was running, without properly shutting down the standby database and closing redo log apply service first. Now after internal error mounting the database i am getting this error: ORA-00600: internal error code, arguments: [3020], [2], [137], [8388745], [], [], [], [] ORA-10567: Redo is inconsistent with data block (file# 2, block# 137) ORA-10564: tablespace UNDOTBS1 ORA-01110: data file 2: 'O:\ORADATA\ABCD\UNDOTBS01.DBF' ORA-10560: block type 'KTU SMU HEADER BLOCK' Restarting the redo log apply service didn't help to clear internal error code the error. What is my best course of action to get rid of the error and make logs properly apply to standby regards skodman Report message to a moderator Re: managed recovery - redo is inconsistent with data block [message #161551 is a reply to message #161503] Mon, 06 March 2006 00:39 alexzeng Messages: 133Registered: August 2005 Location: alexzeng.wordpress.com Senior Member Please refer this article: https://metalink.oracle.com/metalink/plsql/f?p=130:14:2019257725183951363::::p14_database_id,p14_docid,p14_show_header,p14_show_help, p14_black_frame,p14_font:NOT,30866.1,1,0,1,helvetica Also post here, FYI ------------------------------------------------------------------ Subject: ORA-600 [3020] "Stuck Recovery" Doc ID: Note:30866.1 Type: REFERENCE Last Revision Date: 09-JAN-2006 Status: PUBLISHED Note: For additional ORA-600 related information please read Note 146580.1 PURPOSE: This article discusses the internal error "ORA-600 [3020]", what it means and possible actions. The information here is only applicable to the versions listed and is provided only for guidance. ERROR: ORA-600 [3020] [a] [b] [c] [d] [e] VERSIONS: version 6.0 to 10.1 DESCRIPTION: This is called a 'STUCK RECOVERY'. There is an inconsistency between the information stored in the redo and the information stored in a database block
DBA Cause: ====== 1) It can be a lost write happened (could be NON-ORACLE issue) 2) It could be a bug What is lost write ============== Considering single block Step 1) Block 1 had scn of 1395 STEP 2) Block 1 was updated and scn incremented to 20000 in buffer cache. So the change vector in the redo recorded the previous SCN to be 1395 and changed scn to 20000. STEP 3) Block 1 was indicated to be flushed to disk but due an I/O issue the block was flushed but not written to disk. So the SCN for the block in disk remains 1395. STEP 4) Again the same block gets updated and the scn gets incremented from 1395 to 50000. So the change vector in the redo recorded the previous SCN to be 1395 and changed scn to 50000. STEP 5) The redolog gets shipped to standby STEP 6) The recovery applies first redo change vector and changes the block scn from 1395 to 20000. STEP 7) The recovery tries to apply the second change vector. It finds the block scn to be 20000 whereas it is expecting it to be 1395 since for this change vector the previous scn recorded is 1395. Recovery stops with ora-00600 [3020] because of the lost write which happened in step 3. When this issue can occur: ========================= 1) during recovery this will be reported -> It can be normal hot backup and recovery -> It can be RMAN backup and recovery -> it can be reported in standby recovery Solution: ========= If it is backup and recover (including RMAN) -> we will have to cancel the recovery and open the database till that point OR -> if you cannot stop the recovery till that time and if you want to recover further then you will have to allow corruption into your database and perform recovery (as below) SQL> recover database allow 1 corruption; Note: doing recovery by allowing corruption may create many issue, you may get errors as below ORA-600 [4194] ORA-600 [4193] ORA-600 [2662] ORA-600 [4663] ORA-08103 ORA-08102 If you face this issue with standby, then you may have to take backup of your affected datafiles from primary and restore in standby… "Note: this decision has to be taken by you along with oracle support" SQL> recover standby database test;