Oracle-Data-Guard-12c-Redo-Gap-Resolutions-Part-I (1)

Oracle Database 12c - Redo Gap Resolutions (Part I)

Joel Perez (Oracle ACED), Mahir M. Quluzade (OCP) & Carlos H. Y. Furushima (OCE)

In this article we will discuss Redo Gap in Oracle Data Guard configurations and resolution methods of Redo Gap; also you will see Redo Gap resolution with RMAN using new feature of Oracle Database 12c (12.1.0.1) – (Restore/Recover files over the network).

Overview Oracle Data Guard

Oracle Data Guard ensures high availability, data protection, and disaster recovery for primary (production) database. Data Guard provides a comprehensive set of services that create, maintain, manage, and monitor one or more standby databases to enable primary databases to survive disasters and data corruptions. A Standby database is copy of the primary database. Then, if the primary database becomes unavailable Data Guard can switch any standby database role to the primary role.

A Data Guard configuration consists of one database that in the primary role and one or more (30) databases that in the (Physical, Logical or Snapshot) standby role and data guard services. Redo Transport service transmit redo data from the primary database to the standby database or FAR SYNC instance in the configuration. The redo data transmitted from the primary database is written to the standby redo log on the standby database or FAR SYNC instance. Apply services automatically apply the redo data on the standby database to maintain consistency with the primary database. Redo Apply running on Physical standby and SQL Apply running on Logical standby database. Role transition service initiate a role transition between the primary database and one standby database in the Data Guard configuration.

You can manage primary and standby databases using either the SQL Command–Line (SQL*Plus) or the Oracle Data Guard Broker Manager Command-Line (DGMGRL) interfaces. The broker provides a command-line interface (DGMGRL) and a graphical user interface that is integrated in Oracle Enterprise Manager Cloud Control.

Redo Transport Modes

Redo Transport destination should be standby database or FAR SYNC instance. Each redo transport destination is configured to receive redo data via one of two redo transport modes: synchronous (SYNC) and asynchronous (ASYNC).

Synchronous (SYNC)

The SYNC redo transport mode transmits changes from primary to standby database synchronously with respect to transaction commitment. A transaction cannot commit on primary database until all redo generated by that transaction has been successfully sent to standby databases that uses the synchronous redo transport mode. This transport mode is used by the Maximum Protection and Maximum Availability data protection modes.

2 Oracle Database 12c - Redo Gap Resolutions (Part I)

Joel Perez (Oracle ACED), Mahir M. Quluzade (OCP) & Carlos H. Y. Furushima (OCE)

Note: There is no limit on the distance between a primary database and a SYNC redo transport destination, transaction commit latency increases as network latency increases between a primary database and a SYNC redo transport destination.

Asynchronous (ASYNC)

The asynchronous redo transport mode transmits redo data asynchronously with respect to transaction commitment. A transaction can commit without waiting for the redo generated by that transaction to be successfully sent to any redo transport destination that uses the asynchronous redo transport mode. This transport mode is used by the Maximum Performance data protection mode.

Apply Services

Apply services automatically apply redo to standby databases to maintain synchronization with the primary database and allow transactional consistent access to the data.

Redo Apply on Physical standby database:

Redo apply is basically a block-by-block physical replica of the primary database, redo apply uses media recovery to read records from the SRL into memory and apply change vectors directly to the standby database.

Note: USING CURRENT LOGFILE is already deprecated for start command Real-Time Apply on Oracle Database 12c (12.1). You can start Real-Time Apply as below command:

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;

SQL Apply on Logical standby database:

SQL apply uses the (LSP) logical standby process to coordinate apply of changes to the standby database. SQL apply requires more processing than redo apply, the processes that make up SQL apply, read the SRL and "mine" the redo by converting it to logical change records and then building SQL transactions and applying SQL to the standby database.

Active Data Guard

You can open physical standby database read-only while running apply service after “11g” version. Active Data Guard solves the read consistency problem in previous releases, so database can opening READ ONLY WITH APPLY mode. Read only opened standby database is very useful for read-only reporting, ad-hoc queries and read-mostly applications. If Active Data Guard enabled, then physical block corruptions are repaired automatically wherever they occur, at either the

3 Oracle Database 12c - Redo Gap Resolutions (Part I)

Joel Perez (Oracle ACED), Mahir M. Quluzade (OCP) & Carlos H. Y. Furushima (OCE)

primary or standby, preventing any interruption in service to users and eliminating manual intervention by administrators.

Note: Active Data Guard is a licensed option for Oracle Database 12c Enterprise Edition.

What is Redo Gap?

A log file gap occurs whenever a primary database continues to commit transactions while the LNS process has ceased transmitting redo to the standby database. This can happen when the network or the standby database is down and your Data Guard protection mode is not Maximum Protection.

Automatic Redo Gap Resolution

LGWR process continues writing all transactions to the current ORL (online redo log). ARCH process archives current ORL, when ORL file is switched or full filled. This cycle can repeat itself many times before the connection between the primary and the standby is restored, result a large log file gap. If problem occurs on connection between primary and standby database, then ARCH process on the primary database to continuously ping the standby database. When the communication with the standby is restored, then ARCH processes start send not shipped archived logs. ARCH process determine the last complete log file (that the standby received from the primary database) from the standby control file (via RFS process). LNS process is continuously send current redo while the ARCH processes resolve the gap in the background (Figure 1).

Note: FRS processes of standby database can receive redo changes from LNS and archived redo log files from ARCH processes of primary database same time.

4 Oracle Database 12c - Redo Gap Resolutions (Part I)

Joel Perez (Oracle ACED), Mahir M. Quluzade (OCP) & Carlos H. Y. Furushima (OCE)

Redo transport services has two options that may reduce redo gap resolution time when low performance network links are used:

Redo Transport Compression

The COMPRESSION attribute of the LOG_ARCHIVE_DEST_n parameter is used to specify that redo data be compressed before transmission to the destination.

Parallel Redo Transport Network Sessions

The MAX_CONNECTIONS attribute of the LOG_ARCHIVE_DEST_n parameter can be used to specify that more than one network session be used to send the redo needed to resolve a redo gap.

Demos

You will see some demos in this article. We are using same environments for all demos. Our demo environments are as below:

Introduction to databases:

On Primary database:

[oracle@oel62-prmdb-12c ~]$ export ORACLE_SID=prmcdb [oracle@oel62-prmdb-12c ~]$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.1.0 Production on Thu Aug 1 15:07:07 2014

Copyright (c) 1982, 2013, Oracle. All rights reserved.

Connected to:

Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production

With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select cdb, name, database_role, log_mode from v$database;

CDB NAME DATABASE_ROLE LOG_MODE

5 Oracle Database 12c - Redo Gap Resolutions (Part I)

Joel Perez (Oracle ACED), Mahir M. Quluzade (OCP) & Carlos H. Y. Furushima (OCE)

------ ------------- ------------------------ ------------

YES PRMCDB PRIMARY ARCHIVELOG

SQL> alter system switch logfile;

System altered.

SQL> select max(sequence#) from v$archived_log;

MAX(SEQUENCE#)

-------------------------

254

On Physical Standby database:

[oracle@oel62-stbdb-12c ~]$ export ORACLE_SID=stbcdb [oracle@oel62-stbdb-12c ~]$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.1.0 Production on Thu Aug 1 15:07:26 2014

Copyright (c) 1982, 2013, Oracle. All rights reserved.

Connected to:

Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production

With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> select cdb, db_unique_name, database_role, open_mode from v$database;

SQL> select process from v$managed_standby;

PROCESS

---------

ARCH

ARCH

ARCH

ARCH

RFS

RFS

MRP0

RFS

8 rows selected.

SQL> select max(sequence#) from v$archived_log;

MAX(SEQUENCE#)

--------------

6 Oracle Database 12c - Redo Gap Resolutions (Part I)

Joel Perez (Oracle ACED), Mahir M. Quluzade (OCP) & Carlos H. Y. Furushima (OCE)

254

SQL> select max(sequence#) from v$archived_log where applied='YES';

MAX(SEQUENCE#)

--------------

254

As it is seen from outputs, Data Guard configured well; also Redo Apply process is running.

Demo 1:

Automatic Gap Resolution: In this demo we try make gap and analyzing alert logs for automatic gap resolution process. In this case we are not stopped any Data Guard services. So Transport and Redo apply services running, we only stopped Network service of standby database server.

1. Stop network service on standby side.

2. More time switch log file for large archive gap on Primary database.

SQL> alter system switch logfile;

System altered.

SQL> /

System altered.

SQL> /

System altered.

SQL> select max(sequence#) from v$archived_log;

MAX(SEQUENCE#)

----------------------

258

Alert log file of primary database:

...

Thu Aug 01 15:29:45 2014

7 Oracle Database 12c - Redo Gap Resolutions (Part I)

Joel Perez (Oracle ACED), Mahir M. Quluzade (OCP) & Carlos H. Y. Furushima (OCE)

Thread 1 advanced to log sequence 256 (LGWR switch)

Current log# 1 seq# 256 mem# 0: /u01/app/oracle/oradata/prmcdb/redo01.log Thu Aug 01 15:29:46 2014

Archived Log entry 457 added for thread 1 sequence 255 ID 0x94205ccc dest 1: Thu Aug 01 15:29:47 2014

Thread 1 advanced to log sequence 257 (LGWR switch)

Current log# 2 seq# 257 mem# 0: /u01/app/oracle/oradata/prmcdb/redo02.log Thu Aug 01 15:29:48 2014

...

TNS-12543: TNS:destination host unreachable ns secondary err code: 12560

nt main err code: 513

TNS-00513: Destination host unreachable nt secondary err code: 113

nt OS err code: 0

Thu Aug 01 15:30:07 2014

Error 12543 received logging on to the standby

FAL[server, ARC0]: Error 12543 creating remote archivelog file 'stbcdb' FAL[server, ARC0]: FAL archive failed, see trace file.

ARCH: FAL archive failed. Archiver continuing Thu Aug 01 15:30:07 2014

ORACLE Instance prmcdb - Archival Error. Archiver continuing.

…

3. Now start network service on standby side.

Alert log of primary database:

…

ARCH: Detected ARCH process failure Thu Aug 01 15:35:57 2014

ARC2: STARTING ARCH PROCESSES Starting background process ARC1 Thu Aug 01 15:35:57 2014

ARC1 started with pid=24, OS id=23065 Thu Aug 01 15:35:58 2014

ARC0: Becoming the heartbeat ARCH

Thu Aug 01 15:35:58 2014

ARC1: Archival started

ARC2: STARTING ARCH PROCESSES COMPLETE

Thu Aug 01 15:36:02 2014

Thread 1 advanced to log sequence 259 (LGWR switch)

Current log# 1 seq# 259 mem# 0: /u01/app/oracle/oradata/prmcdb/redo01.log

8 Oracle Database 12c - Redo Gap Resolutions (Part I)

Joel Perez (Oracle ACED), Mahir M. Quluzade (OCP) & Carlos H. Y. Furushima (OCE)

Thu Aug 01 15:36:03 2014

Archived Log entry 463 added for thread 1 sequence 258 ID 0x94205ccc dest 1: Thu Aug 01 15:36:04 2014

ARC3: Standby redo logfile selected for thread 1 sequence 258 for destination LOG_ARCHIVE_DEST_2 Thu Aug 01 15:41:02 2014

******************************************************************

TT00: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2

******************************************************************

TT00: Standby redo logfile selected for thread 1 sequence 259 for destination LOG_ARCHIVE_DEST_2

…

Alert log of standby database:

…

Thu Aug 01 15:34:20 2014

Media Recovery Waiting for thread 1 sequence 257 Thu Aug 01 15:35:59 2014

RFS[4]: Assigned to RFS process (PID:4691)

RFS[4]: Opened log for thread 1 sequence 257 dbid 2485119180 branch 820252236 Thu Aug 01 15:35:59 2014

Archived Log entry 7 added for thread 1 sequence 257 rlc 820252236 ID 0x94205ccc dest 2: Thu Aug 01 15:35:59 2014

Media Recovery Log /u01/app/oracle/fast_recovery_area/STBCDB/archivelog/2014_08_01/o1_mf_1_257_8zngjh9g_.arc Media Recovery Waiting for thread 1 sequence 258

Thu Aug 01 15:36:03 2014

SRL log 4 needs clearing because log has not been created

RFS[4]: Selected log 5 for thread 1 sequence 258 dbid 2485119180 branch 820252236 Thu Aug 01 15:36:04 2014

Recovery of Online Redo Log: Thread 1 Group 5 Seq 258 Reading mem 0 Mem# 0: /u01/app/oracle/oradata/stbcdb/sredo02.log

Thu Aug 01 15:36:05 2014

Archived Log entry 8 added for thread 1 sequence 258 ID 0x94205ccc dest 1: Thu Aug 01 15:36:07 2014

Media Recovery Waiting for thread 1 sequence 259 Thu Aug 01 15:41:02 2014

Primary database is in MAXIMUM PERFORMANCE mode SRL log 4 needs clearing because log has not been created RFS[5]: Assigned to RFS process (PID:4710)

RFS[5]: Selected log 5 for thread 1 sequence 259 dbid 2485119180 branch 820252236 Thu Aug 01 15:41:08 2014

Recovery of Online Redo Log: Thread 1 Group 5 Seq 259 Reading mem 0 Mem# 0: /u01/app/oracle/oradata/stbcdb/sredo02.log

…

On Primary database:

SQL> select max(sequence#) from v$archived_log;

MAX(SEQUENCE#)

9 Oracle Database 12c - Redo Gap Resolutions (Part I)

Joel Perez (Oracle ACED), Mahir M. Quluzade (OCP) & Carlos H. Y. Furushima (OCE)

--------------------------

258

On Standby database:

SQL> select max(sequence#) from v$archived_log;

MAX(SEQUENCE#)

--------------------------------

258

SQL> select max(sequence#) from v$archived_log where applied=’YES’;

MAX(SEQUENCE#)

--------------------------------

258

As it is seen from outputs, when network service started on standby server, transport service of

Data Guard sent “not shipped” redo logs to standby database. Data Guard processes resolved gap automatically.

Redo Gap Resolution using FAL.

FAL is Data Guard’s capability of Fetch Archive Log. It is only used on a physical standby database. When a physical standby database finds a problem of missing or corrupted archive log file, it can go and fetch it from one of the FAL Server (primary, cascading standby and FAR SYNC instance) in the Data Guard configuration. This is also referred as reactive gap resolution. In this case RFS processes of standby database using FAL_SERVER parameter for request required archived redo logs for archive gap resolution. FAL_SERVER parameter is defined as a list of TNS entries that exist on the standby server and point to the primary and any of the standby databases or FAR Sync service. As you know, FAL_CLIENT is already deprecated parameter Oracle Database 11g R2.

Note: Redo Apply process must be running for detection archive Gap on standby database.

Corruption detection/protection

Redo transmitted directly from SGA (redo buffer) by ASYNC or SYNC is completely isolated from physical I/O corruptions. Redo Apply provide data protection by preventing physical corruptions that can occur at the primary database from being applied to a standby database. Transport service and Redo Apply service automatic detection corruptions on redo logs. Transport Service not transport any corrupted archived log, redo apply cannot apply any corrupted archived redo log to standby database. In other words, transport service and redo apply service ensuring quality of data transported and apply to standby database.

10 Oracle Database 12c - Redo Gap Resolutions (Part I)

Joel Perez (Oracle ACED), Mahir M. Quluzade (OCP) & Carlos H. Y. Furushima (OCE)