Real Application Clusters (RAC) or Oracle Parallel Server (OPS)

What is RAC / OPS ?

Real Application Clusters (RAC) - previously called Oracle Parallel Server (OPS)
Real Application Clusters (RAC) is a database clustering solution.
It is provided as an option to Oracle Enterprise Edition, but is part of Oracle Standad Edition.

RAC allows applications to exploit cluster availability, scalability and performance with no (or minimal) application modifications.
A normal Oracle installation consists of a single Oracle instance that accesses a database on the same computer system.
In contrast, RAC allows multiple instances on different computer systems (nodes in a cluster) to access the same database files simultaneously.

RAC is mainly used for scalability and high availability reasons, but can also provide improved performance by using CPU resources across multiple nodes of the cluster.

However, one needs to be carefull, performance can be worse if it is not planned and the application is not correctly partitioned.
With RAC, multiple instances mount the same database files.
Communication between instances is managed by the Distributed Lock Manager (DLM).

To address the possibility of two or more instances attempting to modify the same information simultaneously, Oracle uses up to ten additional background processes, named LCK0 through LCK9, to lock the resources in use by these instances.

RAC is normally used on Linux Clusters, Windows Clusters, DEC VAX Clusters, UNIX Clusters, IBM's SYSPLEX etc.

Some history: In Oracle 8i and below, RAC was called Parallel Server (OPS).
Oracle branded it RAC when they released Oracle 9i.

What are the benefits for running RAC

Running Oracle in RAC mode can benefit you in the following ways:

High availability
If some nodes fail, the remainder of the nodes will still be available for processing requests.
Failover support is available from Oracle 8.0 with Oracle's Transparent Application Failover (TAF) technology.

Speedup (increased transaction response time)
RAC normally adds some overhead. However, for some DSS applications one can achieve increased performace by running operations in parallel (mainly for DSS applications).

Scale-up (increased transaction volume)
RAC can be used to provide increased application scalability (mainly used for OLTP applications).

Can any application be deployed on RAC?

Most applications can be deployed on RAC without any modifications and still scale (almost) linearly.

However, applications with 'hot' rows (the same row being accessed by processes on different nodes) will not work well. This is because data blocks will constantly be moved from one Oracle Instance to another. In such cases the application needs to be partitioned based on function or data to eliminate
contention.

Do you need special hardware to run RAC ?

RAC requires the following hardware components:

  • A dedicated network interconnect - might be as simple as a fast network connection between nodes; and
  • A shared disk subsystem. Example systems that can be used with RAC:

  • Windows NT Clusters
  • Linux Clusters
  • Unix Clusters like SUN PDB (Parallel DB).
  • IBM z/OS in SYSPLEX

How does one convert a Single Instance Database to RAC ?

Oracle 10gR2 introduces a utiliy called rconfig (located in $ORACLE_HOME/bin) that will convert a single instance database to a RAC database.

$ cp $ORACLE_HOME/assistants/rconfig/sampleXMLs/ConvertToRAC.xml
racconv.xml

$ vi racconv.xml

$ rconfig racconv.xml

One can also use dbca and enterprise manager to convert the database to RAC mode.
For prior releases, follow these steps:
  • Shut Down your Database : 
    SQL> CONNECT SYS AS SYSDBA
    SQL> SHUTDOWN NORMAL

  • Enable RAC : On Unix this is done by relinking the Oracle software.

  • Make the software available on all computer systems that will run RAC : This can be
    done by copying the software to all systems or to a shared clustered file system.

  • Each instance requires its own set of Redo Log Files. Create additional log files:
     SQL>CONNECT SYS AS SYSBDA
     SQL>STARTUP EXCLUSIVE
     SQL>ALTER DATABASE ADD LOGFILE THREAD 2;
        SQL>GROUP G4 ('RAW_FILE1') SIZE 500k,
     SQL>GROUP G5 ('RAW_FILE2') SIZE 500k,
     SQL>GROUP G6 ('RAW_FILE3') SIZE 500k;
     SQL>ALTER DATABASE ENABLE PUBLIC THREAD 2;

  • Each instance requires its own set of rollback/undo segments. Add Rollback Segments for New Nodes :
     UNDO_MANAGEMENT = auto
     UNDO_TABLESPACE = undots2
 or...

  SQL>CREATE ROLLBACK SEGMENT RB2 TABLESPACE RBS;

  • Edit the INIT.ORA files and number the instances 1, 2,... :
         CLUSTER_DATABASE = TRUE         (PARALLEL_SERVER = TRUE prior to Oracle9i).          INSTANCE_NUMBER = 1
     THREAD = 1
     UNDO_TABLESPACE = undots1  (or
     ROLLBACK_SEGMENTS if you use UNDO_MANAGEMENT=manual)
     #Include %T for the thread in the LOG_ARCHIVE_FORMAT string.
     # Set LM_PROCS to the number of nodes * PROCESSES
     # etc....

  • Create the dictionary views needed for RAC: Run catclust.sql (previously called catparr.sql):

SQL> START ?/rdbms/admin/catclust.sql
  • On all the computer systems, startup the instances:
     SQL>CONNECT / as SYSDBA
     SQL>STARTUP ;

How does one Stop and Start RAC Instances ?

There are no difference between the way you start a normal database and RAC database, except that a RAC database needs to be started from multiple nodes.

The CLUSTER_DATABASE=TRUE (PARALLEL_SERVER=TRUE) parameter needs to be set before a database can be started in cluster mode.

In Oracle 10g one can use the srvctl utility to start instances and listener across the cluster from a single node. Here are some examples:

$ srvctl status database -d RACDB
$ srvctl start database -d RACDB
$ srvctl start instance -d RACDB -i RACDB1
$ srvctl start instance -d RACDB -i RACDB2
$ srvctl stop database -d RACDB
$ srvctl start asm -n node2

Before Oracle 8.0, use the following command sequence from each node (using the old server manager):


SVRMGR> connect INTERNAL
SVRMGR> set retries 5
SVRMGR> startup parallel retry      .. or SVRMGR> startup shared

You can also use the SET INSTANCE instanceN command to switch between instances (if defined in TNSNAMES.ORA).

How can I test if a Database is running in RAC mode ?

Use the DBMS_UTILITY package to determine if a database is running in RAC mode or not:
BEGIN
  IF dbms_utility.is_cluster_database THEN         dbms_output.put_line('Running in SHARED/RAC mode.');
  ELSE         dbms_output.put_line('Running in EXCLUSIVE mode.');
  END IF;
END;
/

For Oracle8i and prior releases:

BEGIN
  IF dbms_utility.is_parallel_server THEN         dbms_output.put_line('Running in SHARED/PARALLEL mode.');
  ELSE
    dbms_output.put_line('Running in EXCLUSIVE mode.');
  END IF;
END;
/

In SQL*Plus :

SQL>show parameter parallel_server

If the value of parallel_server is FALSE then database is not running in Parallel Mode.

How many PCM locks are allocated per Instance/ File ?

PCM Locks per instance : 
select count(*) "Number of hashed PCM locks"
from v$lock_element where  bitand(flags, 4) != 0
/


select count(*) "Number of fine grain PCM locks"
from v$lock_element where  bitand(flags, 4) = 0
/

PCM locks per file with the ping rate on them :

col file_name format a29
col tablespace format a12
col blocking format 9999999
col nlocks format 99999
col start_lk format 9999999
select l.file_id || ' ' || l.file_name file_name,    l.ts_name "TABLESPACE",start_lk,nlocks, blocking, frequency "PING COUNT"
  from   sys.file_ping p, sys.file_lock l
 where  l.file_id = p.file_id
 order  by l.file_id
/

How can I keep track of active Instances ?

You can keep track of active RAC instances by executing one of the following queries:

SELECT * FROM SYS.V_$ACTIVE_INSTANCES;
SELECT * FROM SYS.V_$THREAD;

To list the active instances from PL/SQL, use DBMS_UTILITY.ACTIVE_INSTANCES().

What is Pinging and why is it so bad ?

Starting with Oracle 9i, RAC can transfer blocks from one instance to another across the intreconnect (cache fusion). This method is much faster than the old "pinging" method, where one instance had to write the block to disk before another instance could read it.

Orcle 8i and below:

Pinging is the process whereby one instance  requests another to write a set of blocks from its SGA to disk so it can obtain it in exclusive mode.
This method of moving data blocks from one instance's SGA to another is extremely slow. The challenge of tuning RAC/OPS is to minimize pinging activity.

How does one monitor PCM Lock activity ?

NOTE: This FAQ is not applicable to Oracle 9i RAC and later versions. PCM (Parallel Cache Managemen) locks control the mapping of pre-release 9.0.1 parallel cache management (PCM) locks to data files.

Summary of PCM lock activity for the current instance:

select * from sys.v$lock_activity;

PCM lock activity per database object:

col table format a40

select file#, kind||' '||username||'.'||name "TABLE", sum(xnc) pings
from sys.v$false_ping p, sys.dba_users u
where u.user_id = p.owner#
group  by file#, kind||' '||username||'.'||name, xnc
order  by xnc desc;

PCM locks per instance:


select count(*) "Number of hashed PCM locks"
from v$lock_element where  bitand(flags, 4) != 0
/
select count(*) "Number of fine grain PCM locks"
from v$lock_element where  bitand(flags, 4) = 0
/

PCM locks per data file:
The following statement shows how many PCM locks are allocated per file and what the ping rate on those files are:

col file_name format a29
col tablespace format a12
col blocking  format 9999999
col nlocks format 99999
col start_lk format 9999999

select l.file_id || ' ' || l.file_name file_name,l.ts_name "TABLESPACE",
     start_lk, nlocks, blocking, frequency "PING COUNT"
from  sys.file_ping p, sys.file_lock l
where l.file_id = p.file_id
order  by l.file_id
/

Tracing DLM activity

Look at this example:

SQLPLUS> REM Dump Parallel Server DLM locks
SQLPLUS> oradebug lkdebug -a convlock
SQLPLUS> oradebug lkdebug -a convres
SQLPLUS> oradebug lkdebug -r <resource
handle> (i.e 0x8066d338 from convres dump)

Execute the "ORADEBUG HELP" command from sqlplus (or svrmgrl) for a list of valid commands.

How does one set up a Common SQL*Net connect string for all RAC Instances ?

Define all Oracle System Identifiers (SIDs) to be the same on all nodes of the system. If you already have your SIDs defined, you can rename them by:

  • Shutting down all  instances of the database;
  • Change the ORACLE_SID  environment variable to the common name you have chosen;
  • Copy the current initOLDSID.ora file to initCOMMON.ora;
  • Restart the Oracle instances.
This is a sample TNSNAMES entry to connect to one of two instances in a RAC cluster:

rprd = (DESCRIPTION= 
          (ADDRESS_LIST= 
              (ADDRESS=
                     (PROTOCOL=tcp)
                     (HOST=node_1)
                     (PORT=1526)
               )
                                 (ADDRESS=
                     (PROTOCOL=tcp)
                     (HOST=node_2)
                     (PORT=1526)                                 )
           )
                     (CONNECT_DATA=(SID=rprd)
      )
)
Posted by
Labels:

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)