Skip Navigation Links
Exit Print View
| Oracle Solaris Cluster Geographic Edition Data Replication Guide for EMC Symmetrix Remote Data Facility

Search Scope:

- This Document
- Entire Library

## Initial Configuration of SRDF Software

This section describes the steps you need to perform to configure SRDF software on the primary and secondary clusters. It also includes information about the preconditions for creating SRDF protection groups.

- Setting the Path to the SRDF SYMCLI

- Configuring Data Replication With SRDF Software on the Primary Cluster

- Configuring Data Replication With SRDF Software on the Secondary Cluster

- Configuring the Other Entities on the Secondary Cluster

Initial configuration of the primary and secondary clusters includes the following:

- Configuring an SRDF device group, `devgroup1`, with the required number of disks

- If using a raw-disk device group, configuring a raw-disk group, `rawdg`

- If using ZFS, configuring a zpool

- If using Veritas Volume Manager (VxVM):

  - Configuring the VxVM disk group, `dg1`

  - Configuring the VxVM volume, `vol1`

  - Configuring the Oracle Solaris Cluster device group for the VxVM volume

- Configuring the file system, which includes creating the file system, creating mount points, and adding entries to the `/etc/vfstab` file

- Creating an application resource group, `apprg1`, which contains an `HAStoragePlus` resource

Geographic Edition software supports the hardware configurations that are supported by the Oracle Solaris Cluster software. Contact your Oracle service representative for information about current supported Oracle Solaris Cluster configurations.

The Geographic Edition software installation process on a single-node cluster creates the `/var/cluster/rgm/physnode_affinities` file. Its existence causes positive and negative resource group affinities to be enforced at the level of the physical node, as they are in all multi-node clusters. Without this file, a single-node cluster uses resource group affinities at the level of the zone-node. The absence of this file can cause the malfunction of clustered applications, so do not remove it unless you clearly understand the potential consequences of its removal.

Table 1-2 Task Map: Steps in Configuring SRDF Data Replication for Geographic Edition Systems

Task | Instructions
--- | ---
Setting the path to the correct version of SRDF | Setting the Path to the SRDF SYMCLI
Configuring the SRDF device group | Setting Up SRDF Device Groups
Configuring a raw-disk device group | How to Set Up Raw-Disk Device Groups for Geographic Edition Systems
Configuring a VxVM device group | Perform both of the following procedures: How to Configure VxVM Volumes for Use With SRDF Replication How to Configure the Oracle Solaris Cluster Device Group for a VxVM Disk Group
Configuring the file system and creating the application resource group | How to Configure a Highly Available Local File System for SRDF Replication

### Setting the Path to the SRDF SYMCLI

To ensure that the Geographic Edition infrastructure uses a current, supported version of SRDF, you must manual set the location of the correct SYMCLI on all nodes of all clusters in the partnership.

### How to Set the Path to the SRDF SYMCLI

Perform this procedure on each cluster node, in each partner cluster.

- On each node of the cluster, create a symbolic link to the SYMCLI of the SRDF software to be used by the Geographic Edition infrastructure.

      # ln -s /opt/emc/SYMCLI/srdfversion /opt/emc/SYMCLI/scgeo_default

  If `/opt/emc/SYMCLI/scgeo_default` is not found, Geographic Edition software uses the SYMCLI of the latest version of SRDF software that is currently installed on the node and that is supported by Geographic Edition software.

### Configuring Data Replication With SRDF Software on the Primary Cluster

This section describes the steps you must perform on the primary cluster before you can configure SRDF data replication with Geographic Edition software.

- Setting Up SRDF Device Groups

- How to Set Up Raw-Disk Device Groups for Geographic Edition Systems

- How to Configure VxVM Volumes for Use With SRDF Replication

- How to Configure the Oracle Solaris Cluster Device Group for a VxVM Disk Group

- How to Configure a Highly Available Local File System with ZFS for SRDF Replication

- How to Configure a Highly Available Local File System for SRDF Replication

#### Setting Up SRDF Device Groups

SRDF devices are configured in pairs. The mirroring relationship between the pairs becomes operational as soon as the SRDF links are online. If you have dynamic SRDF available, you have the capability to change relationships between R1 and R2 volumes in your device pairings on the fly without requiring a BIN file configuration change.

Note - Do not configure a replicated volume as a quorum device. Locate any quorum devices on a shared, unreplicated volume or use a quorum server.

The EMC Symmetrix database file on each host stores configuration information about the EMC Symmetrix units attached to the host. The EMC Symmetrix global memory stores information about the pair state of operating EMC SRDF devices.

EMC SRDF device groups are the entities that you add to Geographic Edition protection groups to enable the Geographic Edition software to manage EMC Symmetrix pairs.

The SRDF device group can hold one of two types of devices:

- RDF1 source device, which acts as the primary

- RDF2 target device, which acts as the secondary

As a result, you can create two types of SRDF device group, RDF1 and RDF2. An SRDF device can be moved to another device group only if the source and destination groups are of the same group type.

You can create RDF1 device groups on a host attached to the EMC Symmetrix software that contains the RDF1 devices. You can create RDF2 device groups on a host attached to the EMC Symmetrix software that contains the RDF2 devices. You can perform the same SRDF operations from the primary or secondary cluster, using the device group that was built on that side.

When you add remote data facility devices to a device group, all of the devices must adhere to the following restrictions:

- The device must be an SRDF device.

- The device must be either an RDF1 or RDF2 type device, as specified by the device group type.

- The device must belong to the same SRDF group number.

- The SRDF device group configuration must be the same on all nodes of both the primary and secondary clusters. For example, if you have a device group `DG1`, which is configured as `RDF1`, on `node1` of `clusterA`, then `node2` of `clusterA` should also have a device group called `DG1` with the same disk set. Also, `clusterB` should have an SRDF device group called `DG1`, which is configured as `RDF2`, defined on all nodes.

##### Checking the Configuration of SRDF Devices

Before adding SRDF devices to a device group, use the `symrdf list` command to list the EMC Symmetrix devices configured on the EMC Symmetrix units attached to your host.

    # symrdf list

By default, the command displays devices by their EMC Symmetrix device name, a hexadecimal number that the EMC Symmetrix software assigns to each physical device. To display devices by their physical host name, use the `pd` argument with the `symrdf` command.

    # symrdf list pd

##### Creating an RDF1 Device Group

The following steps create a device group of type RDF1 and add an RDF1 EMC Symmetrix device to the group.

1. Create a device group named `devgroup1`.

       phys-paris-1# symdg create devgroup1 -type rdf1

2. Add an RDF1 device, with the EMC Symmetrix device name of 085, to the device group on the EMC Symmetrix storage unit identified by the number 000000003264.

   A default logical name of the form `DEV001` is assigned to the RDF1 device.

       phys-paris-1# symld -g devgroup1 -sid 3264 add dev 085

### How to Set Up Raw-Disk Device Groups for Geographic Edition Systems

Geographic Edition supports the use of raw-disk device groups in addition to various volume managers. When you initially configure Oracle Solaris Cluster, device groups are automatically configured for each raw device in the cluster. Use this procedure to reconfigure these automatically created device groups for use with Geographic Edition.

1. For the devices that you want to use, unconfigure the predefined device groups.

   The following commands remove the predefined device groups for `d7` and `d8`.

       phys-paris-1# cldevicegroup disable dsk/d7 dsk/d8
       phys-paris-1# cldevicegroup offline dsk/d7 dsk/d8
       phys-paris-1# cldevicegroup delete dsk/d7 dsk/d8

2. Create the new raw-disk device group, including the desired devices.

   Ensure that the new DID does not contain any slashes. The following command creates a global device group, `rawdg`, which contains `d7` and `d8`.

       phys-paris-1# cldevicegroup create -n phys-paris-1,phys-paris-2 \
       -t rawdisk -d d7,d8 rawdg

Example 1-1 Configuring a Raw-Disk Device Group

This example illustrates configuring the device group on the primary cluster, configuring the same device group on the partner cluster, and adding the group to an EMC Symmetrix protection group. Geographic Edition requires that the same Oracle Solaris Cluster device group, in this example `rawdg`, exists on both clusters..

    Remove the automatically created device groups from the primary cluster.
    phys-paris-1# cldevicegroup disable dsk/d7 dsk/d8
    phys-paris-1# cldevicegroup offline dsk/d7 dsk/d8
    phys-paris-1# cldevicegroup delete dsk/d7 dsk/d8

    Create the raw-disk device group on the primary cluster.
    phys-paris-1# cldevicegroup create -n phys-paris-1,phys-paris-2 \
    -t rawdisk -d d7,d8 rawdg

    Remove the automatically created device groups from the partner cluster.
    phys-newyork-1# cldevicegroup disable dsk/d5 dsk/d6
    phys-newyork-1# cldevicegroup offline dsk/d5 dsk/d6
    phys-newyork-1# cldevicegroup delete dsk/d5 dsk/d6

    Create the raw-disk device group on the partner cluster.
    phys-newyork-1# cldevicegroup create -n phys-newyork-1,phys-newyork-2 \
    -t rawdisk -d d5,d6 rawdg

    Add the raw-disk device group to the protection group rawpg.
    phys-paris-1# geopg create -d srdf -p Nodelist=phys-paris1,phys-paris-2 \
    -o Primary -p cluster_dgs=rawdg -s paris-newyork-ps rawpg

Next Steps

Create a raw-disk device group on the partner cluster with the same name as the one you created here. See How to Replicate the Configuration Information From the Primary Cluster, When Using Raw-Disk Device Groups for the instructions about this task.

After you have configured the device group on both clusters, you can use the device group name wherever one is required in Geographic Edition commands such as `geopg`.

### How to Configure VxVM Volumes for Use With SRDF Replication

SRDF data replication is supported with VxVM volumes and raw-disk device groups. If you are using VxVM, you must configure VxVM volumes on the disks you selected for your SRDF device group.

1. On `cluster-paris`, create VxVM disk groups on shared disks that will be replicated to the partner cluster `cluster-newyork`.

   For example, the `d1` and `d2` disks are configured as part of a VxVM disk group which is called `dg1`, by using commands such as `vxdiskadm` and `vxdg`. These disks are the ones that will be replicated to the partner cluster.

2. After configuration is complete, verify that the disk group was created by using the `vxdg list` command.

   This command should list `dg1` as a disk group.

3. Create the VxVM volume.

   For example, a volume that is called `vol1` is created in the `dg1` disk group. The appropriate VxVM commands, such as `vxassist`, are used to configure the volume.

Next Steps

Perform the steps in How to Configure the Oracle Solaris Cluster Device Group for a VxVM Disk Group to configure the VxVM volume as an Oracle Solaris Cluster device group.

### How to Configure the Oracle Solaris Cluster Device Group for a VxVM Disk Group

1. Register the VxVM disk group that you configured in the previous procedure with Oracle Solaris Cluster.

   Use the Oracle Solaris Cluster commands `clsetup` or `cldevice` and `cldevicegroup`.

   For more information about these commands, refer to the `clsetup`(1CL) man page or the `cldevice`(1CL) and `cldevicegroup`(1CL) man pages.

2. Synchronize the VxVM configuration with Oracle Solaris Cluster software, again by using the `clsetup` or `cldevice` and `cldevicegroup` commands.
3. After configuration is complete, verify the disk group registration.

       phys-paris-1# cldevicegroup show devicegroupname

   The VxVM disk group, `dg1`, should be displayed in the output.

   For more information about the `cldevicegroup` command, see the `cldevicegroup`(1CL) man page.

### How to Configure a Highly Available Local File System with ZFS for SRDF Replication

Follow this procedure to configure a highly available local file system that uses a ZFS zpool.

Before You Begin

Ensure that the Oracle Solaris Cluster application resource group has already been configured.

Observe the following requirements and restrictions for ZFS:

- Mirrored and unmirrored ZFS zpools are supported.

- ZFS zpool spares are not supported with storage-based replication in a Geographic Edition configuration. The information about the spare that is stored in the zpool results in the zpool being incompatible with the remote system after it has been replicated.

- ZFS can be used with either Synchronous or Asynchronous mode. If you use asynchronous mode, ensure that SRDF is configured to preserve write ordering, even after a rolling failure.

1. Create a ZFS zpool.

       # zpool create appdataz mirror cNtXdY cNtAdB

   `create appdataz`

     Specifies the name of the zpool to create.

   `mirror cNtXdY cNtAdB`

     Specifies the LUNs to replicate with SRDF.

2. Add an HAStoragePlus resource to the application resource group, app-rg.

       # clresource create -g app-rg \
       -t HAStoragePlus \
       -p zpools=appdataz \
       hasp4appdataz

   `-g app-rg`

     Specifies the application resource group.

   `-p zpools=appdataz`

     Specifies the zpool

   `hasp4appdataz`

     Specifies the name of the HAStoragePlus resource to create.

Example 1-2 Configuring a Highly Available Local File System With ZFS for SRDF Replication

This example creates a locally mounted file system, with `HAStoragePlus` using a ZFS zpool. The file system created in this example is mounted locally every time the resource is brought online.

This example assumes that the `app-rg1` resource group already exists

1. Create the zpool `appdata1`.

   # `zpool create appdata1 mirror c6t6006048000018790002353594D313137d0 c6t6006048000018790002353594D313143d0`

2. Add the HAStoragePlus resource `hasp4appdata-rs` to the application resource group `app-rg1`.

       # clresource create -g app-rg1 \
       -t HAStoragePlus \
       -p zpools=appdata1 \
       hasp4appdata-rs

### How to Configure a Highly Available Local File System for SRDF Replication

Note - If you want to create a highly available local file system that uses a ZFS zpool, do not perform this procedure. Instead, go to How to Configure a Highly Available Local File System with ZFS for SRDF Replication.

Before You Begin

Before you configure the file system on `cluster-paris`, ensure that the Oracle Solaris Cluster entities you require, such as application resource groups, device groups, and volumes, have already been configured.

1. Create the required file system on the `vol1` volume at the command line.
2. On each node in the cluster, create mount points for the file system you just created.

       # mkdir -p /mounts/sample

   /mounts/sample

     Your mount point.

3. Add an entry to the `/etc/vfstab` file that contains information such as the mount location.

   Whether the file system is to be mounted locally or globally depends on various factors, such as your performance requirements, or the type of application resource group you are using.

   Note - You must set the `mount at boot` field in this file to `no`. This value prevents the file system from mounting on the secondary cluster at cluster startup. Instead, the Oracle Solaris Cluster software and the Geographic Edition framework handle mounting the file system by using the `HAStoragePlus` resource when the application is brought online on the primary cluster.

4. Add the `HAStoragePlus` resource to the application resource group, `apprg1`.

   Adding the resource to the application resource group ensures that the necessary file systems are mounted before the application is brought online.

   For more information about the `HAStoragePlus` resource type, refer to Enabling Highly Available Local File Systems in Oracle Solaris Cluster Data Services Planning and Administration Guide.

5. Verify that the device group was registered properly.

   The following command should display the device group `dg1`.

       phys-paris-1# cldevicegroup show dg1

Example 1-3 Configuring a Highly Available Local File System

This example creates a locally mounted file system, with `HAStoragePlus`. The file system created in this example is mounted locally every time the resource is brought online.

This example assumes that the following already exist:

- The `apprg1` resource group

- The `dg1` VxVM device group

- The `vol1` VxVM volume

1. Create a UNIX file system (UFS).

       phys-paris-1# newfs dev/vx/dsk/dg1/vol1

2. On each node in the cluster, create mount points for the file system.

       phys-paris-1# mkdir -p /mounts/sample
       phys-paris-2# mkdir -p /mounts/sample

3. Create mount points on all cluster `paris` nodes.

       phys-paris-1# mkdir /mounts/sample

4. Add the following entry to the `/etc/vfstab` file:

       phys-paris-1# /dev/vs/dsk/dg1/vol1 /dev/vx/rdsk/dg1/vol1 /mounts/sample \
       ufs 2 no logging

5. Add the `HAStoragePlus` resource type.

       phys-paris-1# clresource create -g apprg1 -t SUNW.HAStoragePlus \
       -p FilesystemMountPoints=/mounts/sample -p Affinityon=TRUE \
       -p GlobalDevicePaths=dg1 rs-hasp

### Configuring Data Replication With SRDF Software on the Secondary Cluster

This section describes the steps you must complete on the secondary cluster before you can configure SRDF data replication in Geographic Edition software.

### How to Create the RDF2 Device Group on the Secondary Cluster

Before You Begin

Before you can issue the SRDF commands on the secondary cluster, you need to create a RDF2 type device group on the secondary cluster that contains the same definitions as the RDF1 device group.

Note - Do not configure a replicated volume as a quorum device. Locate any quorum devices on a shared, unreplicated volume or use a quorum server.

1. Use the `symdg export` command to create a text file, `devgroup1.txt`, that contains the RDF1 group definitions.

       phys-paris-1# symdg export devgroup -f devgroup.txt -rdf

2. Use the `rcp` or `ftp` command to transfer the file to the secondary cluster.

       phys-paris-1# rcp devgroup1.txt phys-newyork-1:/.
       phys-paris-1# rcp devgroup1.txt phys-newyork-2:/.

3. On the secondary cluster, use the `symdg import` command to create the RDF2 device group by using the definitions from the text file.

   Run the following command on each node in the `newyork` cluster.

       # symdg import devgroup1 -f devgroup1.txt

       Adding standard device 054 as DEV001...
       Adding standard device 055 as DEV002...

### Configuring the Other Entities on the Secondary Cluster

Next, you need to configure any volume manager, the Oracle Solaris Cluster device groups, and the highly available cluster file system. This process is slightly different depending on whether you are using VxVM or raw-disk device groups. The following procedures provide instructions:

- How to Replicate the VxVM Configuration Information From the Primary Cluster

- How to Replicate the Configuration Information From the Primary Cluster, When Using Raw-Disk Device Groups

- How to Replicate the Configuration Information From the Primary Cluster, When Using ZFS zpools

### How to Replicate the VxVM Configuration Information From the Primary Cluster

1. Start replication for the `devgroup1` device group.

       phys-paris-1# symrdf -g devgroup1 -noprompt establish

       An RDF 'Incremental Establish' operation execution is in progress for device group
       'devgroup1'. Please wait...
       Write Disable device(s) on RA at target (R2)..............Done.
       Suspend RDF link(s).......................................Done.
       Mark target (R2) devices to refresh from source (R1)......Started.
       Device: 054 ............................................. Marked.
       Mark target (R2) devices to refresh from source (R1)......Done.
       Suspend RDF link(s).......................................Done.
       Merge device track tables between source and target.......Started.
       Device: 09C ............................................. Merged.
       Merge device track tables between source and target.......Done.
       Resume RDF link(s)........................................Done.

       The RDF 'Incremental Establish' operation successfully initiated for device group
       'devgroup1'.

2. Confirm that the state of the SRDF pair is synchronized.

       phys-newyork-1# symrdf -g devgroup1 verify

       All devices in the RDF group 'devgroup1' are in the 'Synchronized' state.

3. Split the pair by using the `symrdf split` command.

       phys-paris-1# symrdf -g devgroup1 -noprompt split

       An RDF 'Split' operation execution is in progress for device group 'devgroup1'.
       Please wait...

       Suspend RDF link(s).......................................Done.
       Read/Write Enable device(s) on RA at target (R2)..........Done.
       The RDF 'Split' operation device group 'devgroup1'.

4. Enable all the volumes to be scanned.

       phys-newyork-1# vxdctl enable

5. Import the VxVM disk group, `dg1`.

       phys-newyork-1# vxdg -C import dg1

6. Verify that the VxVM disk group was successfully imported.

       phys-newyork-1# vxdg list

7. Enable the VxVM volume.

       phys-newyork-1# /usr/sbin/vxrecover -g dg1 -s -b

8. Verify that the VxVM volumes are recognized and enabled.

       phys-newyork-1# vxprint

9. Create the VxVM disk group, `dg1`, in Oracle Solaris Cluster software.

       phys-newyork-1# cldevicegroup create -n phys-newyork-1,phys-newyork-2 \
       -t vxvm dg1

10. Add an entry to the `/etc/vfstab` file on `phys-newyork-1`.

        /dev/vx/dsk/dg1/vol1 /dev/vx/rdsk/dg1/vol1 /mounts/sample ufs 2 no logging

11. Create a mount directory on `newyork`.

        phys-newyork-1# mkdir -p /mounts/sample
        phys-newyork-2# mkdir -p /mounts/sample

12. Create an application resource group, `apprg1`, by using the `clresourcegroup` command.

        phys-newyork-1# clresourcegroup create apprg1

13. Create the `HAStoragePlus` resource in `apprg1`.

        phys-newyork-1# clresource create -g apprg1 -t SUNW.HAStoragePlus \
        -p FilesystemMountPoints=/mounts/sample -p AffinityOn=TRUE \
        -p GlobalDevicePaths=dg1 rs-hasp

    This `HAStoragePlus` resource is required for Geographic Edition systems because the software relies on the resource to bring the device groups and file systems online when the protection group starts on the primary cluster.

14. Confirm that the application resource group is correctly configured by bringing it online and taking it offline again.

        phys-newyork-1# clresourcegroup online -emM apprg1
        phs-newyork-1# clresourcegroup offline apprg1

15. Unmount the file system.

        phys-newyork-1# umount /mounts/sample

16. Take the Oracle Solaris Cluster device group offline.

        phys-newyork-1# cldevicegroup offline dg1

17. Verify that the VxVM disk group was deported.

        phys-newyork-1# vxdg list

18. Reestablish the SRDF pair.

        phys-newyork-1# symrdf -g devgroup1 -noprompt establish

    Initial configuration on the secondary cluster is now complete.

### How to Replicate the Configuration Information From the Primary Cluster, When Using Raw-Disk Device Groups

1. On the primary cluster, start replication for the `devgroup1` device group.

       phys-paris-1# symrdf -g devgroup1 -noprompt establish

       An RDF 'Incremental Establish' operation execution is in progress for device group
       'devgroup1'. Please wait...
       Write Disable device(s) on RA at target (R2)..............Done.
       Suspend RDF link(s).......................................Done.
       Mark target (R2) devices to refresh from source (R1)......Started.
       Device: 054 ............................................. Marked.
       Mark target (R2) devices to refresh from source (R1)......Done.
       Suspend RDF link(s).......................................Done.
       Merge device track tables between source and target.......Started.
       Device: 09C ............................................. Merged.
       Merge device track tables between source and target.......Done.
       Resume RDF link(s)........................................Done.

       The RDF 'Incremental Establish' operation successfully initiated for device group
       'devgroup1'.

2. On the primary cluster, confirm that the state of the SRDF pair is synchronized.

       phys-newyork-1# symrdf -g devgroup1 verify

       All devices in the RDF group 'devgroup1' are in the 'Synchronized' state.

3. On the primary cluster, split the pair by using the `symrdf split` command.

       phys-paris-1# symrdf -g devgroup1 -noprompt split

       An RDF 'Split' operation execution is in progress for device group 'devgroup1'.
       Please wait...

       Suspend RDF link(s).......................................Done.
       Read/Write Enable device(s) on RA at target (R2)..........Done.
       The RDF 'Split' operation device group 'devgroup1'.

4. Map the EMC disk drive to the corresponding DID numbers.

   You use these mappings when you create the raw-disk device group.

   1. Use the `symrdf` command to find devices in the SRDF device group.

          phys-paris-1# symrdf -g devgroup1 query
          …
          DEV001  00DD RW       0        3 NR 00DD RW       0        0 S..   Split
          DEV002  00DE RW       0        3 NR 00DE RW       0        0 S..   Split
          …

   2. Use the `powermt` command to write detailed information about all devices into a temporary file.

          phys-paris-1# /etc/powermt display dev=all > /tmp/file

   3. Open the temporary file and look for the ctd label that applies to the appropriate device.

          Logical device ID=00DD
          state=alive; policy=BasicFailover; priority=0; queued-IOs=0
          ==============================================================================
          ---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
          ### HW Path                 I/O Paths    Interf.   Mode    State  Q-IOs Errors
          ==============================================================================
          3073 pci@1d/SUNW,qlc@1         c6t5006048ACCC81DD0d18s0 FA  1dA   active  alive
               0      0
          3075 pci@1d/SUNW,qlc@2         c8t5006048ACCC81DEFd18s0 FA 16cB   unlic   alive
               0      0

      In this example, you see that the logical device ID `00DD` maps to the ctd label `c6t5006048ACCC81DD0d18`.

   4. Once you know the ctd label, use the `cldevice` command to see more information about that device.

          phys-paris-1# cldevice show c6t5006048ACCC81DD0d18

          === DID Device Instances ===

          DID Device Name:                                /dev/did/rdsk/d5
            Full Device Path:
          pemc3:/dev/rdsk/c8t5006048ACCC81DEFd18
            Full Device Path:
          pemc3:/dev/rdsk/c6t5006048ACCC81DD0d18
            Full Device Path:
          pemc4:/dev/rdsk/c6t5006048ACCC81DD0d18
            Full Device Path:
          pemc4:/dev/rdsk/c8t5006048ACCC81DEFd18
            Replication:                                     none
            default_fencing:                                 global

      In this example, you see that the ctd label `c6t5006048ACCC81DD0d18` maps to `/dev/did/rdsk/d5`.

   5. Repeat steps as needed for each of the disks in the device group and on each cluster.
5. Create a raw-disk device group on the partner cluster.

   Use the same device group name as you used for the one on the primary cluster.

   In the following command, the `newyork` cluster is the partner of the `paris` cluster.

       phys-newyork-1# cldevicegroup disable dsk/d5 dsk/d6
       phys-newyork-1# cldevicegroup offline dsk/d5 dsk/d6
       phys-newyork-1# cldevicegroup delete dsk/d5 dsk/d6
       phys-newyork-1# cldevicegroup create -n phys-newyork-1,phys-newyork-2 \
       -t rawdisk -d d5,d6 rawdg

6. Verify that the device group `rawdg` was created.

       phys-newyork-1# cldevicegroup show rawdg

7. Add an entry to the `/etc/vfstab` file on `phys-newyork-1`.

       /dev/global/dsk/d5s2 /dev/global/rdsk/d5s2 /mounts/sample ufs 2 no logging

8. Create a mount directory on `newyork`.

       phys-newyork-1# mkdir -p /mounts/sample
       phys-newyork-2# mkdir -p /mounts/sample

9. Make a file system for the new device.

       phys-newyork-1# newfs /dev/global/rdsk/d5s2
       phys-newyork-1# mount /mounts/sample

10. Create an application resource group, `apprg1`, by using the `clresourcegroup` command.

        phys-newyork-1# clresourcegroup create apprg1

11. Create the `HAStoragePlus` resource in `apprg1`.

        phys-newyork-1# clresource create -g apprg1 -t SUNW.HAStoragePlus \
        -p FilesystemMountPoints=/mounts/sample -p AffinityOn=TRUE \
        -p GlobalDevicePaths=rawdg rs-hasp

    This `HAStoragePlus` resource is required for Geographic Edition systems, because the software relies on the resource to bring the device groups and file systems online when the protection group starts on the primary cluster.

12. Confirm that the application resource group is correctly configured by bringing it online and taking it offline again.

        phys-newyork-1# clresourcegroup online -emM apprg1
        phs-newyork-1# clresourcegroup offline apprg1

13. Unmount the file system.

        phys-newyork-1# umount /mounts/sample

14. Take the Oracle Solaris Cluster device group offline.

        phys-newyork-1# cldevicegroup offline rawdg

15. Reestablish the SRDF pair.

        phys-newyork-1# symrdf -g devgroup1 -noprompt establish

    Initial configuration on the secondary cluster is now complete.

### How to Replicate the Configuration Information From the Primary Cluster, When Using ZFS zpools

1. From one node of either cluster, ensure that the SRDF device group that replicates the LUNs in the zpool is in a synchronized state.

       phys-paris-1# symrdf -g devgroup1 -noprompt establish

       All devices in the RDF group 'devgroup1' are in the 'Synchronized' state.

2. From the primary cluster, export the zpool
   1. From one node of the primary cluster, disable the HAStoragePlus resource that manages the zpool.

          phys-paris-1# clresource disable hasp4appdataz

   2. On each node of the primary cluster, verify that the zpool is exported.

          phys-paris# zpool list

3. From one node of either cluster, split the pair by using the `symrdf split` command.

       phys-paris-1# symrdf -g devgroup1 -noprompt split

       An RDF 'Split' operation execution is in progress for device group 'devgroup1'.
       Please wait...

       Suspend RDF link(s).......................................Done.
       Read/Write Enable device(s) on RA at target (R2)..........Done.
       The RDF 'Split' operation device group 'devgroup1'.

4. From one node of the secondary cluster, import the zpool.

       phys-newyork-1# zpool import appdataz

5. From one node of the secondary cluster, add the HAStoragePlus resource for the zpool.

       phys-newyork-1# clresource create -g app-rg \
       -t HAStoragePlus \
       -p zpools=appdataz \
       hasp4appdataz

6. From one node of the secondary cluster, unmanage the resource group where the HAStoragePlus resource was created.

   This step exports the zpool on the secondary cluster.

       phys-newyork-1# clresource disable -g app-rg +
       phys-newyork-1# clresourcegroup offline app-rg
       phys-newyork-1# clresourcegroup unmanage app-rg

7. From one node of either cluster, reestablish the SRDF pair.

       phys-paris-1# symrdf -g devgroup1 -noprompt establish

   Initial configuration of the zpool on the secondary cluster is now complete.

Copyright © 2004, 2011, Oracle and/or its affiliates. All rights reserved. Legal Notices |