How to Consolidate Zones Storage on an Oracle ZFS Storage Appliance

Table of Contents

• About Zones on Shared Storage
• Overview of the Components, Process, and Architecture
• Create an ISCSI Target and Export the LU
• Create and Install the Zone on the First Oracle Solaris System
• Detach the Zone and Migrate It to the Second Oracle Solaris System
• Add Multipathing to the iSCSI Storage Configuration
• Control iSCSI LU and Target Discovery
• (Optional) Use ZFS-Encrypted File Systems
• See Also
• About the Authors

About Zones on Shared Storage

Introduced in Oracle Solaris 11, the Zones on Shared Storage (ZOSS) feature provides the following advantages:

• Hosts zone installation and data on shared storage over flexible FC, iSCSI, or SAS protocols for easier storage management
• Provides a simple ZFS configuration:
  
  
  • One rpool (<zonename>_rpool) per zone
  • Data pools (<zonename>_<zpool>)
• Uses standard zone command-line interfaces and storage setup capabilities:
  
  
  • Perform basic zone configuration (zonecfg)
  • Identify storage components (suriadm)
  • Create, import, and export a pool (zoneadm)
• Easily migrates a zone between systems using a standard command-line interface to detach and attach zones (zoneadm)
• Uses iSCSI initiator groups and the Challenge-Handshake Authentication Protocol (CHAP) to control the discovery of iSCSI logical units and targets
• Secures storage and data paths with ZFS encryption (optional)

Note: The steps mentioned below are for Native Zones. For Kernel Zones, which can also use shared storage, most steps are the same. In the places where things are different the article will note so.

Overview of the Components, Process, and Architecture

In addition to an Oracle Solaris Zone, the configuration described in this article consists of the following components:

• The iSCSI protocol, a block-based storage protocol over an IP network, is used.
• The logical unit (LU) is a numbered storage component identified by a logical unit ID.
• The iSCSI initiator is a client (driver) that initiates iSCSI requests to the iSCSI target. In this article, two Oracle Solaris systems are initiators.
• The iSCSI target, which is hosted from an Oracle ZFS Storage Appliance, is the storage component that receives the iSCSI requests.

The basic configuration steps are as follows:

• Create an iSCSI target and export the LU on the Oracle ZFS Storage Appliance.
• Create and install an Oracle Solaris Zone on the first Oracle Solaris system.
• Detach the Oracle Solaris Zone and migrate it to the second Oracle Solaris system.
• Enable iSCSI multipathing.
• Control iSCSI LU and target discovery.
• (Optional) Secure the zone's ZFS data by manually creating an encrypted zone rpool.

The next sections of this article demonstrate how to configure an Oracle Solaris Zone (zoss-zone) on an iSCSI device that is hosted from an Oracle ZFS Storage Appliance (zfssa) and then migrate the zone from one Oracle Solaris system (hostA) to a second Oracle Solaris system (hostB).

Create an ISCSI Target and Export the LU

Perform the following steps on the Oracle ZFS Storage Appliance.

1. Create the iSCSI target, as shown in Figure 2:
   
   
   1. Select Configuration->SAN->iSCSI.
   2. Click the + (plus) sign next to Targets.
   3. Create an iSCSI target by specifying the zone name (zoss-zone) in the Alias field.
   4. From the Network interfaces list, select the network interface.
2. Create and export the iSCSI LUN, as shown in Figure 3:
   
   
   1. Select Configuration->Shares->LUNs, and then click the + (plus) sign next to LUNs.
   2. Create and export the LUN by completing the Create LUN screen components:
      
      
      1. Project: default
      2. Name: zoss-zone-rpool-primary
      3. Volume size: 10 G
      4. Volume block size: 8K
      5. Target group: All targets
      6. Initiator group(s): All initiators
      7. LU number: Auto-assign
   3. Click Apply.

Create and Install the Zone on the First Oracle Solaris System

This section describes how to create and install the zone on the first Oracle Solaris system (hostA).

1. Create an Oracle Solaris Zone called zoss-zone by running the following commands:
   
   

   Note: When you run the add storage command, specify the LU GUID shown in Figure 3 as the storage identifier.

   Copy

   
   
   root@hostA:~# zonecfg -z zoss-zone
   zonecfg:zoss-zone> create
   zonecfg:zoss-zone> set zonepath=/zones/zoss-zone
   zonecfg:zoss-zone> add rootzpool
   zonecfg:zoss-zone:rootzpool> add storage \
   iscsi://192.168.202.10/luname.naa.600144F0949056290000529625DD001
   zonecfg:zoss-zone:rootzpool> end
   zonecfg:zoss-zone> commit
   zonecfg:zoss-zone> exit
   

Note: In the case of Kernel Zones you don't add a rootzpool but instead select the device that already exists as part of the template and then use set storage=iscsi://192.168.202.10/luname.naa.600144F0949056290000529625DD001 to point the device to the iSCSI LUN.

2. Install the zone:
   
   

   Installing the zone completes the following tasks:

   1. Configures the iSCSI initiator
   2. Discovers the iSCSI target and LU
   3. Creates logical device links
   4. Creates the zone rpool
   Copy

   
   
   root@hostA:~# zoneadm -z zoss-zone install
   Configured zone storage resource(s) from:
   iscsi://192.168.202.10/luname.naa.600144F0949056290000529625DD0001
   Created zone zpool: zoss-zone_rpool
   

Detach the Zone and Migrate It to the Second Oracle Solaris System

This section describes how to detach the zone from hostA and migrate it to the second Oracle Solaris system (hostB).

1. Shut down and detach the zone:
   
   
   Copy

   
   
   root@hostA:~# zoneadm -z zoss-zone shutdown
   root@hostA:~# zoneadm -z zoss-zone detach
   Exported zone zpool: zoss-zone_rpool
   Unconfigured zone storage resource(s) from:
   iscsi://192.168.202.10/luname.naa.600144F0949056290000529625DD0001
   

2. Transfer the zone configuration from hostA to hostB:
   
   
   Copy

   
   
   root@hostA:~$ zonecfg -z zoss-zone export | ssh hostB 'cat - > /root/zonecfg.zfg' 
   root@hostB:~# zonecfg -z zoss-zone -f /root/zonecfg.zfg
   

3. Attach the zone to hostB:
   
   

   Attaching the zone completes the following tasks:

   1. Configures the iSCSI initiator
   2. Discovers the iSCSI target and LU
   3. Creates the logical device links
   Copy

   
   
   root@hostB:~# zoneadm -z zoss-zone attach
   

Note: Kernel Zones support Live Migration, which means they don't need to be shut down, detached and attached. For more in set the documentation on Migrating an Oracle Solaris Kernel Zone.

Add Multipathing to the iSCSI Storage Configuration

No additional changes are required to the basic configuration established in the previous sections. However, consider adding multipathing to achieve the following additional benefits:

• Ability to use dedicated NICs and network links for iSCSI traffic
• Ability to connect iSCSI targets to multiple networks
• Ability to limit LU discovery to a dedicated iSCSI target group

1. Assign the iSCSI target to multiple NICs on the Oracle ZFS Storage Appliance:
   
   
   1. Select Configuration->SAN, and create a target group called zoss-zone.
      
      
      1. Select the LU and drag it over to the Target Groups box.
      2. Click the edit button.
      3. Add the zoss-zone name.
   2. Assign multiple network interfaces to the target group:
      
      
      1. Select the LU target.
      2. Select the network interface.
2. Limit LU discovery to the zoss-zone target group on the Oracle ZFS Storage Appliance, as shown in Figure 5:
   
   
   1. Select Configuration->Shares, and then select the zoss-zone LU.
   2. Click the edit button.
   3. Click the Protocols tab.
   4. Select the zoss-zone target group.
3. Configure dedicated iSCSI network links on the first Oracle Solaris initiator (hostA).
   
   

   If multiple addresses are configured for the Oracle ZFS Storage Appliance, multiple discovery addresses are configured automatically.

   Copy

   
   
   root@hostA:~# cat /etc/hosts | grep zfssa
   192.168.202.10  zfssa-iscsi
   192.168.203.10  zfssa-iscsi
   root@hostA:~# zonecfg -z zoss-zone info rootzpool
   rootzpool:
          storage: iscsi://zfssa-iscsi/luname.naa.600144F0949056290000529625DD0001
   root@hostA:~# ipadm create-ip net1
   root@hostA:~# ipadm create-ip net2
   root@hostA:~# ipadm create-addr -T static -a 192.168.202.7 net1/iscsi202
   root@hostA:~# ipadm create-addr -T static -a 192.168.203.7 net2/iscsi203
   root@cantaloup:~# iscsiadm modify discovery -t enable
   root@cantaloup:~# iscsiadm list discovery-address
   Discovery Address: 192.168.202.7:3260
   Discovery Address: 192.168.203.7:3260
   

4. List the targets that have multipathing disabled (via stmsboot -d -D iscsi) on the Oracle Solaris initiator.
   Copy

   
   
   root@hostA:~# iscsiadm list target -S
   
   Target: iqn.1986-03.com.sun:02:12c5dc60-1854-62c4-8f61-aa78d0815dc2
       Alias: zoss-zone
       TPGT: 3
       ISID: 4000002a0000
       Connections: 1
       LUN: 0
            Vendor:  SUN     
            Product: Sun Storage 7000
            OS Device Name: /dev/rdsk/c1t6d0s2
   
   Target: iqn.1986-03.com.sun:02:12c5dc60-1854-62c4-8f61-aa78d0815dc2
       Alias: zoss-zone
       TPGT: 2
       ISID: 4000002a0000
       Connections: 1
       LUN: 0
            Vendor:  SUN     
            Product: Sun Storage 7000
            OS Device Name: /dev/rdsk/c1t3d0s2
   

5. Enabled iSCSI multipathing and list the iSCSI targets on the Oracle Solaris initiator:
   
   
   Copy

   
   
   root@hostA:~# stmsboot -e -D iscsi
   root@hostA:~# iscsiadm list target -S
   
   Target: iqn.1986-03.com.sun:02:12c5dc60-1854-62c4-8f61-aa78d0815dc2
       Alias: zoss-zone
       TPGT: 3
       ISID: 4000002a0000
       Connections: 1
       LUN: 0
            Vendor:  SUN     
            Product: Sun Storage 7000
            OS Device Name: /dev/rdsk/c0t600144F0949056290000529625DD0001d0s 2
   
   Target: iqn.1986-03.com.sun:02:12c5dc60-1854-62c4-8f61-aa78d0815dc2
       Alias: zoss-zone
       TPGT: 2
       ISID: 4000002a0000
       Connections: 1
       LUN: 0
            Vendor:  SUN     
            Product: Sun Storage 7000
            OS Device Name: /dev/rdsk/c0t600144F0949056290000529625DD0001d0s2
   

Control iSCSI LU and Target Discovery

You can control the discovery of an iSCSI target or an LU. If a target or LU is not discovered, it is not visible and it is not accessible.

To control LU discovery, you assign an LU to group of initiators and then specify which initiator group is permitted to discover the LU. To control target discovery, you enable unidirectional CHAP to control which initiators are allowed to discover a target.

1. Identify the iSCSI qualified name (IQN) on the first Oracle Solaris initiator:
   Copy

   
   
   root@hostA:~# iscsiadm list initiator-node
   Initiator node name: iqn.1986-03.com.sun:01:e00000000000.5295332b
   

2. On the Oracle ZFS Storage Appliance, create an initiator group by dragging the allowed initiators into a new initiator group, and then assign the initiator group to the LU, as shown in Figure 6.
   
   
   1. Select the initiator and drag it to the new initiator group, zoss-zone-hosts.
   2. Select Configuration->Shares, and then select the zoss-zone target LU.
   3. Click the edit button.
   4. Click the Protocols tab.
   5. Click the edit button next to the Initiator group and select the zoss-zone-hosts initiator group.
3. Enable unidirectional CHAP on the first Oracle Solaris initiator:
   
   Copy

   
   
   root@hostA:~# iscsiadm modify initiator-node --authentication CHAP
   root@hostA:~# iscsiadm modify initiator-node --CHAP-secret
   Enter secret: ************
   Re-enter secret: ************
   
   root@hostA:~# iscsiadm list initiator-node
   Initiator node name: iqn.1986-03.com.sun:01:e00000000000.5295332b
   Initiator node alias: hostA
       Authentication Type: CHAP
       CHAP Name: iqn.1986-03.com.sun:01:e00000000000.5295332b
   

4. On the Oracle ZFS Storage Appliance, enable CHAP for the initiator and the target, as shown in Figure 7:
   
   
   1. Select Configuration->SAN->Initiators, and then select the initiator and click the edit button.
   2. Click the Use CHAP button.
   3. Enter the CHAP name, which is the IQN.
   4. Enter the CHAP secret password.
   5. Select Configuration->SAN->Targets, and then select the target and click the edit button.
   6. For Initiator authentication node, select the CHAP option.
5. Disable and then re-enable discovery on the first Oracle Solaris initiator:
   
   
   Copy

   
   
   root@hostA:~# iscsiadm modify discovery -t disable
   root@hostA:~# iscsiadm modify discovery -t enable
   

(Optional) Use ZFS-Encrypted File Systems

You can secure storage and the data path by enabling ZFS encryption when you create a ZFS file system or volume. To do this, you select a wrapping key; the source can be the prompt, a file, HTTPS, or PKCS#1. The system selects the random encryption key.

In the following steps, the zoss-zone rpool is re-created because the ZOSS framework creates only unencrypted datasets. After the rpool is re-created, ZOSS finds and uses the re-created rpool.

1. Identify the iSCSI device of the existing zoss-zone rootzpool device:
   
   Copy

   
   
   root@hostA:~# zonecfg -z zoss-zone info rootzpool
   rootzpool:
       storage: iscsi://zfssa-iscsi/luname.naa.600144F0949056290000529625DD0001
   

2. Identify the zone's iSCSI device to the mapped device name:
   
   
   Copy

   
   
   root@hostA:~# suriadm map iscsi://zfssa-iscsi/luname.naa.600144F0949056290000529625DD0001
   PROPERTY     VALUE
   mapped-dev   /dev/dsk/c0t600144F0949056290000529625DD0001d0s0
   

3. Create the encryption key for the zone's rpool (zoss-zone_rpool) by using the pktool command to generate an AES256 encryption key:
   
   
   Copy

   
   
   root@hostA:~# pktool genkey keystore=file outkey=/etc/zones/zoss-zonekey.file keytype=aes keylen=256
   root@hostA:~# chmod 600 /etc/zones/zoss-zonekey.file  
   

4. Create the zone's rpool (zoss-zone_rpool) and specify that the top-level pool file system be encrypted. Also specify the encrypted key file and device that was mapped in Step 2:
   
   
   Copy

   
   
   root@hostA:~# zpool create -m /zones/zoss-zone -O encryption=on 
   -O keysource=raw,file:///etc/zones/zoss-zonekey.file zoss-zone_rpool  c0t600144F0949056290000529625DD0001d0
   

5. Export the zoss-zone rpool and re-install the zoss-zone:
   
   
   Copy

   
   
   root@hostA:~# zpool export zoss-zone_rpool
   root@hostA:~# zoneadm -z zoss-zone install 
   

See Also

• Download Oracle Solaris 11
• Access Oracle Solaris 11 product documentation
• Access all Oracle Solaris 11 how-to articles
• Learn more with Oracle Solaris 11 training and support
• See the official Oracle Solaris blog
• Follow Oracle Solaris on Facebook and Twitter

About the Authors

Detlef Drewanz is a Master Principal Sales Consultant on the Oracle Systems Sales Consulting team in Germany. He is an Oracle Solaris and virtualization specialist.

Cindy Swearingen is an Oracle Solaris Product Manager who specializes in ZFS and storage features.

Revision 1.1, 07/02/2018
Revision 1.0, 06/18/2014