Solaris 11: Attach 2nd disk for mirrored root pool Failed

[Hawkwindeb]

VB was not previously installed on this host.
Using:
- VirtualBox 4.0.0 on Windows 7 64-bit Home Premium Host: VirtualBox-4.0.0-69151-Win.exe
- Oracle VirtualBox extension package: Oracle_VM_VirtualBox_Extension_Pack-4.0.0-69151.vbox-extpack
- Solaris 11 Express Guest via LiveCD iso: sol-11-exp-201011-live-x86.iso
- Manuals/documentation for VBox and Solaris 11 Express for configuration and installation with VirtualBox, which is a section within the Solaris 11 Express getting started guide.

"Getting_Started_With_Oracle_Solaris_11_Express_Guide_of_2010_1223_820-7799.pdf" (pages 34-40)

"VB_4_0_0_UserManual.pdf"

"Oracle_Solaris_11_ZFS_Administration_Guide_2010_Nov_821-1448.pdf"

- Downloaded VirtualBox 4.0.0 for Windows 7 files
- Downloaded the "LiveCD iso" shown above.
- MD5's for all downloads confirmed
- Created VM for Solaris 11:
- Changes in settings vs doc: 1572MB Memory, 2nd disk added (see below), USB Flash Drive added
- Some difficulty in GUI: Add/Create disks, when adding the disk Icon and GU only allow delete initial created during Wizard until OK then go back in to settings then can add 2nd disk.
- Created exact same disk as during wizard except name has 1 vs 0 (zero) at the end of the name. Both 20GB dynamic disks

- During install of Solaris, all OK but 1st disk device c7t0d0s0 and 2nd c7t1d0s0 as named by Solaris Installer. Solaris won't allow setup of 2nd disk (Solaris installer thing - see quote from p129 of ZFS admin guide below, but nothing to do with VB) so waited until OS installation completed then tried to use Solaris admin commands to add 2nd disk, per Solaris 11 ZFS Administration Guide, starting with page 129 "How to Configure a Mirrored Root Pool."

- "You cannot configure a mirrored root pool with any of the Oracle Solaris 11 Express installation methods, but you can easily configure a mirrored root pool after the installation."

- Display your current root pool status. (worked by showing only 1st disk as expected)

- Attach a second disk to configure a mirrored root pool. This Failed. Used the following command per the doc to add the 2nd disk into the rpool so Solaris can use it to mirror the 1st OS disk failed. Command:

zpool attach rpool c7t0d0s0 c7t1d0s0

- Tried many things all failed including format, fdisk within the format command...

- New steps just tried:
- From within Solaris as root role, ran touch /reconfigure
- Shutdown Solaris VM (which does a power off as well)
- From VM's settings, removed the 2nd disk
- From Virtual Disk Manager, removed and deleted 2nd disk.
- Started VM so it would clean-out the removed disk info
- Confirmed 2nd disk was gone (format command only shows c7t0d0s0)
- From within Solaris as root role, ran touch /reconfigure
- Shutdown which powers off
- From VM's settings, created new disk (wizard) for a 2nd disk using port 2 and "Dynamically expanding storage," and 20GB as before all matching the 1st disk except in name (all as before)
- Started VM
- From within Solaris as root role, ran format and it shows both c7t0d0s0 c7t1d0s0
- See following commands:

root@cygnusx1s:~# zpool status
pool: rpool
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c7t0d0s0 ONLINE 0 0 0

errors: No known data errors
root@cygnusx1s:~# zpool attach rpool c7t0d0s0 c7t1d0s0
cannot open '/dev/dsk/c7t1d0s0': I/O error
root@cygnusx1s:~#

Note: Even tried deleting both disk and the CDROM from SATA ctrl and recreated them SATA ports defaulted to: 1st disk Port 0, CDROM port 1, 2nd disk port 2. When recreated, Disks are on ports 0 and 1 with the CDROM on port 2.

Also note that the ZFS manual does not tell me to format the disk or do anything else to it either.

Thanks,

[Hawkwindeb]

****** bump ******* No one has seen this? Has anyone successfully added a 2nd disk into the ZFS pool for root to be mirrored???

[Hawkwindeb]

OK, with the help from the following site that states the problem that there is important steps missing from the Solaris documentation and then step-by-step explains the actions required to get the 2nd disk mirrored with the boot disk. The author actually used VirtualBox in his development and testing of this how-to, so please read his note at the end (see excerpt below - and my detailed steps accomplishing his test scenario)

http://constantin.glez.de/blog/2011/03/ ... ress#fdisk

Note the end. Excerpt:

BTW: I did not find a way to tell VirtualBox what disk to boot from (it only allows to specify what type of device to boot from, not what individual disk), so I reverted to just pull out (figuratively speaking) the original boot disk, then test if if boots from the mirrored one.

So in case one needs to know the steps in VirtualBox (from ver 4.1.0 Windows 7 64bit Host)

- Once resilvering is complete, rebooted tested and you're satisfied it's mirrored, shutdown Solaris
- In VBox, goto Settings for the Solaris VM, Storage, under the Controller for the 1st disks used to install Solaris, take note of the file name and its location that you used for the 1st boot disk, then right-click that disk and choose "Remove Attachment" (note that this does not delete or change anything inside that disk, it really is like physically disconnecting or removing the disk from the VM so it can't use it)
- Click OK and then Start the VM
- Watch Solaris boot, login, open a terminal, su, format and note only the 2nd disk is listed which obviously means it booted from the mirror. "zpool status" shows it also.
- Shutdown Solaris
- In VBox, Settings for the Solaris VM, Storage, under the Controller for the 1st disks used to install Solaris, click the plus to add a disk (not the CD looking icon)
- A file-open dialog box appears, go to the file used for the 1st boot disk and select it
- Click OK and then Start
- Watch Solaris boot, login, open a terminal, su, format and note both 1st and 2nd disk are listed. Run "zpool status" which shows it's not only mirrored again but it states: Resilvered ... at the time of booting.

Enjoy!