virtualbox.org

by **sunwizard** » 4. Dec 2018, 09:56

Hello Guys.

I had a question while testing Solaris 10 x86 on Oracle VirtualBox.

I can use various kinds of disk controllers in a virtual machine on VirtualBox. But I have checked that only SATA controllers support Hot Plug function.

I have used the devfsadm command on the Solaris OS (SPARC Server) to recognize a new disk. However, this command did not recognize the new disk in Solaris 10 x86 on VirtualBox.
So I used the cfgadm command to recognize the new disk.

# cfgadm -x sata_port_activate AP_ID
# cfgadm -c configure AP_ID

To perform some tests, such as OS disk mirroring, I removed the first disk and added a new disk to the same port.
Then, I had the following results and the disk that was replaced was not recognized.

-bash-3.2# cfgadm -al | grep 'sata0/[01]\>'
sata0/0 disk connected configured unusable
sata0/1::dsk/c0t1d0 disk connected configured ok

Normally I use the following commands to recognize the disk in Solaris 10 SPARC.

# devfsadm -Cv -c disk
# devfsadm -c disk

But not recognized by the above commands and could not be changed to ok condition by using the cfgadm command.

- unconfigure/configure
- disconnect/connect
- disconnect/delete /dev/(r)dsk/c#t# files/connect/configure

Of course, rebooting the Solaris will save the problem, but I want it to solve without rebooting.

How can I solve this problem? I need your help.

I'm not familiar with English. But thanks for reading so far.

by **stes** » 12. Dec 2018, 10:19

The current release of Solaris is 11.4.

I suspect the support in Solaris 10 would be more limited, for example at

Code: Select all Expand view Collapse view: https://www.oracle.com/technetwork/server-storage/solaris10/overview/sata0-136306.html

they say that the Solaris SPARC has no SATA support (only PATA).

Did you use SATA on Solaris SPARC ? I suppose not, you are comparing different technologies, for example "fibre channel" (FC) disk addition is much more advanced, and I suspect on SPARC you would have been using Emulex or Qlogic HBA with FC disk addition.

So it is a comparison "doomed to fail" because you'd have to see how it behaves on SPARC with SATA, which is not what you probably had to start with (I think you used FC on SPARC ?)

And VirtualBox as far as I know does not offer a "virtual" Emulex or Qlogic adapter to a guest.

On Solaris 11.4 there is a manpage:

Code: Select all Expand view Collapse view: man cfgadm_sata

about SATA hotplug.

Anyway on Solaris 11.4 I tried on VirtualBox to add a SATA disk (hotplug) and then before the disk addition on a SATA controller with 2 ports (port count = 2) one port has the boot disk the 2nd port nothing.

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al | more Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1 sata-port disconnected unconfigured failed

After adding the disk online (and without reboot of the guest):

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -x sata_port_activate sata0/1 root@pluto:~# cfgadm -c configure sata0/1 root@pluto:~# cfgadm -al | head -4 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1::dsk/c1t1d0 disk connected configured ok usb0/1 usb-input connected configured ok

and then I can use the disk on the Solaris 11.4 guest

Code: Select all Expand view Collapse view: format -e Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c1t0d0 <VBOX-HARDDISK-1.0-16.00GB> /pci@0,0/pci8086,2829@d/disk@0,0 1. c1t1d0 <VBOX-HARDDISK-1.0-2.00GB> /pci@0,0/pci8086,2829@d/disk@1,0 Specify disk (enter its number): ^C root@pluto:~# zpool create qpool c1t1d0 root@pluto:~# zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT qpool 1.98G 140K 1.98G 0% 1.00x ONLINE - rpool 15.6G 3.67G 12.0G 23% 1.00x ONLINE -

Adding a second disk (to get a total of 3 disks) online fails because the SATA controller has only 2 ports I think, because it is a controller that has only sata0/0 and sata0/1 slots available.

In any case I'm pessimistic about the goal of simulating this operation on Solaris 10,
because I'm under the impression that you are trying to get the advanced features of Solaris SPARC hotadd fibre channel under the simple SATA virtualbox guest ?

by **stes** » 12. Dec 2018, 11:21

In any case in a Fibre Channel or FICON environment it has been many years completely normal to add hotplug 128 LUNS or so on a HBA.

Or with SCSI 15 LUNs or more and certainly SAS probably has plenty of features in this area.

So it is a bit unfair to compare the "high end" SPARC platform with its plugin possibilties to low-end VirtualBox guest with SATA.

The Solaris 11.4 has a package called

Code: Select all Expand view Collapse view: # pkg list pkg:/system/dynamic-reconfiguration system/dynamic-reconfiguration 11.4-11.4.0.0.1.15.0 i--

with the cfgadm_sata support in it (among the cfgadm_scsi which VirtualBox does not seem to support).

By default when I create a guest in VirtualBox it has SATA controller port count set to 2.

When I set the port count of the SATA controller to 4.

And reinstall the Solaris 11.4 guest (via the AI automated install).

I get:

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1 sata-port empty unconfigured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok

Then if I add "hotplug" 2 disks without shutting down the guest, I get

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1 sata-port disconnected unconfigured failed sata0/2 sata-port disconnected unconfigured failed sata0/3 sata-port empty unconfigured ok

Then the disks are correctly added it seems

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -x sata_port_activate sata0/1 Activate the port: /devices/pci@0,0/pci8086,2829@d:1 This operation will enable activity on the SATA port Continue (yes/no)? yes root@pluto:~# cfgadm -x sata_port_activate sata0/2 Activate the port: /devices/pci@0,0/pci8086,2829@d:2 This operation will enable activity on the SATA port Continue (yes/no)? yes

and

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1 disk connected unconfigured unknown sata0/2 disk connected unconfigured unknown sata0/3 sata-port empty unconfigured ok

and

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -c configure sata0/1 root@pluto:~# cfgadm -c configure sata0/2 root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2::dsk/c1t2d0 disk connected configured ok sata0/3 sata-port empty unconfigured ok

so that

Code: Select all Expand view Collapse view: root@pluto:~# format -e Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c1t0d0 <VBOX-HARDDISK-1.0-16.00GB> /pci@0,0/pci8086,2829@d/disk@0,0 1. c1t1d0 <VBOX-HARDDISK-1.0-7.00GB> /pci@0,0/pci8086,2829@d/disk@1,0 2. c1t2d0 <VBOX-HARDDISK-1.0-2.00GB> /pci@0,0/pci8086,2829@d/disk@2,0 Specify disk (enter its number): ^C

by **stes** » 12. Dec 2018, 14:43

It appears you can set the port count to 30 in VirtualBox and then install Solaris 11.4 (on the boot disk) and then you get:

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1 sata-port empty unconfigured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok sata0/4 sata-port empty unconfigured ok sata0/5 sata-port empty unconfigured ok sata0/6 sata-port empty unconfigured ok sata0/7 sata-port empty unconfigured ok sata0/8 sata-port empty unconfigured ok sata0/9 sata-port empty unconfigured ok sata0/10 sata-port empty unconfigured ok sata0/11 sata-port empty unconfigured ok sata0/12 sata-port empty unconfigured ok sata0/13 sata-port empty unconfigured ok sata0/14 sata-port empty unconfigured ok sata0/15 sata-port empty unconfigured ok sata0/16 sata-port empty unconfigured ok sata0/17 sata-port empty unconfigured ok sata0/18 sata-port empty unconfigured ok sata0/19 sata-port empty unconfigured ok sata0/20 sata-port empty unconfigured ok sata0/21 sata-port empty unconfigured ok sata0/22 sata-port empty unconfigured ok sata0/23 sata-port empty unconfigured ok sata0/24 sata-port empty unconfigured ok sata0/25 sata-port empty unconfigured ok sata0/26 sata-port empty unconfigured ok sata0/27 sata-port empty unconfigured ok sata0/28 sata-port empty unconfigured ok sata0/29 sata-port empty unconfigured ok

This means that you possibly have a lot of empty slots to add virtual SATA disks (attach) them to it online (hotplug).

I haven't tried adding 28 disks, but for example after:

Code: Select all Expand view Collapse view: VBoxManage storageattach pluto --storagectl SATA --medium "5d63e15c-5dd8-4c63-8c67-f16ea8b2ade0" --port 28 --type hdd

to add a disk to sata0/29 I get on the guest:

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0/28 Ap_Id Type Receptacle Occupant Condition sata0/28 sata-port disconnected unconfigured failed

and then

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -x sata_port_activate sata0/28 Activate the port: /devices/pci@0,0/pci8086,2829@d:28 This operation will enable activity on the SATA port Continue (yes/no)? y root@pluto:~# cfgadm -al sata0/28 Ap_Id Type Receptacle Occupant Condition sata0/28 disk connected unconfigured unknown

if I run at this point

Code: Select all Expand view Collapse view: root@pluto:~# ls /dev/rdsk c1t0d0 c1t0d0p1 c1t0d0p3 c1t0d0s0 c1t0d0s2 c1t0d0s4 c1t0d0s6 c1t0d0p0 c1t0d0p2 c1t0d0p4 c1t0d0s1 c1t0d0s3 c1t0d0s5

and then

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -c configure sata0/28 root@pluto:~# ls /dev/rdsk c1t0d0 c1t0d0p3 c1t0d0s2 c1t0d0s6 c1t28d0p2 c1t28d0s1 c1t28d0s5 c1t0d0p0 c1t0d0p4 c1t0d0s3 c1t28d0 c1t28d0p3 c1t28d0s2 c1t28d0s6 c1t0d0p1 c1t0d0s0 c1t0d0s4 c1t28d0p0 c1t28d0p4 c1t28d0s3 c1t0d0p2 c1t0d0s1 c1t0d0s5 c1t28d0p1 c1t28d0s0 c1t28d0s4 root@pluto:~# cfgadm -al sata0/28 Ap_Id Type Receptacle Occupant Condition sata0/28::dsk/c1t28d0 disk connected configured ok sata0/28: No matching library found

So there is a strange message "No matching library found" which possibly is a Solaris 11.4 bug

but it seems to work because the disk device names under /dev/rdsk were automatically created (without explicitley running devfsadm).

I think you would run devfsadm if you remove the device files under /dev/rdsk for some reason (to cleanup) and then want to regenerate the device files.

This is for example if you want to renumber the device files I think, but it was not required for me to run devfsadm.

In any case it seems to work fine

Code: Select all Expand view Collapse view: root@pluto:~# format -e Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c1t0d0 <VBOX-HARDDISK-1.0-16.00GB> /pci@0,0/pci8086,2829@d/disk@0,0 1. c1t28d0 <VBOX-HARDDISK-1.0-7.00GB> /pci@0,0/pci8086,2829@d/disk@1c,0

and

Code: Select all Expand view Collapse view: root@pluto:~# zpool create -f apool c1t28d0 root@pluto:~# zpool status apool pool: apool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM apool ONLINE 0 0 0 c1t28d0 ONLINE 0 0 0 errors: No known data errors

The -f option with the zpool create was because I had already used the disk before in a different pool.

by **sunwizard** » 16. Dec 2018, 09:19

>> Hello, stes.

>> I use SCSI, FC, SAS type on Solaris SPARC.
>> I am using a virtualization program to simply test Solaris 10.
>> When I tested OS Mirroring in VirtualBox, I got a problem when I removed the first disk and added it to new disk(same SATA port).
>> This happened in VirtualBox and I needed to reboot the virtual machine for recognizing new disk. Of course, it worked fine on physical SPARC machines and VMware Workstations without system reboot.
>> So I posted the question.

>> Thanks for your explanation.

by **stes** » 16. Dec 2018, 17:04

Well for completeness the same test for me works on host Solaris 11.4 with VirtualBox 6.0.0 RC1 and a guest running the (old version of) Solaris 10 :

On a guest running the following version of Solaris 10 with a UFS root filesystem :

Code: Select all Expand view Collapse view: # cat /etc/release Oracle Solaris 10 8/11 s10x_u10wos_17b X86 Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved. Assembled 23 August 2011 # uname -a SunOS neptune 5.10 Generic_147441-01 i86pc i386 i86pc

The root filesystem of the guest is UFS (not ZFS) :

Code: Select all Expand view Collapse view: /dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no - /dev/dsk/c0t0d0s6 /dev/rdsk/c0t0d0s6 /usr ufs 1 no - /dev/dsk/c0t0d0s1 /dev/rdsk/c0t0d0s1 /var ufs 1 no -

Then I have in VirtualBox the SATA port count set to 4 :

Code: Select all Expand view Collapse view: # cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c0t0d0 disk connected configured ok sata0/1::dsk/c0t1d0 cd/dvd connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok

So there are two empty ports on the SATA controller for "online" hotplug adding disks.

Initially on port sata0/2 there is nothing, and then when I add in VirtualBox a disk drive:

Code: Select all Expand view Collapse view: # cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c0t0d0 disk connected configured ok sata0/1::dsk/c0t1d0 cd/dvd connected configured ok sata0/2 sata-port disconnected unconfigured failed sata0/3 sata-port empty unconfigured ok

then without a reboot of the guest :

Code: Select all Expand view Collapse view: # cfgadm -x sata_port_activate sata0/2 Activate the port: /devices/pci@0,0/pci8086,2829@d:2 This operation will enable activity on the SATA port Continue (yes/no)? yes

this changes the status to:

Code: Select all Expand view Collapse view: # cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c0t0d0 disk connected configured ok sata0/1::dsk/c0t1d0 cd/dvd connected configured ok sata0/2 disk connected unconfigured unknown sata0/3 sata-port empty unconfigured ok

at that point the device links under /dev do not yet exist:

Code: Select all Expand view Collapse view: # ls /dev/rdsk c0t0d0p0 c0t0d0s1 c0t0d0s15 c0t0d0s7 c0t1d0p3 c0t1d0s12 c0t1d0s4 c0t0d0p1 c0t0d0s10 c0t0d0s2 c0t0d0s8 c0t1d0p4 c0t1d0s13 c0t1d0s5 c0t0d0p2 c0t0d0s11 c0t0d0s3 c0t0d0s9 c0t1d0s0 c0t1d0s14 c0t1d0s6 c0t0d0p3 c0t0d0s12 c0t0d0s4 c0t1d0p0 c0t1d0s1 c0t1d0s15 c0t1d0s7 c0t0d0p4 c0t0d0s13 c0t0d0s5 c0t1d0p1 c0t1d0s10 c0t1d0s2 c0t1d0s8 c0t0d0s0 c0t0d0s14 c0t0d0s6 c0t1d0p2 c0t1d0s11 c0t1d0s3 c0t1d0s9

but when I configure the port :

Code: Select all Expand view Collapse view: # cfgadm -c configure sata0/2

the disk is available to the Solaris 10 guest without reboot and the device links under /dev were created:

Code: Select all Expand view Collapse view: # ls /dev/rdsk c0t0d0p0 c0t0d0s12 c0t0d0s7 c0t1d0s1 c0t1d0s4 c0t2d0p3 c0t2d0s15 c0t0d0p1 c0t0d0s13 c0t0d0s8 c0t1d0s10 c0t1d0s5 c0t2d0p4 c0t2d0s2 c0t0d0p2 c0t0d0s14 c0t0d0s9 c0t1d0s11 c0t1d0s6 c0t2d0s0 c0t2d0s3 c0t0d0p3 c0t0d0s15 c0t1d0p0 c0t1d0s12 c0t1d0s7 c0t2d0s1 c0t2d0s4 c0t0d0p4 c0t0d0s2 c0t1d0p1 c0t1d0s13 c0t1d0s8 c0t2d0s10 c0t2d0s5 c0t0d0s0 c0t0d0s3 c0t1d0p2 c0t1d0s14 c0t1d0s9 c0t2d0s11 c0t2d0s6 c0t0d0s1 c0t0d0s4 c0t1d0p3 c0t1d0s15 c0t2d0p0 c0t2d0s12 c0t2d0s7 c0t0d0s10 c0t0d0s5 c0t1d0p4 c0t1d0s2 c0t2d0p1 c0t2d0s13 c0t2d0s8 c0t0d0s11 c0t0d0s6 c0t1d0s0 c0t1d0s3 c0t2d0p2 c0t2d0s14 c0t2d0s9

Also the disk is available in Solaris 10 for using it in a ZFS pool for example (without reboot) :

Code: Select all Expand view Collapse view: # format -e Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t0d0 <DEFAULT cyl 2085 alt 2 hd 255 sec 63> /pci@0,0/pci8086,2829@d/disk@0,0 1. c0t2d0 <DEFAULT cyl 2086 alt 2 hd 255 sec 63> /pci@0,0/pci8086,2829@d/disk@2,0 Specify disk (enter its number): ^C # zpool create mypool c0t2d0

So the Solaris 10 guest behaves like the Solaris 11.4 guest for me ...

by **stes** » 16. Dec 2018, 18:18

Sunwizard wrote:

I removed the first disk and added a new disk to the same port.

How did you do this exactly ? I think the cfgadm_sata manpage gives a clue here :

The emergency "sata_port_deactivate" operation is not supported on
ports with attached disks containing critical partitions such as root
(/), /usr, swap, or /var. The deactivate operation should not be
attempted on such ports. Incorrect usage can result in a system hang
and require a reboot.

Also the cfgadm -al output that you posted has the "unusable" flag ...

For example on a 11.4 guest I tried the following to replace a disk :

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1 sata-port empty unconfigured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok

then add two disks to make a mirror:

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -x sata_port_activate sata0/1 Activate the port: /devices/pci@0,0/pci8086,2829@d:1 This operation will enable activity on the SATA port Continue (yes/no)? yes root@pluto:~# cfgadm -x sata_port_activate sata0/2 Activate the port: /devices/pci@0,0/pci8086,2829@d:2 This operation will enable activity on the SATA port Continue (yes/no)? yes root@pluto:~# cfgadm -c configure sata0/1 root@pluto:~# cfgadm -c configure sata0/2

so the mirror is :

Code: Select all Expand view Collapse view: root@pluto:~# zpool create mymirror mirror c1t1d0 c1t2d0 root@pluto:~# zpool status mymirror pool: mymirror state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM mymirror ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 errors: No known data errors

then before "removing the disk" in VirtualBox I'm preparing the operating system (guest) first to remove the disk:

Code: Select all Expand view Collapse view: root@pluto:~# zpool detach mymirror c1t2d0 root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2::dsk/c1t2d0 disk connected configured ok sata0/3 sata-port empty unconfigured ok root@pluto:~# root@pluto:~# cfgadm -c unconfigure sata0/2 Unconfigure the device at: /devices/pci@0,0/pci8086,2829@d:2 This operation will suspend activity on the SATA device Continue (yes/no)? yes root@pluto:~# cfgadm -al sata0/2 Ap_Id Type Receptacle Occupant Condition sata0/2 disk connected unconfigured ok

Did you unconfigure the disk before removing it ?

I've just tested that I can replace the disk by a different disk on the same port without a reboot.
(guest Solaris 11.4)

However the test pool I used above is a simple "mymirror" test zfs pool , not a pool for operating system mirroring,
with the root / and /var etc. on the disk, the cfgadm_sata manpage indicates that there are some limitations.

Provided proper unconfigure of the disk , I did not have to reboot to replace a disk in a mirror,
and attach it again afterwards (with a new disk).

Also the manpage cfgadm_sata warns:

If the SATA connectors are the hot-pluggable type and the SATA con-
troller supports hotplugging, unplugging a device without unconfiguring
it may result in system hang or data loss.

by **sunwizard** » 16. Dec 2018, 19:22

Hello, stes.

Here is my work for OS mirroring:

■ Solaris 10 x86 Info:

Code: Select all Expand view Collapse view: # more /etc/release Oracle Solaris 10 1/13 s10x_u11wos_24a X86 Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights reserved. Assembled 17 January 2013 # uname -a SunOS solaris101 5.10 Generic_147148-26 i86pc i386 i86pc # df -h / Filesystem size used avail capacity Mounted on /dev/dsk/c0t0d0s0 18G 3.9G 14G 23% /

■ OS disk information and partition table:

Code: Select all Expand view Collapse view: OS Disk : SATA 20GB (Port 0) CD/DVD : SATA (Port 29) # cfgadm -al | grep 'sata0/0\>' sata0/0::dsk/c0t0d0 disk connected configured ok # echo | format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t0d0 <ATA -VBOX HARDDISK -1.0 cyl 2607 alt 2 hd 255 sec 63> /pci@0,0/pci8086,2829@d/disk@0,0 Specify disk (enter its number): Specify disk (enter its number): Part Tag Flag Cylinders Size Blocks 0 root wm 1 - 2340 17.93GB (2340/0/0) 37592100 1 swap wu 2341 - 2602 2.01GB (262/0/0) 4209030 2 backup wm 0 - 2606 19.97GB (2607/0/0) 41881455 3 unassigned wm 2603 - 2606 31.38MB (4/0/0) 64260 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0 8 boot wu 0 - 0 7.84MB (1/0/0) 16065 9 unassigned wm 0 0 (0/0/0) 0

■ Adding new disk (20GB/Port 1):

Code: Select all Expand view Collapse view: # cfgadm -al | grep 'sata0/[01]\>' sata0/0::dsk/c0t0d0 disk connected configured ok sata0/1 sata-port disconnected unconfigured failed # cfgadm -x sata_port_activate sata0/1 Activate the port: /devices/pci@0,0/pci8086,2829@d:1 This operation will enable activity on the SATA port Continue (yes/no)? yes # cfgadm -c configure sata0/1 # cfgadm -al | grep 'sata0/[01]\>' sata0/0::dsk/c0t0d0 disk connected configured ok sata0/1::dsk/c0t1d0 disk connected configured ok # echo | format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t0d0 <ATA -VBOX HARDDISK -1.0 cyl 2607 alt 2 hd 255 sec 63> /pci@0,0/pci8086,2829@d/disk@0,0 1. c0t1d0 <ATA -VBOX HARDDISK -1.0 cyl 2608 alt 2 hd 255 sec 63> /pci@0,0/pci8086,2829@d/disk@1,0 Specify disk (enter its number): Specify disk (enter its number):

■ Labeling new disk by using the prtvtoc and fmthard commands:

Code: Select all Expand view Collapse view: # fdisk -B /dev/rdsk/c0t1d0p0 # prtvtoc /dev/rdsk/c0t0d0s2 | fmthard -s - /dev/rdsk/c0t1d0s2 fmthard: New volume table of contents now in place. # prtvtoc /dev/rdsk/c0t0d0s2 | grep -v '^*' 0 2 00 16065 37592100 37608164 / 1 3 01 37608165 4209030 41817194 2 5 00 0 41881455 41881454 3 0 00 41817195 64260 41881454 8 1 01 0 16065 16064 # prtvtoc /dev/rdsk/c0t1d0s2 | grep -v '^*' 0 2 00 16065 37592100 37608164 1 3 01 37608165 4209030 41817194 2 5 00 0 41881455 41881454 3 0 00 41817195 64260 41881454 8 1 01 0 16065 16064

■ Creating MetaDB:

Code: Select all Expand view Collapse view: # metadb -f -a c0t0d0s3 # metadb -a c0t1d0s3 # metadb flags first blk block count a u 16 8192 /dev/dsk/c0t0d0s3 a u 16 8192 /dev/dsk/c0t1d0s3

■ Creating one-way mirror:

Code: Select all Expand view Collapse view: # metainit -f d11 1 1 c0t0d0s0 d11: Concat/Stripe is setup # metainit -f d21 1 1 c0t0d0s1 d21: Concat/Stripe is setup # metainit d10 -m d11 d10: Mirror is setup # metainit d20 -m d21 d20: Mirror is setup # metaroot d10 # echo 'set md:mirrored_root_flag=1' >> /etc/system # vi /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # fd - /dev/fd fd - no - /proc - /proc proc - no - /dev/md/dsk/d20 - - swap - no - # changed /dev/dsk/c0t0d0s1 to /dev/md/dsk/d20 /dev/md/dsk/d10 /dev/md/rdsk/d10 / ufs 1 no - /devices - /devices devfs - no - sharefs - /etc/dfs/sharetab sharefs - no - ctfs - /system/contract ctfs - no - objfs - /system/object objfs - no - swap - /tmp tmpfs - yes - # sync # init 6 # df -h / Filesystem size used avail capacity Mounted on /dev/md/dsk/d10 18G 3.9G 14G 23% / # swap -l swapfile dev swaplo blocks free /dev/md/dsk/d20 85,20 8 4209016 4209016 # dumpadm -d swap Dump content: kernel pages Dump device: /dev/md/dsk/d20 (swap) Savecore directory: /var/crash Savecore enabled: yes Save compressed: on

■ Creating two-way mirror:

Code: Select all Expand view Collapse view: # metainit d12 1 1 c0t1d0s0 d12: Concat/Stripe is setup # metainit d22 1 1 c0t1d0s1 d22: Concat/Stripe is setup # metattach d10 d12 d10: submirror d12 is attached # metattach d20 d22 d20: submirror d22 is attached # metastat d10 d10: Mirror Submirror 0: d11 State: Okay Submirror 1: d12 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 37592100 blocks (17 GB) d11: Submirror of d10 State: Okay Size: 37592100 blocks (17 GB) Stripe 0: Device Start Block Dbase State Reloc Hot Spare c0t0d0s0 0 No Okay Yes d12: Submirror of d10 State: Okay Size: 37592100 blocks (17 GB) Stripe 0: Device Start Block Dbase State Reloc Hot Spare c0t1d0s0 0 No Okay Yes Device Relocation Information: Device Reloc Device ID c0t0d0 Yes id1,sd@SATA_____VBOX_HARDDISK____VBe4b17a10-75090f7e c0t1d0 Yes id1,sd@SATA_____VBOX_HARDDISK____VB27121653-c2012b38 # metastat d20 d20: Mirror Submirror 0: d21 State: Okay Submirror 1: d22 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 4209030 blocks (2.0 GB) d21: Submirror of d20 State: Okay Size: 4209030 blocks (2.0 GB) Stripe 0: Device Start Block Dbase State Reloc Hot Spare c0t0d0s1 0 No Okay Yes d22: Submirror of d20 State: Okay Size: 4209030 blocks (2.0 GB) Stripe 0: Device Start Block Dbase State Reloc Hot Spare c0t1d0s1 0 No Okay Yes Device Relocation Information: Device Reloc Device ID c0t0d0 Yes id1,sd@SATA_____VBOX_HARDDISK____VBe4b17a10-75090f7e c0t1d0 Yes id1,sd@SATA_____VBOX_HARDDISK____VB27121653-c2012b38

■ Modifying the boot parameter:

Code: Select all Expand view Collapse view: # eeprom bootpath bootpath=/pci@0,0/pci8086,2829@d/disk@0,0:a # eeprom bootpath=/pseudo/md@0:0,10,blk

■ Installing a boot block on second disk:

Code: Select all Expand view Collapse view: # cd /boot/grub # installgrub stage1 stage2 /dev/rdsk/c0t1d0s0 stage1 written to partition 0 sector 0 (abs 16065) stage2 written to partition 0, 274 sectors starting at 50 (abs 16115)

■ Removing the first disk to force a failure:

Code: Select all Expand view Collapse view: # echo | format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t0d0 <drive type unknown> /pci@0,0/pci8086,2829@d/disk@0,0 1. c0t1d0 <ATA -VBOX HARDDISK -1.0 cyl 2607 alt 2 hd 255 sec 63> /pci@0,0/pci8086,2829@d/disk@1,0 Specify disk (enter its number): Specify disk (enter its number): # cfgadm -al | grep 'sata0/[01]\>' sata0/0 sata-port empty unconfigured ok sata0/1::dsk/c0t1d0 disk connected configured ok # metadb flags first blk block count Wm p l 16 8192 /dev/dsk/c0t0d0s3 a p luo 16 8192 /dev/dsk/c0t1d0s3 # metastat d10 d10: Mirror Submirror 0: d11 State: Needs maintenance Submirror 1: d12 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 37592100 blocks (17 GB) d11: Submirror of d10 State: Needs maintenance Invoke: metareplace d10 c0t0d0s0 <new device> Size: 37592100 blocks (17 GB) Stripe 0: Device Start Block Dbase State Reloc Hot Spare c0t0d0s0 0 No Maintenance Yes d12: Submirror of d10 State: Okay Size: 37592100 blocks (17 GB) Stripe 0: Device Start Block Dbase State Reloc Hot Spare c0t1d0s0 0 No Okay Yes Device Relocation Information: Device Reloc Device ID c0t0d0 Yes id1,sd@SATA_____VBOX_HARDDISK____VBe4b17a10-75090f7e c0t1d0 Yes id1,sd@SATA_____VBOX_HARDDISK____VB27121653-c2012b38 # metastat d20 d20: Mirror Submirror 0: d21 State: Needs maintenance Submirror 1: d22 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 4209030 blocks (2.0 GB) d21: Submirror of d20 State: Needs maintenance Invoke: metareplace d20 c0t0d0s1 <new device> Size: 4209030 blocks (2.0 GB) Stripe 0: Device Start Block Dbase State Reloc Hot Spare c0t0d0s1 0 No Maintenance Yes d22: Submirror of d20 State: Okay Size: 4209030 blocks (2.0 GB) Stripe 0: Device Start Block Dbase State Reloc Hot Spare c0t1d0s1 0 No Okay Yes Device Relocation Information: Device Reloc Device ID c0t0d0 Yes id1,sd@SATA_____VBOX_HARDDISK____VBe4b17a10-75090f7e c0t1d0 Yes id1,sd@SATA_____VBOX_HARDDISK____VB27121653-c2012b38

■Removing failed device information:

Code: Select all Expand view Collapse view: # metadb -f -d c0t0d0s3 # metadb flags first blk block count a p luo 16 8192 /dev/dsk/c0t1d0s3 # metadetach -f d10 d11 d10: submirror d11 is detached # metadetach -f d20 d21 d20: submirror d21 is detached # metaclear d11 d21 d11: Concat/Stripe is cleared d21: Concat/Stripe is cleared # devfsadm -Cv

■Adding new disk(20GB/Port 0):

Code: Select all Expand view Collapse view: # cfgadm -al | grep 'sata0/[01]\>' sata0/0::dsk/c0t0d0 disk connected configured unusable <<<<<---- 'My question is about this point.' sata0/1::dsk/c0t1d0 disk connected configured ok

I tried many tests but could not change unusable to ok.
I need help how to recognize new disk without system reboot at this point.

Thank you.

by **stes** » 17. Dec 2018, 12:19

From what you've said it sounds like the actual problem on SPARC with SCSI/FC is solved now.

When I tested OS Mirroring in VirtualBox, I got a problem when I removed the first disk and added it to new disk(same SATA port). This happened in VirtualBox and I needed to reboot the virtual machine for recognizing new disk. Of course, it worked fine on physical SPARC machines and VMware Workstations without system reboot.

This confirms my belief that it is necessary to be careful with comparison between SATA hotplug and FC/SCSI hotplug.

For what it's worth, I tried on a guest with Solaris 11.4 to replace a disk, and I was able to add a new disk (same SATA port) without rebooting the guest.

The Solaris 11.4 was installed on a mirrored disk configuration via the AI manifest:

Code: Select all Expand view Collapse view: <target> <disk in_vdev="mirror_vdev" in_zpool="rpool" whole_disk="true"> <disk_name name="c1t0d0" name_type="ctd"/> </disk> <disk in_vdev="mirror_vdev" in_zpool="rpool" whole_disk="true"> <disk_name name="c1t1d0" name_type="ctd"/> </disk> <logical> <zpool name="rpool" is_root="true"> <vdev name="mirror_vdev" redundancy="mirror"/>

Like that I obtained a mirrored root pool (with the operating system on 2 hot-pluggable SATA disks) :

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok root@pluto:~# zpool status pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t0d0 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 errors: No known data errors

Then at the VirtualBox level without preparing the guest operating system (i.e. without first detaching the disk and without unconfiguring it), I tried to remove a disk at the VirtualBox level :

Code: Select all Expand view Collapse view: root@pluto:~# zpool status pool: rpool state: DEGRADED status: One or more devices has been removed by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Online the device using 'zpool online' or replace the device with 'zpool replace'. scan: none requested config: NAME STATE READ WRITE CKSUM rpool DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 c1t0d0 ONLINE 0 0 0 c1t1d0 REMOVED 0 0 0 errors: No known data errors root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1 sata-port empty unconfigured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok

Note that the disk went cleanly to status 'unconfigured' and condition OK and receptable empty (apparently by the removal action at the VirtualBox level).

Then I manually removed the REMOVED disk:

Code: Select all Expand view Collapse view: root@pluto:~# zpool detach rpool c1t1d0 root@pluto:~# zpool status pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c1t0d0 ONLINE 0 0 0

Also when I added a new disk in the same SATA slot this went very smoothly:

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok

Then attach and resilver (everything without reboot) :

Code: Select all Expand view Collapse view: root@pluto:~# zpool attach rpool c1t0d0 c1t1d0 Make sure to wait until resilver is done before rebooting.

and back to

Code: Select all Expand view Collapse view: root@pluto:~# zpool status rpool pool: rpool state: ONLINE scan: resilvered 2.57G in 1m33s with 0 errors on Mon Dec 17 11:00:39 2018 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t0d0 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 errors: No known data errors

So at least with a guest running Solaris 11.4 and host OS Solaris 11.4 (VirtualBox 6.0.0RC1) I am able to remove a SATA hotplug disk and add a new SATA hotplug disk on the same port without reboot of the guest.

by **sunwizard** » 17. Dec 2018, 15:32

>>> Hello. stes.

>>> Thanks for your hard work and time.

>>> I had rpool and SVM mirroring tests. I also tested the recognition after removing the second disk on VirtualBox. I handled the second disk failure well.
>>> This is a problem when removing the first disk on VirtualBox.

>>> I am so sorry. I should have mentioned it to you in advance.

by **stes** » 17. Dec 2018, 16:21

No problem , it is an interesting question, and I'm just trying to help out.

So to summarize, you can add a new disk (SATA hotplug) and you can also remove the second disk without a problem, where "second" is it the disk that is not the active bootdevice ?

When you try to remove the 'first disk' is it also the boot device ?

Code: Select all Expand view Collapse view: root@pluto:~# prtconf -v|sed -n '/bootpath/,/value/p' name='bootpath' type=string items=1 value='/pci@0,0/pci8086,2829@d/disk@0,0:b'

where the disks are:

Code: Select all Expand view Collapse view: root@pluto:~# echo | format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c1t0d0 <VBOX-HARDDISK-1.0-16.00GB> /pci@0,0/pci8086,2829@d/disk@0,0 1. c1t1d0 <VBOX-HARDDISK-1.0-16.00GB> /pci@0,0/pci8086,2829@d/disk@1,0 Specify disk (enter its number): Specify disk (enter its number):

by **stes** » 17. Dec 2018, 16:49

On Solaris 11.4 it seems I can remove the bootpath (the "first disk") and it seems to work.

I did not have to boot from an "alternate disk" to start with (which would be reasonable to try).

For a Solaris 11.4 guest :

Code: Select all Expand view Collapse view: root@pluto:~# prtconf -v|sed -n '/bootpath/,/value/p' name='bootpath' type=string items=1 value='/pci@0,0/pci8086,2829@d/disk@0,0:b' root@pluto:~# zpool status pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t0d0 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 errors: No known data errors

Then I remove at the VirtualBox level the first disk :

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0 sata-port empty unconfigured ok sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok root@pluto:~# prtconf -v|sed -n '/bootpath/,/value/p' name='bootpath' type=string items=1 value='/pci@0,0/pci8086,2829@d/disk@0,0:b'

Note that "Receptacle" is "empty" for sata0/0.

At that point I have:

Code: Select all Expand view Collapse view: root@pluto:~# zpool status rpool pool: rpool state: DEGRADED status: One or more devices has been removed by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Online the device using 'zpool online' or replace the device with 'zpool replace'. scan: none requested config: NAME STATE READ WRITE CKSUM rpool DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 c1t0d0 REMOVED 0 0 0 c1t1d0 ONLINE 0 0 0

Then I add on the same port 0 a new disk (without reboot of the guest) and it works for me:

I get:

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok

Then I resilver the mirror as follows (not sure whether this is the easiest best way) :

Code: Select all Expand view Collapse view: root@pluto:~# zpool detach rpool c1t0d0

and

Code: Select all Expand view Collapse view: root@pluto:~# zpool attach rpool c1t1d0 c1t0d0 Make sure to wait until resilver is done before rebooting. root@pluto:~# zpool status pool: rpool state: DEGRADED status: One or more devices are currently being resilvered. The pool will continue to function in a degraded state. action: Wait for the resilver to complete. Run 'zpool status -v' to see device specific details. scan: resilver in progress since Mon Dec 17 15:32:14 2018 2.80G scanned out of 3.58G at 719M/s, 1s to go 0 resilvered config: NAME STATE READ WRITE CKSUM rpool DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 c1t1d0 ONLINE 0 0 0 c1t0d0 DEGRADED 0 0 0 (resilvering)

which results after a while in the following setup:

Code: Select all Expand view Collapse view: root@pluto:~# zpool status pool: rpool state: ONLINE scan: resilvered 2.58G in 1m32s with 0 errors on Mon Dec 17 15:33:46 2018 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 c1t0d0 ONLINE 0 0 0

I believe that "zpool online" and "zpool clear" should be used instead.

However in this test I can remove the first disk and I can add a new SATA hotplug disk at the same (first) port.

These results are on Solaris 11.4 host and guest also Solaris 11.4.

by **stes** » 17. Dec 2018, 17:32

An alternative scenario was to boot with only the second disk (so that I boot from an alternate disk).

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0 sata-port empty unconfigured ok sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok root@pluto:~# prtconf -v|sed -n '/bootpath/,/value/p' name='bootpath' type=string items=1 value='/pci@0,0/pci8086,2829@d/disk@1,0:b'

In that case the bootpath is the disk@1,0 (because I removed the first disk prior to booting).

So that

Code: Select all Expand view Collapse view: root@pluto:~# zpool status pool: rpool state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or 'fmadm repaired', or replace the device with 'zpool replace'. Run 'zpool status -v' to see device specific details. scan: none requested config: NAME STATE READ WRITE CKSUM rpool DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 c1t0d0 UNAVAIL 0 0 0 c1t1d0 ONLINE 0 0 0

In that case when I add (without rebooting the guest) the hotplug original disk in the same port as where it was, I have to configure it manually it seems:

Code: Select all Expand view Collapse view: root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0 sata-port disconnected unconfigured failed sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0 sata-port disconnected unconfigured failed sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok root@pluto:~# cfgadm -x sata_port_activate sata0/0 Activate the port: /devices/pci@0,0/pci8086,2829@d:0 This operation will enable activity on the SATA port Continue (yes/no)? yes root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0 disk connected unconfigured unknown sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok root@pluto:~# cfgadm -c configure sata0/0 root@pluto:~# cfgadm -al sata0 Ap_Id Type Receptacle Occupant Condition sata0/0::dsk/c1t0d0 disk connected configured ok sata0/1::dsk/c1t1d0 disk connected configured ok sata0/2 sata-port empty unconfigured ok sata0/3 sata-port empty unconfigured ok

After this (where I added the initial disk back in the same port as where it was during installation),

Code: Select all Expand view Collapse view: root@pluto:~# zpool clear rpool c1t0d0 root@pluto:~# zpool online rpool c1t0d0 root@pluto:~# zpool status pool: rpool state: ONLINE scan: resilvered 29.9M in 2s with 0 errors on Mon Dec 17 16:27:47 2018 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t0d0 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 errors: No known data errors

So this is bringing the first disk online again without reboot.

by **stes** » 17. Dec 2018, 17:47

What could be interesting in the Solaris 10 case is to boot without the first disk, so that the bootpath is like

Code: Select all Expand view Collapse view: prtconf -v|sed -n '/bootpath/,/value/p' name='bootpath' type=string items=1 value='/pci@0,0/pci8086,2829@d/disk@1,0:b'

that is the bootpath is the second disk, then try to add the first disk in the Solaris 10 case.

Then proceed without reboot to remove the first disk again and replace it with a new disk in the same port.

by **socratis** » 17. Dec 2018, 20:33

@sunwizard
The messages from "stes" are rather lengthy and explanatory. You don't have to quote them whole! There's a "Post Reply" button under the last message, no need to use the "Quote" one. It makes following up on the thread like reading "War and Peace"...

virtualbox.org

How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Who is online