How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Discussions related to using VirtualBox on Solaris hosts.

How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby sunwizard » 4. Dec 2018, 09:56

Hello Guys.

I had a question while testing Solaris 10 x86 on Oracle VirtualBox.

I can use various kinds of disk controllers in a virtual machine on VirtualBox. But I have checked that only SATA controllers support Hot Plug function.

I have used the devfsadm command on the Solaris OS (SPARC Server) to recognize a new disk. However, this command did not recognize the new disk in Solaris 10 x86 on VirtualBox.
So I used the cfgadm command to recognize the new disk.

# cfgadm -x sata_port_activate AP_ID
# cfgadm -c configure AP_ID

To perform some tests, such as OS disk mirroring, I removed the first disk and added a new disk to the same port.
Then, I had the following results and the disk that was replaced was not recognized.

-bash-3.2# cfgadm -al | grep 'sata0/[01]\>'
sata0/0 disk connected configured unusable
sata0/1::dsk/c0t1d0 disk connected configured ok

Normally I use the following commands to recognize the disk in Solaris 10 SPARC.

# devfsadm -Cv -c disk
# devfsadm -c disk

But not recognized by the above commands and could not be changed to ok condition by using the cfgadm command.

- unconfigure/configure
- disconnect/connect
- disconnect/delete /dev/(r)dsk/c#t# files/connect/configure

Of course, rebooting the Solaris will save the problem, but I want it to solve without rebooting.

How can I solve this problem? I need your help.

I'm not familiar with English. But thanks for reading so far.
sunwizard
 
Posts: 5
Joined: 4. Dec 2018, 08:54

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby stes » 12. Dec 2018, 10:19

The current release of Solaris is 11.4.

I suspect the support in Solaris 10 would be more limited, for example at
Code: Select all   Expand viewCollapse view
https://www.oracle.com/technetwork/server-storage/solaris10/overview/sata0-136306.html

they say that the Solaris SPARC has no SATA support (only PATA).

Did you use SATA on Solaris SPARC ? I suppose not, you are comparing different technologies, for example "fibre channel" (FC) disk addition is much more advanced, and I suspect on SPARC you would have been using Emulex or Qlogic HBA with FC disk addition.

So it is a comparison "doomed to fail" because you'd have to see how it behaves on SPARC with SATA, which is not what you probably had to start with (I think you used FC on SPARC ?)

And VirtualBox as far as I know does not offer a "virtual" Emulex or Qlogic adapter to a guest.

On Solaris 11.4 there is a manpage:
Code: Select all   Expand viewCollapse view
man cfgadm_sata

about SATA hotplug.

Anyway on Solaris 11.4 I tried on VirtualBox to add a SATA disk (hotplug) and then before the disk addition on a SATA controller with 2 ports (port count = 2) one port has the boot disk the 2nd port nothing.
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al | more
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1                        sata-port    disconnected unconfigured failed

After adding the disk online (and without reboot of the guest):
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -x sata_port_activate sata0/1
root@pluto:~# cfgadm -c configure sata0/1
root@pluto:~# cfgadm -al | head -4
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
usb0/1                         usb-input    connected    configured   ok

and then I can use the disk on the Solaris 11.4 guest
Code: Select all   Expand viewCollapse view
format -e
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <VBOX-HARDDISK-1.0-16.00GB>
          /pci@0,0/pci8086,2829@d/disk@0,0
       1. c1t1d0 <VBOX-HARDDISK-1.0-2.00GB>
          /pci@0,0/pci8086,2829@d/disk@1,0
Specify disk (enter its number): ^C
root@pluto:~# zpool create qpool c1t1d0
root@pluto:~# zpool list
NAME    SIZE  ALLOC   FREE  CAP  DEDUP  HEALTH  ALTROOT
qpool  1.98G   140K  1.98G   0%  1.00x  ONLINE  -
rpool  15.6G  3.67G  12.0G  23%  1.00x  ONLINE  -

Adding a second disk (to get a total of 3 disks) online fails because the SATA controller has only 2 ports I think, because it is a controller that has only sata0/0 and sata0/1 slots available.

In any case I'm pessimistic about the goal of simulating this operation on Solaris 10,
because I'm under the impression that you are trying to get the advanced features of Solaris SPARC hotadd fibre channel under the simple SATA virtualbox guest ?
stes
 
Posts: 142
Joined: 28. Apr 2018, 11:07

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby stes » 12. Dec 2018, 11:21

In any case in a Fibre Channel or FICON environment it has been many years completely normal to add hotplug 128 LUNS or so on a HBA.

Or with SCSI 15 LUNs or more and certainly SAS probably has plenty of features in this area.

So it is a bit unfair to compare the "high end" SPARC platform with its plugin possibilties to low-end VirtualBox guest with SATA.

The Solaris 11.4 has a package called
Code: Select all   Expand viewCollapse view
# pkg list pkg:/system/dynamic-reconfiguration
system/dynamic-reconfiguration                    11.4-11.4.0.0.1.15.0       i--

with the cfgadm_sata support in it (among the cfgadm_scsi which VirtualBox does not seem to support).

By default when I create a guest in VirtualBox it has SATA controller port count set to 2.

When I set the port count of the SATA controller to 4.

And reinstall the Solaris 11.4 guest (via the AI automated install).

I get:
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1                        sata-port    empty        unconfigured ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok

Then if I add "hotplug" 2 disks without shutting down the guest, I get
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1                        sata-port    disconnected unconfigured failed
sata0/2                        sata-port    disconnected unconfigured failed
sata0/3                        sata-port    empty        unconfigured ok

Then the disks are correctly added it seems
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -x sata_port_activate sata0/1
Activate the port: /devices/pci@0,0/pci8086,2829@d:1
This operation will enable activity on the SATA port
Continue (yes/no)? yes
root@pluto:~# cfgadm -x sata_port_activate sata0/2
Activate the port: /devices/pci@0,0/pci8086,2829@d:2
This operation will enable activity on the SATA port
Continue (yes/no)? yes

and
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1                        disk         connected    unconfigured unknown
sata0/2                        disk         connected    unconfigured unknown
sata0/3                        sata-port    empty        unconfigured ok

and
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -c configure sata0/1
root@pluto:~# cfgadm -c configure sata0/2
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2::dsk/c1t2d0            disk         connected    configured   ok
sata0/3                        sata-port    empty        unconfigured ok

so that
Code: Select all   Expand viewCollapse view
root@pluto:~# format -e
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <VBOX-HARDDISK-1.0-16.00GB>
          /pci@0,0/pci8086,2829@d/disk@0,0
       1. c1t1d0 <VBOX-HARDDISK-1.0-7.00GB>
          /pci@0,0/pci8086,2829@d/disk@1,0
       2. c1t2d0 <VBOX-HARDDISK-1.0-2.00GB>
          /pci@0,0/pci8086,2829@d/disk@2,0
Specify disk (enter its number): ^C
stes
 
Posts: 142
Joined: 28. Apr 2018, 11:07

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby stes » 12. Dec 2018, 14:43

It appears you can set the port count to 30 in VirtualBox and then install Solaris 11.4 (on the boot disk) and then you get:
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1                        sata-port    empty        unconfigured ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok
sata0/4                        sata-port    empty        unconfigured ok
sata0/5                        sata-port    empty        unconfigured ok
sata0/6                        sata-port    empty        unconfigured ok
sata0/7                        sata-port    empty        unconfigured ok
sata0/8                        sata-port    empty        unconfigured ok
sata0/9                        sata-port    empty        unconfigured ok
sata0/10                       sata-port    empty        unconfigured ok
sata0/11                       sata-port    empty        unconfigured ok
sata0/12                       sata-port    empty        unconfigured ok
sata0/13                       sata-port    empty        unconfigured ok
sata0/14                       sata-port    empty        unconfigured ok
sata0/15                       sata-port    empty        unconfigured ok
sata0/16                       sata-port    empty        unconfigured ok
sata0/17                       sata-port    empty        unconfigured ok
sata0/18                       sata-port    empty        unconfigured ok
sata0/19                       sata-port    empty        unconfigured ok
sata0/20                       sata-port    empty        unconfigured ok
sata0/21                       sata-port    empty        unconfigured ok
sata0/22                       sata-port    empty        unconfigured ok
sata0/23                       sata-port    empty        unconfigured ok
sata0/24                       sata-port    empty        unconfigured ok
sata0/25                       sata-port    empty        unconfigured ok
sata0/26                       sata-port    empty        unconfigured ok
sata0/27                       sata-port    empty        unconfigured ok
sata0/28                       sata-port    empty        unconfigured ok
sata0/29                       sata-port    empty        unconfigured ok

This means that you possibly have a lot of empty slots to add virtual SATA disks (attach) them to it online (hotplug).

I haven't tried adding 28 disks, but for example after:
Code: Select all   Expand viewCollapse view
 VBoxManage storageattach pluto  --storagectl SATA --medium "5d63e15c-5dd8-4c63-8c67-f16ea8b2ade0" --port 28 --type hdd

to add a disk to sata0/29 I get on the guest:
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0/28
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/28                       sata-port    disconnected unconfigured failed

and then
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -x sata_port_activate sata0/28
Activate the port: /devices/pci@0,0/pci8086,2829@d:28
This operation will enable activity on the SATA port
Continue (yes/no)? y
root@pluto:~# cfgadm -al sata0/28
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/28                       disk         connected    unconfigured unknown

if I run at this point
Code: Select all   Expand viewCollapse view
root@pluto:~# ls /dev/rdsk
c1t0d0    c1t0d0p1  c1t0d0p3  c1t0d0s0  c1t0d0s2  c1t0d0s4  c1t0d0s6
c1t0d0p0  c1t0d0p2  c1t0d0p4  c1t0d0s1  c1t0d0s3  c1t0d0s5

and then
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -c configure sata0/28
root@pluto:~# ls /dev/rdsk
c1t0d0     c1t0d0p3   c1t0d0s2   c1t0d0s6   c1t28d0p2  c1t28d0s1  c1t28d0s5
c1t0d0p0   c1t0d0p4   c1t0d0s3   c1t28d0    c1t28d0p3  c1t28d0s2  c1t28d0s6
c1t0d0p1   c1t0d0s0   c1t0d0s4   c1t28d0p0  c1t28d0p4  c1t28d0s3
c1t0d0p2   c1t0d0s1   c1t0d0s5   c1t28d0p1  c1t28d0s0  c1t28d0s4
root@pluto:~# cfgadm -al sata0/28
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/28::dsk/c1t28d0          disk         connected    configured   ok
sata0/28: No matching library found

So there is a strange message "No matching library found" which possibly is a Solaris 11.4 bug

but it seems to work because the disk device names under /dev/rdsk were automatically created (without explicitley running devfsadm).

I think you would run devfsadm if you remove the device files under /dev/rdsk for some reason (to cleanup) and then want to regenerate the device files.

This is for example if you want to renumber the device files I think, but it was not required for me to run devfsadm.

In any case it seems to work fine
Code: Select all   Expand viewCollapse view
root@pluto:~# format -e
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <VBOX-HARDDISK-1.0-16.00GB>
          /pci@0,0/pci8086,2829@d/disk@0,0
       1. c1t28d0 <VBOX-HARDDISK-1.0-7.00GB>
          /pci@0,0/pci8086,2829@d/disk@1c,0

and
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool create -f apool c1t28d0
root@pluto:~# zpool status apool
  pool: apool
 state: ONLINE
  scan: none requested
config:

        NAME       STATE      READ WRITE CKSUM
        apool      ONLINE        0     0     0
          c1t28d0  ONLINE        0     0     0

errors: No known data errors

The -f option with the zpool create was because I had already used the disk before in a different pool.
stes
 
Posts: 142
Joined: 28. Apr 2018, 11:07

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby sunwizard » 16. Dec 2018, 09:19

>> Hello, stes.

>> I use SCSI, FC, SAS type on Solaris SPARC.
>> I am using a virtualization program to simply test Solaris 10.
>> When I tested OS Mirroring in VirtualBox, I got a problem when I removed the first disk and added it to new disk(same SATA port).
>> This happened in VirtualBox and I needed to reboot the virtual machine for recognizing new disk. Of course, it worked fine on physical SPARC machines and VMware Workstations without system reboot.
>> So I posted the question.

>> Thanks for your explanation.
sunwizard
 
Posts: 5
Joined: 4. Dec 2018, 08:54

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby stes » 16. Dec 2018, 17:04

Well for completeness the same test for me works on host Solaris 11.4 with VirtualBox 6.0.0 RC1 and a guest running the (old version of) Solaris 10 :

On a guest running the following version of Solaris 10 with a UFS root filesystem :
Code: Select all   Expand viewCollapse view
# cat /etc/release
                    Oracle Solaris 10 8/11 s10x_u10wos_17b X86
  Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved.
                            Assembled 23 August 2011
# uname -a
SunOS neptune 5.10 Generic_147441-01 i86pc i386 i86pc

The root filesystem of the guest is UFS (not ZFS) :
Code: Select all   Expand viewCollapse view
/dev/dsk/c0t0d0s0       /dev/rdsk/c0t0d0s0      /       ufs     1       no     
-
/dev/dsk/c0t0d0s6       /dev/rdsk/c0t0d0s6      /usr    ufs     1       no     
-
/dev/dsk/c0t0d0s1       /dev/rdsk/c0t0d0s1      /var    ufs     1       no     
-

Then I have in VirtualBox the SATA port count set to 4 :
Code: Select all   Expand viewCollapse view
# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c0t0d0            disk         connected    configured   ok
sata0/1::dsk/c0t1d0            cd/dvd       connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok

So there are two empty ports on the SATA controller for "online" hotplug adding disks.

Initially on port sata0/2 there is nothing, and then when I add in VirtualBox a disk drive:
Code: Select all   Expand viewCollapse view
# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c0t0d0            disk         connected    configured   ok
sata0/1::dsk/c0t1d0            cd/dvd       connected    configured   ok
sata0/2                        sata-port    disconnected unconfigured failed
sata0/3                        sata-port    empty        unconfigured ok

then without a reboot of the guest :
Code: Select all   Expand viewCollapse view
# cfgadm -x sata_port_activate sata0/2
Activate the port: /devices/pci@0,0/pci8086,2829@d:2
This operation will enable activity on the SATA port
Continue (yes/no)? yes

this changes the status to:
Code: Select all   Expand viewCollapse view
# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c0t0d0            disk         connected    configured   ok
sata0/1::dsk/c0t1d0            cd/dvd       connected    configured   ok
sata0/2                        disk         connected    unconfigured unknown
sata0/3                        sata-port    empty        unconfigured ok

at that point the device links under /dev do not yet exist:
Code: Select all   Expand viewCollapse view
# ls /dev/rdsk
c0t0d0p0   c0t0d0s1   c0t0d0s15  c0t0d0s7   c0t1d0p3   c0t1d0s12  c0t1d0s4
c0t0d0p1   c0t0d0s10  c0t0d0s2   c0t0d0s8   c0t1d0p4   c0t1d0s13  c0t1d0s5
c0t0d0p2   c0t0d0s11  c0t0d0s3   c0t0d0s9   c0t1d0s0   c0t1d0s14  c0t1d0s6
c0t0d0p3   c0t0d0s12  c0t0d0s4   c0t1d0p0   c0t1d0s1   c0t1d0s15  c0t1d0s7
c0t0d0p4   c0t0d0s13  c0t0d0s5   c0t1d0p1   c0t1d0s10  c0t1d0s2   c0t1d0s8
c0t0d0s0   c0t0d0s14  c0t0d0s6   c0t1d0p2   c0t1d0s11  c0t1d0s3   c0t1d0s9

but when I configure the port :
Code: Select all   Expand viewCollapse view
# cfgadm -c configure sata0/2

the disk is available to the Solaris 10 guest without reboot and the device links under /dev were created:
Code: Select all   Expand viewCollapse view
# ls /dev/rdsk
c0t0d0p0   c0t0d0s12  c0t0d0s7   c0t1d0s1   c0t1d0s4   c0t2d0p3   c0t2d0s15
c0t0d0p1   c0t0d0s13  c0t0d0s8   c0t1d0s10  c0t1d0s5   c0t2d0p4   c0t2d0s2
c0t0d0p2   c0t0d0s14  c0t0d0s9   c0t1d0s11  c0t1d0s6   c0t2d0s0   c0t2d0s3
c0t0d0p3   c0t0d0s15  c0t1d0p0   c0t1d0s12  c0t1d0s7   c0t2d0s1   c0t2d0s4
c0t0d0p4   c0t0d0s2   c0t1d0p1   c0t1d0s13  c0t1d0s8   c0t2d0s10  c0t2d0s5
c0t0d0s0   c0t0d0s3   c0t1d0p2   c0t1d0s14  c0t1d0s9   c0t2d0s11  c0t2d0s6
c0t0d0s1   c0t0d0s4   c0t1d0p3   c0t1d0s15  c0t2d0p0   c0t2d0s12  c0t2d0s7
c0t0d0s10  c0t0d0s5   c0t1d0p4   c0t1d0s2   c0t2d0p1   c0t2d0s13  c0t2d0s8
c0t0d0s11  c0t0d0s6   c0t1d0s0   c0t1d0s3   c0t2d0p2   c0t2d0s14  c0t2d0s9

Also the disk is available in Solaris 10 for using it in a ZFS pool for example (without reboot) :
Code: Select all   Expand viewCollapse view
# format -e
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <DEFAULT cyl 2085 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,2829@d/disk@0,0
       1. c0t2d0 <DEFAULT cyl 2086 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,2829@d/disk@2,0
Specify disk (enter its number): ^C
# zpool create mypool c0t2d0

So the Solaris 10 guest behaves like the Solaris 11.4 guest for me ...
stes
 
Posts: 142
Joined: 28. Apr 2018, 11:07

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby stes » 16. Dec 2018, 18:18

Sunwizard wrote:
I removed the first disk and added a new disk to the same port.

How did you do this exactly ? I think the cfgadm_sata manpage gives a clue here :
The emergency "sata_port_deactivate" operation is not supported on
ports with attached disks containing critical partitions such as root
(/), /usr, swap, or /var. The deactivate operation should not be
attempted on such ports. Incorrect usage can result in a system hang
and require a reboot.

Also the cfgadm -al output that you posted has the "unusable" flag ...

For example on a 11.4 guest I tried the following to replace a disk :
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1                        sata-port    empty        unconfigured ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok

then add two disks to make a mirror:
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -x sata_port_activate sata0/1
Activate the port: /devices/pci@0,0/pci8086,2829@d:1
This operation will enable activity on the SATA port
Continue (yes/no)? yes
root@pluto:~# cfgadm -x sata_port_activate sata0/2
Activate the port: /devices/pci@0,0/pci8086,2829@d:2
This operation will enable activity on the SATA port
Continue (yes/no)? yes
root@pluto:~# cfgadm -c configure sata0/1
root@pluto:~# cfgadm -c configure sata0/2

so the mirror is :
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool create mymirror mirror c1t1d0 c1t2d0
root@pluto:~# zpool status mymirror
  pool: mymirror
 state: ONLINE
  scan: none requested
config:

        NAME        STATE      READ WRITE CKSUM
        mymirror    ONLINE        0     0     0
          mirror-0  ONLINE        0     0     0
            c1t1d0  ONLINE        0     0     0
            c1t2d0  ONLINE        0     0     0

errors: No known data errors

then before "removing the disk" in VirtualBox I'm preparing the operating system (guest) first to remove the disk:
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool detach mymirror c1t2d0
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2::dsk/c1t2d0            disk         connected    configured   ok
sata0/3                        sata-port    empty        unconfigured ok
root@pluto:~# 
root@pluto:~# cfgadm -c unconfigure sata0/2
Unconfigure the device at: /devices/pci@0,0/pci8086,2829@d:2
This operation will suspend activity on the SATA device
Continue (yes/no)? yes
root@pluto:~# cfgadm -al sata0/2
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/2                        disk         connected    unconfigured ok

Did you unconfigure the disk before removing it ?

I've just tested that I can replace the disk by a different disk on the same port without a reboot.
(guest Solaris 11.4)

However the test pool I used above is a simple "mymirror" test zfs pool , not a pool for operating system mirroring,
with the root / and /var etc. on the disk, the cfgadm_sata manpage indicates that there are some limitations.

Provided proper unconfigure of the disk , I did not have to reboot to replace a disk in a mirror,
and attach it again afterwards (with a new disk).

Also the manpage cfgadm_sata warns:
If the SATA connectors are the hot-pluggable type and the SATA con-
troller supports hotplugging, unplugging a device without unconfiguring
it may result in system hang or data loss.
stes
 
Posts: 142
Joined: 28. Apr 2018, 11:07

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby sunwizard » 16. Dec 2018, 19:22

Hello, stes.

Here is my work for OS mirroring:

■ Solaris 10 x86 Info:
Code: Select all   Expand viewCollapse view
# more /etc/release
                    Oracle Solaris 10 1/13 s10x_u11wos_24a X86
  Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights reserved.
                            Assembled 17 January 2013
# uname -a
SunOS solaris101 5.10 Generic_147148-26 i86pc i386 i86pc

# df -h /
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0t0d0s0       18G   3.9G    14G    23%    /

■ OS disk information and partition table:
Code: Select all   Expand viewCollapse view
OS Disk : SATA 20GB (Port 0)
CD/DVD : SATA (Port 29)

# cfgadm -al | grep 'sata0/0\>'
sata0/0::dsk/c0t0d0            disk         connected    configured   ok

# echo | format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <ATA    -VBOX HARDDISK  -1.0  cyl 2607 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,2829@d/disk@0,0
Specify disk (enter its number): Specify disk (enter its number):

Part      Tag    Flag     Cylinders        Size            Blocks
  0       root    wm       1 - 2340       17.93GB    (2340/0/0) 37592100
  1       swap    wu    2341 - 2602        2.01GB    (262/0/0)   4209030
  2     backup    wm       0 - 2606       19.97GB    (2607/0/0) 41881455
  3 unassigned    wm    2603 - 2606       31.38MB    (4/0/0)       64260
  4 unassigned    wm       0               0         (0/0/0)           0
  5 unassigned    wm       0               0         (0/0/0)           0
  6 unassigned    wm       0               0         (0/0/0)           0
  7 unassigned    wm       0               0         (0/0/0)           0
  8       boot    wu       0 -    0        7.84MB    (1/0/0)       16065
  9 unassigned    wm       0               0         (0/0/0)           0

■ Adding new disk (20GB/Port 1):
Code: Select all   Expand viewCollapse view
# cfgadm -al | grep 'sata0/[01]\>'
sata0/0::dsk/c0t0d0            disk         connected    configured   ok
sata0/1                        sata-port    disconnected unconfigured failed

# cfgadm -x sata_port_activate sata0/1
Activate the port: /devices/pci@0,0/pci8086,2829@d:1
This operation will enable activity on the SATA port
Continue (yes/no)? yes

# cfgadm -c configure sata0/1
# cfgadm -al | grep 'sata0/[01]\>'
sata0/0::dsk/c0t0d0            disk         connected    configured   ok
sata0/1::dsk/c0t1d0            disk         connected    configured   ok

# echo | format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <ATA    -VBOX HARDDISK  -1.0  cyl 2607 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,2829@d/disk@0,0
       1. c0t1d0 <ATA    -VBOX HARDDISK  -1.0  cyl 2608 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,2829@d/disk@1,0
Specify disk (enter its number): Specify disk (enter its number):

■ Labeling new disk by using the prtvtoc and fmthard commands:
Code: Select all   Expand viewCollapse view
# fdisk -B /dev/rdsk/c0t1d0p0
# prtvtoc /dev/rdsk/c0t0d0s2 | fmthard -s - /dev/rdsk/c0t1d0s2
fmthard:  New volume table of contents now in place.

# prtvtoc /dev/rdsk/c0t0d0s2 | grep -v '^*'
       0      2    00      16065  37592100  37608164   /
       1      3    01   37608165   4209030  41817194
       2      5    00          0  41881455  41881454
       3      0    00   41817195     64260  41881454
       8      1    01          0     16065     16064

# prtvtoc /dev/rdsk/c0t1d0s2 | grep -v '^*'
       0      2    00      16065  37592100  37608164
       1      3    01   37608165   4209030  41817194
       2      5    00          0  41881455  41881454
       3      0    00   41817195     64260  41881454
       8      1    01          0     16065     16064

■ Creating MetaDB:
Code: Select all   Expand viewCollapse view
# metadb -f -a c0t0d0s3
# metadb -a c0t1d0s3
# metadb
        flags           first blk       block count
     a        u         16              8192            /dev/dsk/c0t0d0s3
     a        u         16              8192            /dev/dsk/c0t1d0s3

■ Creating one-way mirror:
Code: Select all   Expand viewCollapse view
# metainit -f d11 1 1 c0t0d0s0
d11: Concat/Stripe is setup
# metainit -f d21 1 1 c0t0d0s1
d21: Concat/Stripe is setup

# metainit d10 -m d11
d10: Mirror is setup
# metainit d20 -m d21
d20: Mirror is setup

# metaroot d10
# echo 'set md:mirrored_root_flag=1' >> /etc/system

# vi /etc/vfstab

#device         device          mount           FS      fsck    mount   mount
#to mount       to fsck         point           type    pass    at boot options
#
fd      -       /dev/fd fd      -       no      -
/proc   -       /proc   proc    -       no      -
/dev/md/dsk/d20 -       -       swap    -       no      -                                 # changed /dev/dsk/c0t0d0s1 to /dev/md/dsk/d20
/dev/md/dsk/d10 /dev/md/rdsk/d10        /       ufs     1       no      -     
/devices        -       /devices        devfs   -       no      -
sharefs -       /etc/dfs/sharetab       sharefs -       no      -
ctfs    -       /system/contract        ctfs    -       no      -
objfs   -       /system/object  objfs   -       no      -
swap    -       /tmp    tmpfs   -       yes     -

# sync
# init 6

# df -h /
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d10         18G   3.9G    14G    23%    /

# swap -l
swapfile             dev  swaplo blocks   free
/dev/md/dsk/d20     85,20      8 4209016 4209016

# dumpadm -d swap
      Dump content: kernel pages
       Dump device: /dev/md/dsk/d20 (swap)
Savecore directory: /var/crash
  Savecore enabled: yes
   Save compressed: on

■ Creating two-way mirror:
Code: Select all   Expand viewCollapse view
# metainit d12 1 1 c0t1d0s0
d12: Concat/Stripe is setup
# metainit d22 1 1 c0t1d0s1
d22: Concat/Stripe is setup

# metattach d10 d12
d10: submirror d12 is attached
# metattach d20 d22
d20: submirror d22 is attached

# metastat d10
d10: Mirror
    Submirror 0: d11
      State: Okay
    Submirror 1: d12
      State: Okay
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 37592100 blocks (17 GB)

d11: Submirror of d10
    State: Okay
    Size: 37592100 blocks (17 GB)
    Stripe 0:
        Device     Start Block  Dbase        State Reloc Hot Spare
        c0t0d0s0          0     No            Okay   Yes

d12: Submirror of d10
    State: Okay
    Size: 37592100 blocks (17 GB)
    Stripe 0:
        Device     Start Block  Dbase        State Reloc Hot Spare
        c0t1d0s0          0     No            Okay   Yes

Device Relocation Information:
Device   Reloc  Device ID
c0t0d0   Yes    id1,sd@SATA_____VBOX_HARDDISK____VBe4b17a10-75090f7e
c0t1d0   Yes    id1,sd@SATA_____VBOX_HARDDISK____VB27121653-c2012b38

# metastat d20
d20: Mirror
    Submirror 0: d21
      State: Okay
    Submirror 1: d22
      State: Okay
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 4209030 blocks (2.0 GB)

d21: Submirror of d20
    State: Okay
    Size: 4209030 blocks (2.0 GB)
    Stripe 0:
        Device     Start Block  Dbase        State Reloc Hot Spare
        c0t0d0s1          0     No            Okay   Yes

d22: Submirror of d20
    State: Okay
    Size: 4209030 blocks (2.0 GB)
    Stripe 0:
        Device     Start Block  Dbase        State Reloc Hot Spare
        c0t1d0s1          0     No            Okay   Yes

Device Relocation Information:
Device   Reloc  Device ID
c0t0d0   Yes    id1,sd@SATA_____VBOX_HARDDISK____VBe4b17a10-75090f7e
c0t1d0   Yes    id1,sd@SATA_____VBOX_HARDDISK____VB27121653-c2012b38

■ Modifying the boot parameter:
Code: Select all   Expand viewCollapse view
# eeprom bootpath
bootpath=/pci@0,0/pci8086,2829@d/disk@0,0:a

# eeprom bootpath=/pseudo/md@0:0,10,blk

■ Installing a boot block on second disk:
Code: Select all   Expand viewCollapse view
# cd /boot/grub
# installgrub stage1 stage2 /dev/rdsk/c0t1d0s0
stage1 written to partition 0 sector 0 (abs 16065)
stage2 written to partition 0, 274 sectors starting at 50 (abs 16115)

■ Removing the first disk to force a failure:
Code: Select all   Expand viewCollapse view
# echo | format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <drive type unknown>
          /pci@0,0/pci8086,2829@d/disk@0,0
       1. c0t1d0 <ATA    -VBOX HARDDISK  -1.0  cyl 2607 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,2829@d/disk@1,0
Specify disk (enter its number): Specify disk (enter its number):

# cfgadm -al | grep 'sata0/[01]\>'
sata0/0                        sata-port    empty        unconfigured ok
sata0/1::dsk/c0t1d0            disk         connected    configured   ok

# metadb
        flags           first blk       block count
      Wm  p  l          16              8192            /dev/dsk/c0t0d0s3
     a    p  luo        16              8192            /dev/dsk/c0t1d0s3

# metastat d10
d10: Mirror
    Submirror 0: d11
      State: Needs maintenance
    Submirror 1: d12
      State: Okay
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 37592100 blocks (17 GB)

d11: Submirror of d10
    State: Needs maintenance
    Invoke: metareplace d10 c0t0d0s0 <new device>
    Size: 37592100 blocks (17 GB)
    Stripe 0:
        Device     Start Block  Dbase        State Reloc Hot Spare
        c0t0d0s0          0     No     Maintenance   Yes

d12: Submirror of d10
    State: Okay
    Size: 37592100 blocks (17 GB)
    Stripe 0:
        Device     Start Block  Dbase        State Reloc Hot Spare
        c0t1d0s0          0     No            Okay   Yes

Device Relocation Information:
Device   Reloc  Device ID
c0t0d0   Yes    id1,sd@SATA_____VBOX_HARDDISK____VBe4b17a10-75090f7e
c0t1d0   Yes    id1,sd@SATA_____VBOX_HARDDISK____VB27121653-c2012b38

# metastat d20
d20: Mirror
    Submirror 0: d21
      State: Needs maintenance
    Submirror 1: d22
      State: Okay
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 4209030 blocks (2.0 GB)

d21: Submirror of d20
    State: Needs maintenance
    Invoke: metareplace d20 c0t0d0s1 <new device>
    Size: 4209030 blocks (2.0 GB)
    Stripe 0:
        Device     Start Block  Dbase        State Reloc Hot Spare
        c0t0d0s1          0     No     Maintenance   Yes

d22: Submirror of d20
    State: Okay
    Size: 4209030 blocks (2.0 GB)
    Stripe 0:
        Device     Start Block  Dbase        State Reloc Hot Spare
        c0t1d0s1          0     No            Okay   Yes

Device Relocation Information:
Device   Reloc  Device ID
c0t0d0   Yes    id1,sd@SATA_____VBOX_HARDDISK____VBe4b17a10-75090f7e
c0t1d0   Yes    id1,sd@SATA_____VBOX_HARDDISK____VB27121653-c2012b38

■Removing failed device information:
Code: Select all   Expand viewCollapse view
# metadb -f -d c0t0d0s3
# metadb
        flags           first blk       block count
     a    p  luo        16              8192            /dev/dsk/c0t1d0s3

# metadetach -f d10 d11
d10: submirror d11 is detached
# metadetach -f d20 d21
d20: submirror d21 is detached

# metaclear d11 d21
d11: Concat/Stripe is cleared
d21: Concat/Stripe is cleared

# devfsadm -Cv

■Adding new disk(20GB/Port 0):
Code: Select all   Expand viewCollapse view
# cfgadm -al | grep 'sata0/[01]\>'
sata0/0::dsk/c0t0d0            disk         connected    configured   unusable     <<<<<---- 'My question is about this point.'
sata0/1::dsk/c0t1d0            disk         connected    configured   ok

I tried many tests but could not change unusable to ok.
I need help how to recognize new disk without system reboot at this point.

Thank you.
Last edited by socratis on 16. Dec 2018, 20:19, edited 1 time in total.
Reason: Removed unnecessary verbatim quote of the whole previous message.
sunwizard
 
Posts: 5
Joined: 4. Dec 2018, 08:54

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby stes » 17. Dec 2018, 12:19

From what you've said it sounds like the actual problem on SPARC with SCSI/FC is solved now.
When I tested OS Mirroring in VirtualBox, I got a problem when I removed the first disk and added it to new disk(same SATA port). This happened in VirtualBox and I needed to reboot the virtual machine for recognizing new disk. Of course, it worked fine on physical SPARC machines and VMware Workstations without system reboot.

This confirms my belief that it is necessary to be careful with comparison between SATA hotplug and FC/SCSI hotplug.

For what it's worth, I tried on a guest with Solaris 11.4 to replace a disk, and I was able to add a new disk (same SATA port) without rebooting the guest.

The Solaris 11.4 was installed on a mirrored disk configuration via the AI manifest:
Code: Select all   Expand viewCollapse view
    <target>
  <disk in_vdev="mirror_vdev" in_zpool="rpool" whole_disk="true">
    <disk_name name="c1t0d0" name_type="ctd"/>
  </disk>
  <disk in_vdev="mirror_vdev" in_zpool="rpool" whole_disk="true">
    <disk_name name="c1t1d0" name_type="ctd"/>
  </disk>
      <logical>
        <zpool name="rpool" is_root="true">
      <vdev name="mirror_vdev" redundancy="mirror"/>

Like that I obtained a mirrored root pool (with the operating system on 2 hot-pluggable SATA disks) :
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok
root@pluto:~# zpool status
  pool: rpool
 state: ONLINE
  scan: none requested
config:

        NAME        STATE      READ WRITE CKSUM
        rpool       ONLINE        0     0     0
          mirror-0  ONLINE        0     0     0
            c1t0d0  ONLINE        0     0     0
            c1t1d0  ONLINE        0     0     0

errors: No known data errors

Then at the VirtualBox level without preparing the guest operating system (i.e. without first detaching the disk and without unconfiguring it), I tried to remove a disk at the VirtualBox level :
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool status
  pool: rpool
 state: DEGRADED
status: One or more devices has been removed by the administrator.
        Sufficient replicas exist for the pool to continue functioning in a
        degraded state.
action: Online the device using 'zpool online' or replace the device with
        'zpool replace'.
  scan: none requested
config:

        NAME        STATE      READ WRITE CKSUM
        rpool       DEGRADED      0     0     0
          mirror-0  DEGRADED      0     0     0
            c1t0d0  ONLINE        0     0     0
            c1t1d0  REMOVED       0     0     0

errors: No known data errors
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1                        sata-port    empty        unconfigured ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok

Note that the disk went cleanly to status 'unconfigured' and condition OK and receptable empty (apparently by the removal action at the VirtualBox level).

Then I manually removed the REMOVED disk:
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool detach rpool c1t1d0
root@pluto:~# zpool status
  pool: rpool
 state: ONLINE
  scan: none requested
config:

        NAME      STATE      READ WRITE CKSUM
        rpool     ONLINE        0     0     0
          c1t0d0  ONLINE        0     0     0


Also when I added a new disk in the same SATA slot this went very smoothly:
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok

Then attach and resilver (everything without reboot) :
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool attach rpool c1t0d0 c1t1d0
Make sure to wait until resilver is done before rebooting.

and back to
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool status rpool
  pool: rpool
 state: ONLINE
  scan: resilvered 2.57G in 1m33s with 0 errors on Mon Dec 17 11:00:39 2018

config:

        NAME        STATE      READ WRITE CKSUM
        rpool       ONLINE        0     0     0
          mirror-0  ONLINE        0     0     0
            c1t0d0  ONLINE        0     0     0
            c1t1d0  ONLINE        0     0     0

errors: No known data errors

So at least with a guest running Solaris 11.4 and host OS Solaris 11.4 (VirtualBox 6.0.0RC1) I am able to remove a SATA hotplug disk and add a new SATA hotplug disk on the same port without reboot of the guest.
stes
 
Posts: 142
Joined: 28. Apr 2018, 11:07

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby sunwizard » 17. Dec 2018, 15:32

>>> Hello. stes.

>>> Thanks for your hard work and time.

>>> I had rpool and SVM mirroring tests. I also tested the recognition after removing the second disk on VirtualBox. I handled the second disk failure well.
>>> This is a problem when removing the first disk on VirtualBox.

>>> I am so sorry. I should have mentioned it to you in advance.
Last edited by socratis on 17. Dec 2018, 20:29, edited 1 time in total.
Reason: Removed unnecessary verbatim quote of the whole previous message.
sunwizard
 
Posts: 5
Joined: 4. Dec 2018, 08:54

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby stes » 17. Dec 2018, 16:21

No problem , it is an interesting question, and I'm just trying to help out.

So to summarize, you can add a new disk (SATA hotplug) and you can also remove the second disk without a problem, where "second" is it the disk that is not the active bootdevice ?

When you try to remove the 'first disk' is it also the boot device ?
Code: Select all   Expand viewCollapse view
root@pluto:~#  prtconf -v|sed -n '/bootpath/,/value/p'
        name='bootpath' type=string items=1
            value='/pci@0,0/pci8086,2829@d/disk@0,0:b'

where the disks are:
Code: Select all   Expand viewCollapse view
root@pluto:~# echo | format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <VBOX-HARDDISK-1.0-16.00GB>
          /pci@0,0/pci8086,2829@d/disk@0,0
       1. c1t1d0 <VBOX-HARDDISK-1.0-16.00GB>
          /pci@0,0/pci8086,2829@d/disk@1,0
Specify disk (enter its number): Specify disk (enter its number):
stes
 
Posts: 142
Joined: 28. Apr 2018, 11:07

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby stes » 17. Dec 2018, 16:49

On Solaris 11.4 it seems I can remove the bootpath (the "first disk") and it seems to work.

I did not have to boot from an "alternate disk" to start with (which would be reasonable to try).

For a Solaris 11.4 guest :
Code: Select all   Expand viewCollapse view
root@pluto:~# prtconf -v|sed -n '/bootpath/,/value/p'
        name='bootpath' type=string items=1
            value='/pci@0,0/pci8086,2829@d/disk@0,0:b'
root@pluto:~# zpool status
  pool: rpool
 state: ONLINE
  scan: none requested
config:

        NAME        STATE      READ WRITE CKSUM
        rpool       ONLINE        0     0     0
          mirror-0  ONLINE        0     0     0
            c1t0d0  ONLINE        0     0     0
            c1t1d0  ONLINE        0     0     0

errors: No known data errors

Then I remove at the VirtualBox level the first disk :
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0                        sata-port    empty        unconfigured ok
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok
root@pluto:~# prtconf -v|sed -n '/bootpath/,/value/p'
        name='bootpath' type=string items=1
            value='/pci@0,0/pci8086,2829@d/disk@0,0:b'

Note that "Receptacle" is "empty" for sata0/0.

At that point I have:
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool status rpool
  pool: rpool
 state: DEGRADED
status: One or more devices has been removed by the administrator.
        Sufficient replicas exist for the pool to continue functioning in a
        degraded state.
action: Online the device using 'zpool online' or replace the device with
        'zpool replace'.
  scan: none requested
config:

        NAME        STATE      READ WRITE CKSUM
        rpool       DEGRADED      0     0     0
          mirror-0  DEGRADED      0     0     0
            c1t0d0  REMOVED       0     0     0
            c1t1d0  ONLINE        0     0     0


Then I add on the same port 0 a new disk (without reboot of the guest) and it works for me:

I get:
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok

Then I resilver the mirror as follows (not sure whether this is the easiest best way) :
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool detach rpool c1t0d0

and
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool attach rpool c1t1d0 c1t0d0
Make sure to wait until resilver is done before rebooting.
root@pluto:~# zpool status
  pool: rpool
 state: DEGRADED
status: One or more devices are currently being resilvered.  The pool will
        continue to function in a degraded state.
action: Wait for the resilver to complete.
        Run 'zpool status -v' to see device specific details.
  scan: resilver in progress since Mon Dec 17 15:32:14 2018
    2.80G scanned out of 3.58G at 719M/s, 1s to go
    0 resilvered
config:

        NAME        STATE      READ WRITE CKSUM
        rpool       DEGRADED      0     0     0
          mirror-0  DEGRADED      0     0     0
            c1t1d0  ONLINE        0     0     0
            c1t0d0  DEGRADED      0     0     0  (resilvering)

which results after a while in the following setup:
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool status
  pool: rpool
 state: ONLINE
  scan: resilvered 2.58G in 1m32s with 0 errors on Mon Dec 17 15:33:46 2018

config:

        NAME        STATE      READ WRITE CKSUM
        rpool       ONLINE        0     0     0
          mirror-0  ONLINE        0     0     0
            c1t1d0  ONLINE        0     0     0
            c1t0d0  ONLINE        0     0     0


I believe that "zpool online" and "zpool clear" should be used instead.

However in this test I can remove the first disk and I can add a new SATA hotplug disk at the same (first) port.

These results are on Solaris 11.4 host and guest also Solaris 11.4.
stes
 
Posts: 142
Joined: 28. Apr 2018, 11:07

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby stes » 17. Dec 2018, 17:32

An alternative scenario was to boot with only the second disk (so that I boot from an alternate disk).
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0                        sata-port    empty        unconfigured ok
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok
root@pluto:~# prtconf -v|sed -n '/bootpath/,/value/p'
        name='bootpath' type=string items=1
            value='/pci@0,0/pci8086,2829@d/disk@1,0:b'

In that case the bootpath is the disk@1,0 (because I removed the first disk prior to booting).

So that
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool status
  pool: rpool
 state: DEGRADED
status: One or more devices are unavailable in response to persistent errors.
        Sufficient replicas exist for the pool to continue functioning in a
        degraded state.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or 'fmadm repaired', or replace the device
        with 'zpool replace'.
        Run 'zpool status -v' to see device specific details.
  scan: none requested
config:

        NAME        STATE      READ WRITE CKSUM
        rpool       DEGRADED      0     0     0
          mirror-0  DEGRADED      0     0     0
            c1t0d0  UNAVAIL       0     0     0
            c1t1d0  ONLINE        0     0     0

In that case when I add (without rebooting the guest) the hotplug original disk in the same port as where it was, I have to configure it manually it seems:
Code: Select all   Expand viewCollapse view
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0                        sata-port    disconnected unconfigured failed
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0                        sata-port    disconnected unconfigured failed
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok
root@pluto:~# cfgadm -x sata_port_activate sata0/0
Activate the port: /devices/pci@0,0/pci8086,2829@d:0
This operation will enable activity on the SATA port
Continue (yes/no)? yes
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0                        disk         connected    unconfigured unknown
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok
root@pluto:~# cfgadm -c configure sata0/0         
root@pluto:~# cfgadm -al sata0
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c1t0d0            disk         connected    configured   ok
sata0/1::dsk/c1t1d0            disk         connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok

After this (where I added the initial disk back in the same port as where it was during installation),
Code: Select all   Expand viewCollapse view
root@pluto:~# zpool clear rpool c1t0d0
root@pluto:~# zpool online rpool c1t0d0
root@pluto:~# zpool status
  pool: rpool
 state: ONLINE
  scan: resilvered 29.9M in 2s with 0 errors on Mon Dec 17 16:27:47 2018

config:

        NAME        STATE      READ WRITE CKSUM
        rpool       ONLINE        0     0     0
          mirror-0  ONLINE        0     0     0
            c1t0d0  ONLINE        0     0     0
            c1t1d0  ONLINE        0     0     0

errors: No known data errors

So this is bringing the first disk online again without reboot.
stes
 
Posts: 142
Joined: 28. Apr 2018, 11:07

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby stes » 17. Dec 2018, 17:47

What could be interesting in the Solaris 10 case is to boot without the first disk, so that the bootpath is like
Code: Select all   Expand viewCollapse view
prtconf -v|sed -n '/bootpath/,/value/p'
        name='bootpath' type=string items=1
            value='/pci@0,0/pci8086,2829@d/disk@1,0:b'

that is the bootpath is the second disk, then try to add the first disk in the Solaris 10 case.

Then proceed without reboot to remove the first disk again and replace it with a new disk in the same port.
stes
 
Posts: 142
Joined: 28. Apr 2018, 11:07

Re: How to add a new disk(SATA/SCSI Type) in solaris 10 x86 On Oracle VirtualBox?

Postby socratis » 17. Dec 2018, 20:33

@sunwizard
The messages from "stes" are rather lengthy and explanatory. You don't have to quote them whole! There's a "Post Reply" button under the last message, no need to use the "Quote" one. It makes following up on the thread like reading "War and Peace"... ;)
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
socratis
Site Moderator
 
Posts: 26875
Joined: 22. Oct 2010, 11:03
Location: Greece
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Win(*>98), Linux*, OSX>10.5

Next

Return to VirtualBox on Solaris Hosts

Who is online

Users browsing this forum: No registered users and 1 guest