T5-2 RAID configuration

Post by **cah** » Wed Aug 14, 2013 1:10 pm

From ILOM, the following command will send a break signal to OS:

-> set /HOST send_break_action=break
Set 'send_break_action' to 'break'

Then, start the console connection.

Code: Select all

-> start /HOST/console        
Are you sure you want to start /HOST/console (y/n)? y

Serial console started.  To stop, type #.

{0} ok

Then, commands can be issued at ok prompt.

Find the RAID controller and select it to check how many disks are available:

Code: Select all

{0} ok show-devs
...
/pci@3c0/pci@1/pci@0/pci@2/scsi@0
...

{0} ok select /pci@300/pci@1/pci@0/pci@2/scsi@0

{0} ok show-children

FCode Version 1.00.62, MPT Version 2.00, Firmware Version 14.00.00.00

Target 9 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01636fea0  SASAddress 5000cca01636fea1  PhyNum 0 
Target a 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01645e668  SASAddress 5000cca01645e669  PhyNum 1 
Target b 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01642a030  SASAddress 5000cca01642a031  PhyNum 2 
Target c 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca016358dd8  SASAddress 5000cca016358dd9  PhyNum 3

However, there should be 6 internal disks:

Code: Select all

root@:~# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t5000CCA0163EE364d0 <HITACHI-H109060SESUN600G-A31A cyl 64986 alt 2 hd 27 sec 668>  solaris
          /scsi_vhci/disk@g5000cca0163ee364
       1. c0t5000CCA01636FEA0d0 <HITACHI-H109060SESUN600G-A31A cyl 64986 alt 2 hd 27 sec 668>  solaris
          /scsi_vhci/disk@g5000cca01636fea0
       2. c0t5000CCA01642A030d0 <HITACHI-H109060SESUN600G-A31A cyl 64986 alt 2 hd 27 sec 668>  solaris
          /scsi_vhci/disk@g5000cca01642a030
       3. c0t5000CCA01645E668d0 <HITACHI-H109060SESUN600G-A31A cyl 64986 alt 2 hd 27 sec 668>  solaris
          /scsi_vhci/disk@g5000cca01645e668
       4. c0t5000CCA016358DD8d0 <HITACHI-H109060SESUN600G-A31A cyl 64986 alt 2 hd 27 sec 668>  solaris
          /scsi_vhci/disk@g5000cca016358dd8
       5. c0t5000CCA0164357B4d0 <HITACHI-H109060SESUN600G-A31A cyl 64986 alt 2 hd 27 sec 668>  solaris
          /scsi_vhci/disk@g5000cca0164357b4

I opened a ticket (SR 3-7669875891) with Oracle for assistance on 08/13/2013.

Post by **cah** » Thu Aug 15, 2013 5:50 pm

Got the following feedback from Oracle support:

Oracle wrote:
T5-2 has 2 x LSI SAS2308 raid controllers

/pci@300/pci@1/pci@0/pci@2/scsi@0/iport@[1-4]/disk@[0-3] (Controller 0; LSI SAS2308 #1)
/pci@3c0/pci@1/pci@0/pci@2/scsi@0/iport@[1&2]/disk@[4&5] (Controller 1; LSI SAS2308 #2)

You should be able to see the other 2 disks on the 2nd controller from OBP using this path:

{0} ok select /pci@3c0/pci@1/pci@0/pci@2/scsi@0
{0} ok show-children

The other way to select controller is to use "devalias" to show (scsi0 and scsi1 are both RAID controllers):

Code: Select all

{0} ok devalias
screen                   /pci@380/pci@1/pci@0/pci@3/display@0
net3                     /pci@3c0/pci@1/pci@0/pci@1/network@0,1
net2                     /pci@3c0/pci@1/pci@0/pci@1/network@0
disk5                    /pci@3c0/pci@1/pci@0/pci@2/scsi@0/disk@p1
disk4                    /pci@3c0/pci@1/pci@0/pci@2/scsi@0/disk@p0
scsi1                    /pci@3c0/pci@1/pci@0/pci@2/scsi@0
cdrom                    /pci@3c0/pci@1/pci@0/pci@2/scsi@0/disk@p3
net1                     /pci@300/pci@1/pci@0/pci@1/network@0,1
net                      /pci@300/pci@1/pci@0/pci@1/network@0
net0                     /pci@300/pci@1/pci@0/pci@1/network@0
disk3                    /pci@300/pci@1/pci@0/pci@2/scsi@0/disk@p3
disk2                    /pci@300/pci@1/pci@0/pci@2/scsi@0/disk@p2
disk1                    /pci@300/pci@1/pci@0/pci@2/scsi@0/disk@p1
disk                     /pci@300/pci@1/pci@0/pci@2/scsi@0/disk@p0
disk0                    /pci@300/pci@1/pci@0/pci@2/scsi@0/disk@p0
scsi                     /pci@300/pci@1/pci@0/pci@2/scsi@0
scsi0                    /pci@300/pci@1/pci@0/pci@2/scsi@0
virtual-console          /virtual-devices/console@1
name                     aliases
{0} ok select scsi0
{0} ok show-children

FCode Version 1.00.62, MPT Version 2.00, Firmware Version 14.00.00.00

Target 9 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01636fea0  SASAddress 5000cca01636fea1  PhyNum 0 
Target a 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01645e668  SASAddress 5000cca01645e669  PhyNum 1 
Target b 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01642a030  SASAddress 5000cca01642a031  PhyNum 2 
Target c 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca016358dd8  SASAddress 5000cca016358dd9  PhyNum 3

Tried to select:

Code: Select all

{0} ok select /pci@3c0/pci@1/pci@0/pci@2/scsi@0
{0} ok select /pci@300/pci@1/pci@0/pci@2/scsi@0

However, it still shows only 4 targets:

Code: Select all

{0} ok show-children

FCode Version 1.00.62, MPT Version 2.00, Firmware Version 14.00.00.00

Target 9 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01636fea0  SASAddress 5000cca01636fea1  PhyNum 0 
Target a 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01645e668  SASAddress 5000cca01645e669  PhyNum 1 
Target b 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01642a030  SASAddress 5000cca01642a031  PhyNum 2 
Target c 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca016358dd8  SASAddress 5000cca016358dd9  PhyNum 3

So, I unselected devices:

Code: Select all

{0} ok unselect-dev
{0} ok show-children

FCode Version 1.00.62, MPT Version 2.00, Firmware Version 14.00.00.00

Target 9 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca0163ee364  SASAddress 5000cca0163ee365  PhyNum 0 
Target a 
  Unit 0   Removable Read Only device   TEAC     DV-W28SS-W       1.0A
  SATA device  PhyNum 3 
Target b 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca0164357b4  SASAddress 5000cca0164357b5  PhyNum 1 
{0} ok unselect-dev
{0} ok unselect-dev
{0} ok show-children
show-children ?

Then, I select the p3c controller first and show-children:

Code: Select all

{0} ok select /pci@3c0/pci@1/pci@0/pci@2/scsi@0
{0} ok show-children                           

FCode Version 1.00.62, MPT Version 2.00, Firmware Version 14.00.00.00

Target 9 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca0163ee364  SASAddress 5000cca0163ee365  PhyNum 0 
Target a 
  Unit 0   Removable Read Only device   TEAC     DV-W28SS-W       1.0A
  SATA device  PhyNum 3 
Target b 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca0164357b4  SASAddress 5000cca0164357b5  PhyNum 1

I then tried to select another controller:

Code: Select all

{0} ok select /pci@300/pci@1/pci@0/pci@2/scsi@0
{0} ok show-children                           

FCode Version 1.00.62, MPT Version 2.00, Firmware Version 14.00.00.00

Target 9 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01636fea0  SASAddress 5000cca01636fea1  PhyNum 0 
Target a 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01645e668  SASAddress 5000cca01645e669  PhyNum 1 
Target b 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01642a030  SASAddress 5000cca01642a031  PhyNum 2 
Target c 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca016358dd8  SASAddress 5000cca016358dd9  PhyNum 3

It shows the other 4 devices.
Maybe I should select both controllers at the same time....

It didn't work the way I thought:

Code: Select all

{0} ok select /pci@3c0/pci@1/pci@0/pci@2/scsi@0 /pci@300/pci@1/pci@0/pci@2/scsi@0
/pci@300/pci@1/pci@0/pci@2/scsi@0 ?
{0} ok show-children                                                             

MPT Version 2.00, Firmware Version 14.00.00.00

Target 9 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca0163ee364  SASAddress 5000cca0163ee365  PhyNum 0 
Target a 
  Unit 0   Removable Read Only device   TEAC     DV-W28SS-W       1.0A
  SATA device  PhyNum 3 
Target b 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca0164357b4  SASAddress 5000cca0164357b5  PhyNum 1 
{0} ok

Update the ticket again asking how I can see all 6 disks at the same time.

In fact, it doesn't really matter which 2 disks I pick for root disk mirror.
Maybe I will just pick 5000cca0163ee364 and 5000cca0164357b4 from select /pci@3c0/pci@1/pci@0/pci@2/scsi@0 and make it a mirror disk.

Here is what I did:

Code: Select all

{0} ok 9 b create-raid1-volume
Target 9 size is 1169920000 Blocks, 598 GB
Target b size is 1169920000 Blocks, 598 GB
The volume can be any size from 1 MB to 571250 MB
What size do you want?  [571250] 
Volume size will be 1169920000 Blocks, 598 GB
Enter a volume name:  [0 to 15 characters] BOOT_VOLUME
Volume has been created
{0} ok show-volumes           
Volume 0 Target 143  Type RAID1 (Mirroring)
  Name BOOT_VOLUME  WWID 038670488b799130
  Optimal  Enabled  Background Init In Progress 
  2 Members                                         1169920000 Blocks, 598 GB
  Disk 0 
    Primary  Optimal 
    Target 9      HITACHI  H109060SESUN600G A31A   PhyNum 0 
  Disk 1 
    Secondary  Optimal 
    Target b      HITACHI  H109060SESUN600G A31A   PhyNum 1

In case the volume needs to be deleted, use the following command:

Code: Select all

{0} ok 0 delete-volume
The volume and its data will be deleted
Are you sure (yes/no)?  [no] yes
Volume 0 has been deleted

show-volumewill not have any volumes and show-children will have the individual tagets again:

Code: Select all

{0} ok show-volumes   
No volumes to show
{0} ok show-children              

FCode Version 1.00.62, MPT Version 2.00, Firmware Version 14.00.00.00

Target 9 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca0163ee364  SASAddress 5000cca0163ee365  PhyNum 0 
Target a 
  Unit 0   Removable Read Only device   TEAC     DV-W28SS-W       1.0A
  SATA device  PhyNum 3 
Target b 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca0164357b4  SASAddress 5000cca0164357b5  PhyNum 1

Here is a list of FCode Utility Commands

Code: Select all

FCode Command                  Description
-------------------------------------------------------------------------------------------
show-children                  Lists all connected physical drives and logical volumes.
show-volumes                   Lists all connected logical volumes in detail.
create-raid0-volume            Creates a RAID 0 volume (minimum two targets).
create-raid1-volume            Creates a RAID 1 volume (exactly two targets).
create-raid1e-volume           Creates a RAID 1e volume (minimum three targets).
delete-volume                  Deletes a RAID volume.
activate-volume                Re-activate a RAID volume after the motherboard has been replaced.

SPARC T5-2 HTML Collection:

http://docs.oracle.com/cd/E28853_01/htm ... index.html

Configuring Hardware RAID on T5-2:

http://docs.oracle.com/cd/E28853_01/htm ... #scrolltoc

Post by **cah** » Fri Aug 16, 2013 1:10 pm

feedback from Oracle:

Oracle wrote: That is a limitation of the internal raid controllers.

When configuring a volume, or adding a hot spare disk, all the disks must be on the same controller. A volume cannot be built with member disks from both controllers.
Up to two logical volumes can be configured on each onboard RAID controller

Note: You will need to re-install Solaris after building the raid, as this operation wipes any data on existing disks.

Post by **cah** » Fri Aug 16, 2013 1:51 pm

After discussing with Mats, we decided to go for RAIN 1E for data volume. It has both striping and mirroring.

Code: Select all

{0} ok select scsi0
{0} ok show-children

FCode Version 1.00.62, MPT Version 2.00, Firmware Version 14.00.00.00

Target 9 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01636fea0  SASAddress 5000cca01636fea1  PhyNum 0 
Target a 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01645e668  SASAddress 5000cca01645e669  PhyNum 1 
Target b 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca01642a030  SASAddress 5000cca01642a031  PhyNum 2 
Target c 
  Unit 0   Disk   HITACHI  H109060SESUN600G A31A    1172123568 Blocks, 600 GB
  SASDeviceName 5000cca016358dd8  SASAddress 5000cca016358dd9  PhyNum 3 
{0} ok 9 a b c create-raid1e-volume
The number of targets is even, creating RAID10 volume instead
Target 9 size is 1169920000 Blocks, 598 GB
Target a size is 1169920000 Blocks, 598 GB
Target b size is 1169920000 Blocks, 598 GB
Target c size is 1169920000 Blocks, 598 GB
The volume can be any size from 1 MB to 1142500 MB
What size do you want?  [1142500] 
Volume size will be 2339840000 Blocks, 1197 GB
Enter a volume name:  [0 to 15 characters] DATA_VOLUME
Volume has been created
{0} ok show-children               

FCode Version 1.00.62, MPT Version 2.00, Firmware Version 14.00.00.00

Target 143 Volume 0 
  Unit 0   Disk   LSI Logical Volume 3000    2339840000 Blocks, 1197 GB
  VolumeDeviceName 3a326f3c118fec2e  VolumeWWID 0a326f3c118fec2e
{0} ok show-volumes
Volume 0 Target 143  Type RAID10 (Striped Mirroring)
  Name DATA_VOLUME  WWID 0a326f3c118fec2e
  Optimal  Enabled  Background Init In Progress 
  4 Members                                         2339840000 Blocks, 1197 GB
  Disk 0 
    Member 0  Optimal 
    Target 9      HITACHI  H109060SESUN600G A31A   PhyNum 0 
  Disk 1 
    Member 1  Optimal 
    Target a      HITACHI  H109060SESUN600G A31A   PhyNum 1 
  Disk 2 
    Member 2  Optimal 
    Target b      HITACHI  H109060SESUN600G A31A   PhyNum 2 
  Disk 3 
    Member 3  Optimal 
    Target c      HITACHI  H109060SESUN600G A31A   PhyNum 3

T5-2 is ready for OS installation by jumpstart.

Post by **cah** » Mon Feb 24, 2014 4:54 pm

Mats wants to get the information for QA purpose.

Code: Select all

{0} ok show-devs
...
/pci@3c0/pci@1/pci@0/pci@2/scsi@0
...
/pci@300/pci@1/pci@0/pci@2/scsi@0
...

/pci@3c0/pci@1/pci@0/pci@2/scsi@0

Code: Select all

{0} ok select /pci@3c0/pci@1/pci@0/pci@2/scsi@0
{0} ok show-children

FCode Version 1.00.63, MPT Version 2.00, Firmware Version 14.00.00.00

Target a 
  Unit 0   Removable Read Only device   TEAC     DV-W28SS-W       1.0A
  SATA device  PhyNum 3 
Target 143 Volume 0 
  Unit 0   Disk   LSI Logical Volume 3000    1169920000 Blocks, 598 GB
  VolumeDeviceName 338670488b799130  VolumeWWID 038670488b799130
{0} ok show-volumes                            
Volume 0 Target 143  Type RAID1 (Mirroring)
  Name BOOT_VOLUME  WWID 038670488b799130
  Optimal  Enabled 
  2 Members                                         1169920000 Blocks, 598 GB
  Disk 1 
    Primary  Optimal 
    Target 9      HITACHI  H109060SESUN600G A31A   PhyNum 0 
  Disk 0 
    Secondary  Optimal 
    Target b      HITACHI  H109060SESUN600G A31A   PhyNum 1

/pci@300/pci@1/pci@0/pci@2/scsi@0

Code: Select all

{0} ok select /pci@300/pci@1/pci@0/pci@2/scsi@0
{0} ok show-children

FCode Version 1.00.63, MPT Version 2.00, Firmware Version 14.00.00.00

Target 143 Volume 0 
  Unit 0   Disk   LSI Logical Volume 3000    2339840000 Blocks, 1197 GB
  VolumeDeviceName 3a326f3c118fec2e  VolumeWWID 0a326f3c118fec2e
{0} ok show-volumes
Volume 0 Target 143  Type RAID10 (Striped Mirroring)
  Name DATA_VOLUME  WWID 0a326f3c118fec2e
  Optimal  Enabled 
  4 Members                                         2339840000 Blocks, 1197 GB
  Disk 3 
    Member 0  Optimal 
    Target 9      HITACHI  H109060SESUN600G A31A   PhyNum 0 
  Disk 2 
    Member 1  Optimal 
    Target a      HITACHI  H109060SESUN600G A31A   PhyNum 1 
  Disk 1 
    Member 2  Optimal 
    Target b      HITACHI  H109060SESUN600G A31A   PhyNum 2 
  Disk 0 
    Member 3  Optimal 
    Target c      HITACHI  H109060SESUN600G A31A   PhyNum 3

bbs.hsiao.net 交流論壇

T5-2 RAID configuration

T5-2 RAID configuration

T5-2 RAID configuration - II

T5-2 RAID configuration - III

T5-2 RAID configuration - IV

T5-2 RAID configuration - V