ZFS mirroring
Posted: Tue Apr 03, 2012 4:41 pm
After installing Solaris 11, I have left it run on single hard drive.
After almost restoring all functions and features back, I figured it is a good time to mirror the disks to protect the work I have done so far.
Since the file system is under ZFS, it is relatively easy to add a new drive and make them a mirror pool.
1. I have to make sure the newly added drive has the same partition information, I used the following command to do so:
It complained about different size and I don't know which hard drive is smaller.
c5d0s0 is a Maxtor and c6d0s0 is a Western Digital.
My guess is WD is smaller.
I then used format to check partition tables on both HDs:
Both showing the same information.
2. Time to add the new disk (WD) to rpool and form a mirror.
I then ran the following command to override error:
3. Check rpool status
It shows the progress status at each check.
There are people recommending to use a 3-way mirror to provide even faster performance and extra redundancy. This is what I have been thinking lately. I think I will make it a 3-way mirror eventually when I have all the data I need. Now, back to the root cause of this migration. Will I be able to boot from these mirrored rpool? I know I need to installgrub on each disk but I did install grub on Solaris 10 mirror disk but it still didn't boot up. I guess I will need some more testing to claim the mirroring is really working.
I have installed grub to WD disk:
It looks like grub has been installed.
Next test is to boot from this new HD after the silvering is completed.
Unless there's a tool to verify stage1 and stage2 on any given HDs.
After waiting for over an hour, the resilvering (mirroring) was completed on c6d0s0.
I took it out of mirror to see what would happen. Do I need to resilvering if I attach it back again?
All right. c6d0s0 is gone.
Oops....
It is resilvering again! It will take another hour and half to complete it.
Looks like it took only 59 minutes to mirror it.
After almost restoring all functions and features back, I figured it is a good time to mirror the disks to protect the work I have done so far.
Since the file system is under ZFS, it is relatively easy to add a new drive and make them a mirror pool.
1. I have to make sure the newly added drive has the same partition information, I used the following command to do so:
Code: Select all
% prtvtoc /dev/rdsk/c5d0s0 | fmthard -s - /dev/rdsk/c6d0s0
fmthard: Partition 2 specifies the full disk and is not equal
full size of disk. The full disk capacity is 976703805 sectors.
fmthard: Partition 2 specified as 976735935 sectors starting at 0
does not fit. The full disk contains 976703805 sectors.
fmthard: New volume table of contents now in place.
c5d0s0 is a Maxtor and c6d0s0 is a Western Digital.
My guess is WD is smaller.
I then used format to check partition tables on both HDs:
Code: Select all
Total disk cylinders available: 60797 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 1 - 60796 465.72GB (60796/0/0) 976687740
1 unassigned wm 0 0 (0/0/0) 0
2 backup wu 0 - 60798 465.74GB (60799/0/0) 976735935
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 unassigned wm 0 0 (0/0/0) 0
7 unassigned wm 0 0 (0/0/0) 0
8 boot wu 0 - 0 7.84MB (1/0/0) 16065
9 unassigned wm 0 0 (0/0/0) 0
Total disk cylinders available: 60797 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 1 - 60796 465.72GB (60796/0/0) 976687740
1 unassigned wu 0 0 (0/0/0) 0
2 backup wu 0 - 60798 465.74GB (60799/0/0) 976735935
3 unassigned wu 0 0 (0/0/0) 0
4 unassigned wu 0 0 (0/0/0) 0
5 unassigned wu 0 0 (0/0/0) 0
6 unassigned wu 0 0 (0/0/0) 0
7 unassigned wu 0 0 (0/0/0) 0
8 boot wu 0 - 0 7.84MB (1/0/0) 16065
9 unassigned wu 0 0 (0/0/0) 0
2. Time to add the new disk (WD) to rpool and form a mirror.
Code: Select all
%zpool attach rpool c5d0s0 c6d0s0
invalid vdev specification
use '-f' to override the following errors:
/dev/dsk/c6d0s0 overlaps with /dev/dsk/c6d0s2
Code: Select all
%zpool attach -f rpool c5d0s0 c6d0s0
Make sure to wait until resilver is done before rebooting.
Code: Select all
%zpool status
pool: rpool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Tue Apr 3 13:24:12 2012
525M scanned out of 144G at 37.5M/s, 1h5m to go
493M resilvered, 0.36% done
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c5d0s0 ONLINE 0 0 0
c6d0s0 ONLINE 0 0 0 (resilvering)
errors: No known data errors
%zpool status
pool: rpool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Tue Apr 3 13:24:12 2012
5.06G scanned out of 144G at 53.4M/s, 0h44m to go
5.06G resilvered, 3.51% done
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c5d0s0 ONLINE 0 0 0
c6d0s0 ONLINE 0 0 0 (resilvering)
errors: No known data errors
There are people recommending to use a 3-way mirror to provide even faster performance and extra redundancy. This is what I have been thinking lately. I think I will make it a 3-way mirror eventually when I have all the data I need. Now, back to the root cause of this migration. Will I be able to boot from these mirrored rpool? I know I need to installgrub on each disk but I did install grub on Solaris 10 mirror disk but it still didn't boot up. I guess I will need some more testing to claim the mirroring is really working.
I have installed grub to WD disk:
Code: Select all
% /usr/sbin/installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c6d0s0
stage2 written to partition 0, 282 sectors starting at 50 (abs 16115)
stage1 written to partition 0 sector 0 (abs 16065)
Next test is to boot from this new HD after the silvering is completed.
Unless there's a tool to verify stage1 and stage2 on any given HDs.
After waiting for over an hour, the resilvering (mirroring) was completed on c6d0s0.
I took it out of mirror to see what would happen. Do I need to resilvering if I attach it back again?
Code: Select all
%zpool detach rpool c6d0s0
%zpool status
pool: rpool
state: ONLINE
scan: resilvered 144G in 0h59m with 0 errors on Tue Apr 3 14:23:45 2012
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c5d0s0 ONLINE 0 0 0
errors: No known data errors
Code: Select all
%zpool attach -f rpool c5d0s0 c6d0s0
Make sure to wait until resilver is done before rebooting.
%zpool status
pool: rpool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Tue Apr 3 14:25:21 2012
339M scanned out of 144G at 28.3M/s, 1h26m to go
306M resilvered, 0.23% done
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c5d0s0 ONLINE 0 0 0
c6d0s0 ONLINE 0 0 0 (resilvering)
errors: No known data errors
It is resilvering again! It will take another hour and half to complete it.
Looks like it took only 59 minutes to mirror it.
Code: Select all
%zpool status
pool: rpool
state: ONLINE
scan: resilvered 144G in 0h59m with 0 errors on Tue Apr 3 15:24:46 2012
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c5d0s0 ONLINE 0 0 0
c6d0s0 ONLINE 0 0 0
errors: No known data errors