This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
unix:lvm_recovery [2011/10/23 16:24] robm |
unix:lvm_recovery [2013/05/28 09:28] robm [Finding the ext4 file-system] |
||
---|---|---|---|
Line 262: | Line 262: | ||
====== Recognising the nested LVM volumes ====== | ====== Recognising the nested LVM volumes ====== | ||
+ | Since it's been a few days and reboots since I last worked on this, I'll start by plugging the USB drive it. | ||
+ | < | ||
+ | root@ikari: | ||
+ | [ 479.180019] usb 2-5: new high speed USB device number 7 using ehci_hcd | ||
+ | [ 479.313228] scsi13 : usb-storage 2-5:1.0 | ||
+ | [ 480.312605] scsi 13:0:0:0: Direct-Access | ||
+ | [ 480.336633] sd 13:0:0:0: Attached scsi generic sg10 type 0 | ||
+ | [ 480.337029] sd 13:0:0:0: [sdi] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB) | ||
+ | [ 480.337671] sd 13:0:0:0: [sdi] Write Protect is off | ||
+ | [ 480.337671] sd 13:0:0:0: [sdi] Mode Sense: 2f 08 00 00 | ||
+ | [ 480.340027] sd 13:0:0:0: [sdi] No Caching mode page present | ||
+ | [ 480.340027] sd 13:0:0:0: [sdi] Assuming drive cache: write through | ||
+ | [ 480.341806] sd 13:0:0:0: [sdi] No Caching mode page present | ||
+ | [ 480.341811] sd 13:0:0:0: [sdi] Assuming drive cache: write through | ||
+ | [ 480.357290] | ||
+ | [ 480.359346] sd 13:0:0:0: [sdi] No Caching mode page present | ||
+ | [ 480.359350] sd 13:0:0:0: [sdi] Assuming drive cache: write through | ||
+ | [ 480.359354] sd 13:0:0:0: [sdi] Attached SCSI disk | ||
+ | </ | ||
+ | The (outer) LVM PVs are automatically detects, and their VGs + LVs are subsequently detected: | ||
+ | < | ||
+ | root@ikari: | ||
+ | PV | ||
+ | / | ||
+ | |||
+ | root@ikari: | ||
+ | VG #PV #LV #SN Attr VSize VFree | ||
+ | vg_scratch | ||
+ | |||
+ | root@ikari: | ||
+ | LV | ||
+ | lv_scratch vg_scratch owi-a- | ||
+ | snap | ||
+ | </ | ||
+ | |||
+ | Somewhere on the '' | ||
+ | |||
+ | < | ||
+ | 8018600: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018610: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018620: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018630: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018640: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018650: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018660: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018670: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018680: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018690: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80186a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80186b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80186c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80186d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80186e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80186f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018700: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018710: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018720: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018730: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018740: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018750: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018760: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018770: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018780: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018790: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80187a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80187b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80187c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80187d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80187e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80187f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018800: 4c41 4245 4c4f 4e45 0100 0000 0000 0000 LABELONE........ | ||
+ | 8018810: 9148 4053 2000 0000 4c56 4d32 2030 3031 .H@S ...LVM2 001 | ||
+ | 8018820: 5341 7536 6e32 7578 474c 5148 6743 5351 SAu6n2uxGLQHgCSQ | ||
+ | 8018830: 6b56 6b5a 655a 4c78 7874 314b 7652 6a31 kVkZeZLxxt1KvRj1 | ||
+ | 8018840: 00f8 0391 df00 0000 0000 0300 0000 0000 ................ | ||
+ | 8018850: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018860: 0000 0000 0000 0000 0010 0000 0000 0000 ................ | ||
+ | 8018870: 00f0 0200 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018880: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 8018890: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 80188a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | </ | ||
+ | |||
+ | I know from using '' | ||
+ | |||
+ | < | ||
+ | root@ikari: | ||
+ | |||
+ | root@ikari: | ||
+ | / | ||
+ | /dev/loop0 [ | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | /dev/ram10 [ 64.00 MiB] | ||
+ | /dev/ram11 [ 64.00 MiB] | ||
+ | /dev/ram12 [ 64.00 MiB] | ||
+ | /dev/ram13 [ 64.00 MiB] | ||
+ | /dev/ram14 [ 64.00 MiB] | ||
+ | /dev/ram15 [ 64.00 MiB] | ||
+ | / | ||
+ | 0 disks | ||
+ | 24 partitions | ||
+ | 0 LVM physical volume whole disks | ||
+ | 2 LVM physical volumes | ||
+ | |||
+ | root@ikari: | ||
+ | PV | ||
+ | /dev/loop0 store_vg | ||
+ | / | ||
+ | </ | ||
+ | |||
+ | If '' | ||
+ | |||
+ | < | ||
+ | root@ikari: | ||
+ | LV | ||
+ | store_lv | ||
+ | home_zfs | ||
+ | lv_scratch vg_scratch owi-a- 894.27g | ||
+ | snap | ||
+ | </ | ||
+ | |||
+ | ====== Finding the ext4 file-system ====== | ||
+ | |||
+ | OK, so now I can read/write my '' | ||
+ | |||
+ | < | ||
+ | root@ikari: | ||
+ | / | ||
+ | </ | ||
+ | |||
+ | Time to play with a hex editor again to look at a valid '' | ||
+ | |||
+ | After a bit of searching I find the characteristic '' | ||
+ | |||
+ | < | ||
+ | # xxd / | ||
+ | 05c9db0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 05c9dc0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 05c9dd0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 05c9de0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 05c9df0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 05c9e00: 00c0 fc06 c017 f90d c9da b200 0c2a d00a .............*.. | ||
+ | 05c9e10: c604 f906 0000 0000 0200 0000 0200 0000 ................ | ||
+ | 05c9e20: 0080 0000 0080 0000 0040 0000 9d17 aa4d .........@.....M | ||
+ | 05c9e30: 9d17 aa4d 0200 1b00 53ef 0100 0100 0000 ...M....S....... | ||
+ | 05c9e40: 4c2c a24d 004e ed00 0000 0000 0100 0000 L, | ||
+ | 05c9e50: 0000 0000 0b00 0000 8000 0000 3400 0000 ............4... | ||
+ | 05c9e60: 0600 0000 0300 0000 9495 5e9b 7d7e 41da ..........^.}~A. | ||
+ | 05c9e70: a595 6af4 4848 b5c6 7374 6f72 6500 0000 ..j.HH..store... | ||
+ | 05c9e80: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 05c9e90: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 05c9ea0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 05c9eb0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 05c9ec0: 0000 0000 0000 0000 0000 0000 0000 c803 ................ | ||
+ | 05c9ed0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 05c9ee0: 0800 0000 0000 0000 0000 0000 9a4a cd41 .............J.A | ||
+ | 05c9ef0: 88ff 4759 ac42 d083 8b3f 3b0d 0201 0000 ..GY.B...?; | ||
+ | 05c9f00: 0000 0000 0000 0000 cf4a 9f46 0906 0000 .........J.F.... | ||
+ | 05c9f10: 0a06 0000 0b06 0000 0c06 0000 0d06 0000 ................ | ||
+ | </ | ||
+ | |||
+ | By comparing this to my valid file-system, | ||
+ | |||
+ | < | ||
+ | # xxd /dev/sda1 | less | ||
+ | 0000380: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 0000390: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 00003a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 00003b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 00003c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 00003d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 00003e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 00003f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 0000400: 0020 2601 005d 9804 73d1 3a00 8366 bb02 . & | ||
+ | 0000410: d8da 2001 0000 0000 0200 0000 0200 0000 .. ............. | ||
+ | 0000420: 0080 0000 0080 0000 0020 0000 b153 264f ......... ...S&O | ||
+ | 0000430: d9bf f44e 0d00 2200 53ef 0100 0100 0000 ...N.." | ||
+ | 0000440: 98b4 f44e 004e ed00 0000 0000 0100 0000 ...N.N.......... | ||
+ | 0000450: 0000 0000 0b00 0000 0001 0000 3c00 0000 ............< | ||
+ | 0000460: 4602 0000 7b00 0000 89bb eae1 b864 492d F...{........dI- | ||
+ | 0000470: a3d6 2bc9 5336 151e 0000 0000 0000 0000 ..+.S6.......... | ||
+ | 0000480: 0000 0000 0000 0000 2f00 1eae 7c13 0000 ......../ | ||
+ | 0000490: 0000 c099 98a1 0188 ffff 985d 698f 0188 ...........]i... | ||
+ | 00004a0: ffff 307a 87a2 0188 ffff 307a 87a2 0188 ..0z......0z.... | ||
+ | 00004b0: ffff 585c 698f 0188 ffff 645d 1681 ffff ..X\i.....d].... | ||
+ | 00004c0: ffff c000 588f 0188 0000 0000 0000 ed03 ....X........... | ||
+ | 00004d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 00004e0: 0800 0000 0000 0000 b60a 4000 5547 eae9 ..........@.UG.. | ||
+ | 00004f0: 2f3c 45ee 9913 70fb 20b7 7395 0101 0000 /<E...p. .s..... | ||
+ | 0000500: 0000 0000 0000 0000 98b4 f44e 0af3 0200 ...........N.... | ||
+ | 0000510: 0400 0000 0000 0000 0000 0000 ff7f 0000 ................ | ||
+ | 0000520: 0080 4802 ff7f 0000 0100 0000 ffff 4802 ..H...........H. | ||
+ | 0000530: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 0000540: 0000 0000 0000 0000 0000 0000 0000 0008 ................ | ||
+ | 0000550: 0000 0000 0000 0000 0000 0000 1c00 1c00 ................ | ||
+ | 0000560: 0100 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 0000570: 0000 0000 0400 0000 b6b0 300b 0000 0000 ..........0..... | ||
+ | 0000580: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | 0000590: 0000 0000 0000 0000 0000 0000 0000 0000 ................ | ||
+ | </ | ||
+ | |||
+ | So let's create another loopback device with an offset: | ||
+ | |||
+ | < | ||
+ | # losetup /dev/loop1 / | ||
+ | </ | ||
+ | |||
+ | For the record, that makes the current overall loopback settings: | ||
+ | |||
+ | < | ||
+ | # losetup -a | ||
+ | /dev/loop0: [0005]: | ||
+ | /dev/loop1: [0005]: | ||
+ | </ | ||
+ | |||
+ | So is the '' | ||
+ | |||
+ | < | ||
+ | # file -Ls /dev/loop1 | ||
+ | /dev/loop1: Linux rev 1.0 ext3 filesystem data, UUID=94955e9b-7d7e-41da-a595-6af44848b5c6, | ||
+ | </ | ||
+ | |||
+ | Success! | ||
+ | |||
+ | **Update**: I ended up writing a Python script, [[https:// | ||
+ | |||
+ | < | ||
+ | root@skuld:/ | ||
+ | -c | sort -rn | head | ||
+ | 17 origin -134282240 | ||
+ | 16 origin -134514176 | ||
+ | 1 origin 8382976 | ||
+ | 1 origin 8268288 | ||
+ | 1 origin 8256000 | ||
+ | 1 origin 8215040 | ||
+ | 1 origin 8145408 | ||
+ | 1 origin 8133120 | ||
+ | 1 origin 8043008 | ||
+ | 1 origin 8030720 | ||
+ | </ | ||
+ | |||
+ | From this I conclude that my original (working) installation was not actually using the Logical Volume! This may explain why updating Ubuntu to a version which has LVM support by default made my system unable to find the file-system. | ||
+ | |||
+ | So from this point on I'll ignore the Logical Volumes in my disk image ('' | ||
+ | |||
+ | ====== Getting my data back ====== | ||
+ | |||
+ | Obviously I tried mounting it first, to no avail: | ||
+ | |||
+ | < | ||
+ | root@ikari:/ | ||
+ | mount: wrong fs type, bad option, bad superblock on /dev/loop1, | ||
+ | | ||
+ | In some cases useful info is found in syslog - try | ||
+ | dmesg | tail or so | ||
+ | |||
+ | </ | ||
+ | |||
+ | Since this is all sitting on top of the LVM snapshot I made earlier (''/ | ||
+ | |||
+ | First attempt, no joy: | ||
+ | |||
+ | < | ||
+ | # fsck.ext4 -y /dev/loop1 | ||
+ | e2fsck 1.41.14 (22-Dec-2010) | ||
+ | fsck.ext4: Group descriptors look bad... trying backup blocks... | ||
+ | fsck.ext4: Bad magic number in super-block when using the backup blocks | ||
+ | fsck.ext4: going back to original superblock | ||
+ | Error reading block 3226742528 (Invalid argument). | ||
+ | |||
+ | Force rewrite? yes | ||
+ | |||
+ | Superblock has an invalid journal (inode 8). | ||
+ | Clear? yes | ||
+ | |||
+ | *** ext3 journal has been deleted - filesystem is now ext2 only *** | ||
+ | |||
+ | The filesystem size (according to the superblock) is 234428352 blocks | ||
+ | The physical size of the device is 187529782 blocks | ||
+ | Either the superblock or the partition table is likely to be corrupt! | ||
+ | Abort? yes | ||
+ | |||
+ | Error writing block 3226742528 (Invalid argument). | ||
+ | </ | ||
+ | |||
+ | Using my [[https:// | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # head vg_scratch-snap.log | ||
+ | OK / | ||
+ | BAD / | ||
+ | BAD / | ||
+ | BAD / | ||
+ | BAD / | ||
+ | BAD / | ||
+ | BAD / | ||
+ | BAD / | ||
+ | BAD / | ||
+ | BAD / | ||
+ | </ | ||
+ | |||
+ | Obviously the " | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # cut -d' ' -f9- ./ | ||
+ | 22 origin 231936 | ||
+ | 19 origin 0 | ||
+ | 1 origin 880602744320 | ||
+ | 1 origin 8802304 | ||
+ | 1 origin 8790016 | ||
+ | 1 origin 8679424 | ||
+ | 1 origin 8667136 | ||
+ | 1 origin 8556544 | ||
+ | 1 origin 8544256 | ||
+ | 1 origin 8433664 | ||
+ | </ | ||
+ | |||
+ | As above, I mounted this on a loop back device and ran '' | ||
+ | |||
+ | On the advice of [[http:// | ||
+ | |||
+ | < | ||
+ | mke2fs -S /dev/loop0 | ||
+ | </ | ||
+ | |||
+ | followed by '' | ||
+ | |||
+ | ===== Lowering IO and CPU scheduling priority of fsck ===== | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # ps fo pid, | ||
+ | PID %MEM %CPU CMD | ||
+ | 19540 0.0 0.0 -bash | ||
+ | 19666 0.0 0.0 \_ sudo su | ||
+ | 19667 0.0 0.0 \_ su | ||
+ | 19675 0.0 0.0 \_ bash | ||
+ | 20181 0.0 0.0 \_ fsck /dev/loop0 -y | ||
+ | 20182 49.4 86.8 \_ fsck.ext2 -y /dev/loop0 | ||
+ | </ | ||
+ | |||
+ | To keep my system usable I lowered the IO and CPU priority of '' | ||
+ | |||
+ | < | ||
+ | root@Ikari: | ||
+ | </ | ||
+ | |||
+ | Then raise the " | ||
+ | |||
+ | < | ||
+ | root@Ikari: | ||
+ | 20182 (process ID) old priority 0, new priority 10 | ||
+ | </ | ||
+ | |||
+ | ===== Adding more swap to my system ===== | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # free -m | ||
+ | | ||
+ | Mem: 5969 | ||
+ | -/+ buffers/ | ||
+ | Swap: | ||
+ | </ | ||
+ | |||
+ | I was concerned to see that 50% of my swap was in use. I don't know how long '' | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # lsblk | ||
+ | NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT | ||
+ | loop0 | ||
+ | sda | ||
+ | ├─sda1 | ||
+ | ├─sda2 | ||
+ | └─sda5 | ||
+ | sdb | ||
+ | ├─sdb1 | ||
+ | ├─sdb2 | ||
+ | └─sdb5 | ||
+ | sr0 11:0 1 1024M 0 rom | ||
+ | sr1 11:1 1 1024M 0 rom | ||
+ | sde | ||
+ | └─sde1 | ||
+ | ├─vg_scratch-home_zfs (dm-0) | ||
+ | ├─vg_scratch-lv_scratch-real (dm-3) 252:3 0 894.3G | ||
+ | │ ├─vg_scratch-lv_scratch (dm-2) | ||
+ | │ └─vg_scratch-snap (dm-1) | ||
+ | ├─vg_scratch-snap-cow (dm-4) | ||
+ | │ └─vg_scratch-snap (dm-1) | ||
+ | └─vg_scratch-photorec (dm-5) | ||
+ | </ | ||
+ | |||
+ | It's probably ''/ | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # file -Ls /dev/sdb5 | ||
+ | /dev/sdb5: no read permission | ||
+ | |||
+ | # meermanr@Ikari:/ | ||
+ | # sudo !! | ||
+ | sudo file -Ls /dev/sdb5 | ||
+ | [sudo] password for meermanr: | ||
+ | /dev/sdb5: Linux/i386 swap file (new style), version 1 (4K pages), size 1048063 pages, no label, UUID=d0bbff73-a09a-47f6-8387-e27268cdc9fc | ||
+ | </ | ||
+ | |||
+ | Great! Let's enable it! | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # sudo swapon /dev/sdb5 | ||
+ | </ | ||
+ | |||
+ | And verify: | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # lsblk | grep SWAP | ||
+ | └─sda5 | ||
+ | └─sdb5 | ||
+ | </ | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # free -m | ||
+ | | ||
+ | Mem: 5969 | ||
+ | -/+ buffers/ | ||
+ | Swap: 10234 | ||
+ | </ | ||
+ | |||
+ | ===== Extending the snapshot volume ===== | ||
+ | |||
+ | < | ||
+ | # lvdisplay / | ||
+ | |||
+ | --- Logical volume --- | ||
+ | LV Name / | ||
+ | VG Name vg_scratch | ||
+ | LV UUID 4EFJ8Y-bzWT-aif4-MlT9-4234-aS1d-qcipq0 | ||
+ | LV Write Access | ||
+ | LV snapshot status | ||
+ | LV Status | ||
+ | # open 1 | ||
+ | LV Size 894.27 GiB | ||
+ | Current LE | ||
+ | COW-table size 9.63 GiB | ||
+ | COW-table LE 2335 | ||
+ | Allocated to snapshot | ||
+ | Snapshot chunk size 4.00 KiB | ||
+ | Segments | ||
+ | Allocation | ||
+ | Read ahead sectors | ||
+ | - currently set to 256 | ||
+ | Block device | ||
+ | </ | ||
+ | |||
+ | As it happens, I had not allocated all of the volume group: | ||
+ | |||
+ | < | ||
+ | root@Ikari: | ||
+ | VG #PV #LV #SN Attr VSize VFree | ||
+ | vg_scratch | ||
+ | </ | ||
+ | |||
+ | So extending the snapshot is easy: | ||
+ | |||
+ | < | ||
+ | root@Ikari: | ||
+ | </ | ||
+ | |||
+ | Verify: | ||
+ | |||
+ | < | ||
+ | # lvdisplay / | ||
+ | --- Logical volume --- | ||
+ | LV Name / | ||
+ | VG Name vg_scratch | ||
+ | LV UUID 4EFJ8Y-bzWT-aif4-MlT9-4234-aS1d-qcipq0 | ||
+ | LV Write Access | ||
+ | LV snapshot status | ||
+ | LV Status | ||
+ | # open 1 | ||
+ | LV Size 894.27 GiB | ||
+ | Current LE | ||
+ | COW-table size 18.63 GiB <-- Has increased | ||
+ | COW-table LE | ||
+ | Allocated to snapshot | ||
+ | Snapshot chunk size 4.00 KiB | ||
+ | Segments | ||
+ | Allocation | ||
+ | Read ahead sectors | ||
+ | - currently set to 256 | ||
+ | Block device | ||
+ | </ | ||
+ | |||
+ | ===== ... 2 weeks later ===== | ||
+ | |||
+ | It has been two weeks since I started '' | ||
+ | |||
+ | Now: | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # date | ||
+ | Mon May 27 14:53:34 BST 2013 | ||
+ | </ | ||
+ | |||
+ | Size of block device: | ||
+ | |||
+ | < | ||
+ | root@Ikari:/ | ||
+ | Python 2.7.3 (default, Aug 1 2012, 05: | ||
+ | [GCC 4.6.3] on linux2 | ||
+ | Type " | ||
+ | >>> | ||
+ | >>> | ||
+ | >>> | ||
+ | 960218560000 | ||
+ | >>> | ||
+ | ' | ||
+ | >>> | ||
+ | </ | ||
+ | |||
+ | Offsets of '' | ||
+ | |||
+ | < | ||
+ | Every 2.0s: lsof / | ||
+ | |||
+ | COMMAND | ||
+ | fsck.ext2 20182 root 4u | ||
+ | python | ||
+ | </ | ||
+ | |||
+ | That's approximately 28%. :-( | ||
+ | |||
+ | < | ||
+ | | ||
+ | May 13 0.0 0.0 364 su | ||
+ | May 13 0.0 0.0 | ||
+ | May 13 0.0 0.0 1320 \_ watch lvdisplay / | ||
+ | May 13 0.0 0.0 364 su | ||
+ | May 13 0.0 0.0 | ||
+ | May 13 0.0 0.0 | ||
+ | May 13 99.1 24.3 1488468 | ||
+ | May 14 0.0 0.0 1456 watch lsof /dev/loop0 | ||
+ | </ | ||
+ | |||
+ | So '' | ||
+ | |||
+ | < | ||
+ | PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND | ||
+ | 20182 root 30 10 4696m 1.4g 784 R 94 24.3 19564:51 fsck.ext2 -y / | ||
+ | </ | ||
+ | |||
+ | Total system memory: | ||
+ | |||
+ | < | ||
+ | # meermanr@Ikari:/ | ||
+ | # free -m | ||
+ | | ||
+ | Mem: 5969 | ||
+ | -/+ buffers/ | ||
+ | Swap: 10234 | ||
+ | </ | ||
+ | |||
+ | < | ||
+ | Every 2.0s: lvdisplay / | ||
+ | |||
+ | File descriptor 4 (pipe: | ||
+ | --- Logical volume --- | ||
+ | LV Name / | ||
+ | VG Name vg_scratch | ||
+ | LV UUID 4EFJ8Y-bzWT-aif4-MlT9-4234-aS1d-qcipq0 | ||
+ | LV Write Access | ||
+ | LV snapshot status | ||
+ | LV Status | ||
+ | # open 1 | ||
+ | LV Size 894.27 GiB | ||
+ | Current LE | ||
+ | COW-table size 84.89 GiB | ||
+ | COW-table LE 21733 | ||
+ | Allocated to snapshot | ||
+ | Snapshot chunk size 4.00 KiB | ||
+ | Segments | ||
+ | Allocation | ||
+ | Read ahead sectors | ||
+ | - currently set to 256 | ||
+ | Block device | ||
+ | </ | ||
+ | |||
+ | Output from '' | ||
+ | |||
+ | < | ||
+ | File ... (inode #9791282, mod time Thu Oct 5 01:40:26 2006) | ||
+ | has 11143 multiply-claimed block(s), shared with 5 file(s): | ||
+ | < | ||
+ | ... (inode #9791794, mod time Thu Oct 5 01:40:26 2006) | ||
+ | ... (inode #4115835, mod time Thu Aug 20 03:31:06 2009) | ||
+ | ... (inode #4130006, mod time Mon Nov 29 16:38:10 2010) | ||
+ | ... (inode #4784754, mod time Tue Jul 26 06:01:10 2005) | ||
+ | Clone multiply-claimed blocks? yes | ||
+ | </ |