mattintosh note

Hello Raspberry Pi!

久しぶりに HDD を買った

ネットワークストレージの空き領域が無くなったのでいい加減買い足し。地雷と言われてる ST4000DM004 にした。Western Digital はあまり好きじゃないし、昔ほど HDD の性能に拘りは無い。

いまはネットワークインターフェイスの関係で Raspberry Pi 3 から NanoPi NEO2 に移設している ST3000DM001 の 情報を久しぶりに見てみる。この製品は LaCie の LCH-MNR030U3 だった気がする。

$ sudo smartctl -d sat --all /dev/sdc
smartctl 6.5 2016-01-24 r4214 [aarch64-linux-4.14.0] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST3000DM001-1CH166
Serial Number:    
LU WWN Device Id: 5 000c50 072d3946e
Firmware Version: CC47
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Wed Oct 24 23:00:25 2018 JST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
See vendor-specific Attribute list for marginal Attributes.

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                    was completed without error.
                    Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                    without error or no self-test has ever 
                    been run.
Total time to complete Offline 
data collection:        (  575) seconds.
Offline data collection
capabilities:            (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                    General Purpose Logging supported.
Short self-test routine 
recommended polling time:    (   1) minutes.
Extended self-test routine
recommended polling time:    ( 317) minutes.
Conveyance self-test routine
recommended polling time:    (   2) minutes.
SCT capabilities:          (0x3085) SCT Status supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   115   099   006    Pre-fail  Always       -       93865696
  3 Spin_Up_Time            0x0003   095   092   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       341
  5 Reallocated_Sector_Ct   0x0033   099   099   010    Pre-fail  Always       -       1384
  7 Seek_Error_Rate         0x000f   072   060   030    Pre-fail  Always       -       17171518
  9 Power_On_Hours          0x0032   063   063   000    Old_age   Always       -       32806
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       316
183 Runtime_Bad_Block       0x0032   099   099   000    Old_age   Always       -       1
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   099   000    Old_age   Always       -       3 3 3
189 High_Fly_Writes         0x003a   098   098   000    Old_age   Always       -       2
190 Airflow_Temperature_Cel 0x0022   059   044   045    Old_age   Always   In_the_past 41 (Min/Max 26/46 #11)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       298
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       344
194 Temperature_Celsius     0x0022   041   056   000    Old_age   Always       -       41 (0 10 0 0 0)
197 Current_Pending_Sector  0x0012   081   081   000    Old_age   Always       -       3128
198 Offline_Uncorrectable   0x0010   081   081   000    Old_age   Offline      -       3128
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       32804h+35m+07.860s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       9655026104
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       6272904990

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

おや…不良セクタが。

んで ST4000DM004 の S.M.A.R.T 結果。

$ sudo smartctl -d sat --all /dev/sda
smartctl 6.5 2016-01-24 r4214 [aarch64-linux-4.14.0] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     ST4000DM004-2CV104
Serial Number:    
LU WWN Device Id: 5 000c50 0bae57ca7
Firmware Version: 0001
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5425 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Oct 24 23:16:50 2018 JST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART Status not supported: Incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                    was never started.
                    Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                    without error or no self-test has ever 
                    been run.
Total time to complete Offline 
data collection:        (    0) seconds.
Offline data collection
capabilities:            (0x73) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    No Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                    General Purpose Logging supported.
Short self-test routine 
recommended polling time:    (   1) minutes.
Extended self-test routine
recommended polling time:    ( 488) minutes.
Conveyance self-test routine
recommended polling time:    (   2) minutes.
SCT capabilities:          (0x30a5) SCT Status supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   082   065   006    Pre-fail  Always       -       155888737
  3 Spin_Up_Time            0x0003   096   096   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       10
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   064   060   045    Pre-fail  Always       -       2374933
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       74 (114 172 0)
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       2
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   062   060   040    Old_age   Always       -       38 (Min/Max 25/40)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       2
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       18
194 Temperature_Celsius     0x0022   038   040   000    Old_age   Always       -       38 (0 25 0 0 0)
195 Hardware_ECC_Recovered  0x001a   082   065   000    Old_age   Always       -       155888737
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       42 (161 240 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       2713040728
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       9811923

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

大容量 HDD は 1 TB 単位で使うことにしているので MBR でもいいんだけど一応 GPT。あと、サイズは 1 TiB ではなく 1TB にすることにしてる。

$ sudo gdisk -l /dev/sda
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 7814037168 sectors, 3.6 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 720AE297-8557-4BAD-85E0-9514702E8D91
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 7814037134
Partitions will be aligned on 2048-sector boundaries
Total free space is 1537101 sectors (750.5 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      1953127047   931.3 GiB   8300  Linux filesystem
   2      1953128448      3906253447   931.3 GiB   8300  Linux filesystem
   3      3906254848      5859379847   931.3 GiB   8300  Linux filesystem
   4      5859381248      7812506247   931.3 GiB   8300  Linux filesystem

いつもの癖でそのまま mkfs.ext4 したらなんか stripe=8191 とか付いてるから tune2fs で見てみる。

$ mount -t ext4
/dev/mmcblk0p2 on / type ext4 (rw,relatime,data=ordered)
/dev/sdb1 on /media/HDCL-UT1 type ext4 (rw,noatime,data=ordered)
/dev/sdc1 on /media/ST3000DM001-1 type ext4 (rw,noatime,data=ordered)
/dev/sdc2 on /media/ST3000DM001-2 type ext4 (rw,noatime,data=ordered)
/dev/sdc3 on /media/ST3000DM001-3 type ext4 (rw,noatime,data=ordered)
/dev/sda1 on /media/ST4000DM004-1 type ext4 (rw,noatime,stripe=8191,data=ordered)
/dev/sda2 on /media/ST4000DM004-2 type ext4 (rw,noatime,stripe=8191,data=ordered)

なんで RAID stripe width 付くの…。

$ tune2fs -l /dev/sda1
tune2fs 1.42.13 (17-May-2015)
Filesystem volume name:   ST4000DM004-1
Last mounted on:          /media/ST4000DM004-1
Filesystem UUID:          763e8fa3-1bf1-448f-b3c6-4915e144c2b2
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              61038592
Block count:              244140625
Reserved block count:     12207031
Free blocks:              162198379
Free inodes:              60786672
First block:              0
Block size:               4096
Fragment size:            4096
Group descriptor size:    64
Reserved GDT blocks:      1024
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
RAID stripe width:        8191
Flex block group size:    16
Filesystem created:       Sun Oct 21 21:27:39 2018
Last mount time:          Fri Feb 12 01:28:03 2016
Last write time:          Fri Feb 12 01:28:03 2016
Mount count:              3
Maximum mount count:      -1
Last checked:             Sun Oct 21 21:27:39 2018
Check interval:           0 (<none>)
Lifetime writes:          313 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:           256
Required extra isize:     32
Desired extra isize:      32
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      fff5cb63-86e3-4e0b-996e-78a6c3c4fbdf
Journal backup:           inode blocks
Checksum:                 0xffeec98a

多分 Raspberry Pi(Arch Linux だったか?)でフォーマットしたものと NanoPi NEO2(Ubuntu)でフォーマットしたものの差分。Filesystem feature に 64bit が入ってるなぁ。

$ diff -u <(tune2fs -l /dev/sdc1) <(tune2fs -l /dev/sda1)
--- /dev/fd/63    2018-10-24 23:47:34.016343443 +0900
+++ /dev/fd/62    2018-10-24 23:47:34.020343277 +0900
@@ -1,10 +1,10 @@
 tune2fs 1.42.13 (17-May-2015)
-Filesystem volume name:   ST3000DM001-1
-Last mounted on:          /media/ST3000DM001-1
-Filesystem UUID:          c1b88656-446a-4083-9c22-69f49146cf91
+Filesystem volume name:   ST4000DM004-1
+Last mounted on:          /media/ST4000DM004-1
+Filesystem UUID:          763e8fa3-1bf1-448f-b3c6-4915e144c2b2
 Filesystem magic number:  0xEF53
 Filesystem revision #:    1 (dynamic)
-Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
+Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
 Filesystem flags:         signed_directory_hash 
 Default mount options:    user_xattr acl
 Filesystem state:         clean
@@ -13,32 +13,35 @@
 Inode count:              61038592
 Block count:              244140625
 Reserved block count:     12207031
-Free blocks:              12663554
-Free inodes:              60732730
+Free blocks:              162198379
+Free inodes:              60786672
 First block:              0
 Block size:               4096
 Fragment size:            4096
-Reserved GDT blocks:      965
+Group descriptor size:    64
+Reserved GDT blocks:      1024
 Blocks per group:         32768
 Fragments per group:      32768
 Inodes per group:         8192
 Inode blocks per group:   512
+RAID stripe width:        8191
 Flex block group size:    16
-Filesystem created:       Mon Jan 12 22:02:10 2015
-Last mount time:          Fri Feb 12 01:28:02 2016
-Last write time:          Fri Feb 12 01:28:02 2016
-Mount count:              76
+Filesystem created:       Sun Oct 21 21:27:39 2018
+Last mount time:          Fri Feb 12 01:28:03 2016
+Last write time:          Fri Feb 12 01:28:03 2016
+Mount count:              3
 Maximum mount count:      -1
-Last checked:             Mon Feb 22 23:42:01 2016
+Last checked:             Sun Oct 21 21:27:39 2018
 Check interval:           0 (<none>)
-Lifetime writes:          2027 GB
+Lifetime writes:          313 GB
 Reserved blocks uid:      0 (user root)
 Reserved blocks gid:      0 (group root)
 First inode:              11
 Inode size:              256
-Required extra isize:     28
-Desired extra isize:      28
+Required extra isize:     32
+Desired extra isize:      32
 Journal inode:            8
 Default directory hash:   half_md4
-Directory Hash Seed:      367fc8dd-0a4d-4fa7-90ea-77026a99d6a1
+Directory Hash Seed:      fff5cb63-86e3-4e0b-996e-78a6c3c4fbdf
 Journal backup:           inode blocks
+Checksum:                 0xffeec98a

パーティションのセクター数は同じなのにファイルシステムのサイズ微妙に違うし。ディストリによって /etc/mke2fs.conf って違うのか。

$ df -H -t ext4
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p2   16G  3.6G   12G  24% /
/dev/sda1       984G  323G  612G  35% /media/ST4000DM004-1
/dev/sda2       984G  898G   36G  97% /media/ST4000DM004-2
/dev/sdb1       985G  882G   53G  95% /media/HDCL-UT1
/dev/sdc1       985G  933G  1.4G 100% /media/ST3000DM001-1
/dev/sdc2       985G  926G  8.2G 100% /media/ST3000DM001-2
/dev/sdc3       985G  927G  7.5G 100% /media/ST3000DM001-3

そしてデータ移動しつつ1日経って気づいたけど i-node のサイズ変えるの忘れてた(´・ω・`) もう 1 TB 分データ移動したのに…。i-node 余り過ぎやん。

$ df -i -t ext4
Filesystem        Inodes  IUsed     IFree IUse% Mounted on
/dev/mmcblk0p2    906240  68652    837588    8% /
/dev/sda1       61038592 260739  60777853    1% /media/ST4000DM004-1
/dev/sda2       61038592 541012  60497580    1% /media/ST4000DM004-2
/dev/sdb1       61054976 540946  60514030    1% /media/HDCL-UT1
/dev/sdc1       61038592 306389  60732203    1% /media/ST3000DM001-1
/dev/sdc2       61038592 473815  60564777    1% /media/ST3000DM001-2
/dev/sdc3       61038592 322747  60715845    1% /media/ST3000DM001-3

てことで残りの 2 パーティションを xfs にした。

$ df -HT -t ext4 -t xfs
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/mmcblk0p2 ext4   16G  3.6G   12G  24% /
/dev/sda1      ext4  984G  323G  612G  35% /media/ST4000DM004-1
/dev/sda2      ext4  984G  898G   36G  97% /media/ST4000DM004-2
/dev/sda3      xfs   1.0T  148G  853G  15% /media/ST4000DM004-3
/dev/sda4      xfs   1.0T  1.1G  999G   1% /media/ST4000DM004-4
/dev/sdb1      ext4  985G  882G   53G  95% /media/HDCL-UT1
/dev/sdc1      ext4  985G  933G  1.4G 100% /media/ST3000DM001-1
/dev/sdc2      ext4  985G  926G  8.2G 100% /media/ST3000DM001-2
/dev/sdc3      ext4  985G  927G  7.5G 100% /media/ST3000DM001-3

ん???

ext4 と xfs って 1 TB だと 15 GB もサイズに差出るのかよ…。

※ST4000DM004-3 はデータ移動中なので使用量が増えてるけど ST4000DM004-4 は未使用。xfs は未使用でも初期段階で 1.1 GB 使ってる。

一応 i-node 数も見ておく。

$ df -iT -t ext4 -t xfs
Filesystem     Type    Inodes  IUsed     IFree IUse% Mounted on
/dev/mmcblk0p2 ext4    906240  68653    837587    8% /
/dev/sda1      ext4  61038592 260958  60777634    1% /media/ST4000DM004-1
/dev/sda2      ext4  61038592 541012  60497580    1% /media/ST4000DM004-2
/dev/sda3      xfs  488281216  13972 488267244    1% /media/ST4000DM004-3
/dev/sda4      xfs  488281216      3 488281213    1% /media/ST4000DM004-4
/dev/sdb1      ext4  61054976 540946  60514030    1% /media/HDCL-UT1
/dev/sdc1      ext4  61038592 306389  60732203    1% /media/ST3000DM001-1
/dev/sdc2      ext4  61038592 473815  60564777    1% /media/ST3000DM001-2
/dev/sdc3      ext4  61038592 322747  60715845    1% /media/ST3000DM001-3

もう xfs でいいや(´・ω・`)


で、rsync で sdb1 から sdb2、sdb3 にそれぞれ同じデータ量を書き込んだんだけど差がありすぎる。sdb は 512 byte セクタで sda は 4 K セクタでまぁ変わるものなんだろうが、ext4 と xfs で 65 GB も差が出た。

Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sdb1      ext4  985G  882G   53G  95% /media/HDCL-UT1
/dev/sda2      ext4  984G  898G   36G  97% /media/ST4000DM004-2
/dev/sda3      xfs   1.0T  899G  101G  90% /media/ST4000DM004-3

Filesystem     Type    Inodes  IUsed     IFree IUse% Mounted on
/dev/sdb1      ext4  61054976 540946  60514030    1% /media/HDCL-UT1
/dev/sda2      ext4  61038592 541012  60497580    1% /media/ST4000DM004-2
/dev/sda3      xfs  196981880 541004 196440876    1% /media/ST4000DM004-3