FreeBSDのnvmecontrolコマンドでNVMe SSDをローレベルフォーマットしてみる

FreeBSDのnvmecontrolコマンドでNVMeデバイスをフォーマットしてみた記録。

別マシンでは読み込み3GB/sほど出るSSDがFreeBSDマシンでは1.6GB/sほどしか出ず、初期化したら速度回復すんじゃね?という安直な気持ちでformatしてみたら、色々と興味深かったのでメモがてら残しておく。

なお、初期化してみたけど速度は回復しなかった。というか、1.6GB/sはPCIe 2.0×4接続時の速度上限っぽい雰囲気なんだけど、間違いなくPCIe 3.0×4接続になってるし、PCIeスロット変えても変化無いしだし謎。

(2019-10-06 追記)
その後、どう足掻いても1.6GB/sを超えなかった方は、実際に使いだしたら2GB/s超えるようになったり、逆に元から3GB/s出てた方は遅くなったりして、NVMeなんもわからん。そもそもnvmecontrolのベンチマーク機能は、特定の領域をひたすらアクセスするだけの原始的なものらしく、結果に一喜一憂しても仕方ないと思い至った。参考値として見るに留めるのが吉。

  • FreeBSD 12.0-RELEASE-p4
  • SAMSUNG MZQLB960HAJR-00007 (PM983 960GB)
  • smartctl 7.0
> sudo smartctl -a /dev/nvme1
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE-p4 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZQLB960HAJR-00007
Serial Number:                      S437NY0KC00XYZ
Firmware Version:                   EDA5202Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 960,197,124,096 [960 GB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          960,197,124,096 [960 GB]
Namespace 1 Utilization:            1,232,384,000 [1.23 GB]
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Thu Sep 19 00:38:43 2019 JST
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f):   Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     87 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +    10.60W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0
 1 -    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        41 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    36,935,426 [18.9 TB]
Data Units Written:                 4,227,826 [2.16 TB]
Host Read Commands:                 2,288,568,067
Host Write Commands:                96,108,724
Controller Busy Time:               241
Power Cycles:                       417
Power On Hours:                     2,260
Unsafe Shutdowns:                   395
Media and Data Integrity Errors:    0
Error Information Log Entries:      11
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               41 Celsius
Temperature Sensor 2:               48 Celsius
Temperature Sensor 3:               57 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

今回初めて知ったんだけど、NVMeではnamespaceなる仕組みでデバイスを論理的に分割できるそうだ。要はパーティションのようなものだが、namespace毎に論理セクタサイズを変えたりできるらしい。

S.M.A.R.Tの中で気になったのは「Namespace 1 Utilization」という項目。名前の通り、当該namespaceの使用量を表している。デバイス側で使用量を持ってる事に驚いた。ファイルシステム側のファイル作成・削除と連動するのかは不明。なんとなく連動はしてない気はするが、これだけ高機能ならもしかしてもしかするのかも?

nvmecontrol format 名前空間名で、namespaceを初期化してみる。Namespace 1 Utilizationが14.4MBに減った。

> sudo nvmecontrol format nvme1ns1
> sudo smartctl -a /dev/nvme1
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE-p4 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZQLB960HAJR-00007
Serial Number:                      S437NY0KC00XYZ
Firmware Version:                   EDA5202Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 960,197,124,096 [960 GB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          960,197,124,096 [960 GB]
Namespace 1 Utilization:            14,426,112 [14.4 MB]
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Thu Sep 19 00:41:04 2019 JST
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f):   Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     87 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +    10.60W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0
 1 -    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        42 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    36,935,426 [18.9 TB]
Data Units Written:                 4,227,826 [2.16 TB]
Host Read Commands:                 2,288,568,067
Host Write Commands:                96,108,724
Controller Busy Time:               241
Power Cycles:                       417
Power On Hours:                     2,260
Unsafe Shutdowns:                   395
Media and Data Integrity Errors:    0
Error Information Log Entries:      11
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               42 Celsius
Temperature Sensor 2:               48 Celsius
Temperature Sensor 3:               56 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

nvmecontrol format NVMeデバイスファイルでNVMeデバイス全体をフォーマットしてみる。名前空間が無くなっていることがわかる。

> sudo nvmecontrol format nvme1
> sudo smartctl -a /dev/nvme1
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE-p4 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZQLB960HAJR-00007
Serial Number:                      S437NY0KC00XYZ
Firmware Version:                   EDA5202Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 960,197,124,096 [960 GB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          960,197,124,096 [960 GB]
Namespace 1 Utilization:            0
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Thu Sep 19 00:42:30 2019 JST
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f):   Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     87 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +    10.60W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0
 1 -    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        42 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    36,938,628 [18.9 TB]
Data Units Written:                 4,227,826 [2.16 TB]
Host Read Commands:                 2,288,580,574
Host Write Commands:                96,108,724
Controller Busy Time:               241
Power Cycles:                       417
Power On Hours:                     2,260
Unsafe Shutdowns:                   395
Media and Data Integrity Errors:    0
Error Information Log Entries:      11
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               42 Celsius
Temperature Sensor 2:               48 Celsius
Temperature Sensor 3:               57 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

nvmecontrol format -E NVMeデバイスファイルでNVMeデバイスをUser Data Eraseしてみる。いわゆるSecure Eraseに相当するらしい。

> sudo nvmecontrol format -E nvme1
> sudo smartctl -a /dev/nvme1
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE-p4 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZQLB960HAJR-00007
Serial Number:                      S437NY0KC00XYZ
Firmware Version:                   EDA5202Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 960,197,124,096 [960 GB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          960,197,124,096 [960 GB]
Namespace 1 Utilization:            0
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Thu Sep 19 00:43:44 2019 JST
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f):   Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     87 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +    10.60W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0
 1 -    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        44 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    36,945,030 [18.9 TB]
Data Units Written:                 4,227,826 [2.16 TB]
Host Read Commands:                 2,288,605,580
Host Write Commands:                96,108,724
Controller Busy Time:               241
Power Cycles:                       417
Power On Hours:                     2,260
Unsafe Shutdowns:                   395
Media and Data Integrity Errors:    0
Error Information Log Entries:      11
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               44 Celsius
Temperature Sensor 2:               50 Celsius
Temperature Sensor 3:               58 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

nvmecontrol format -f LBAサイズID 名前空間でLBAサイズを指定して名前空間をフォーマットしてみる。LBAサイズIDはsmartctlのSupported LBA Sizesで表示されるIDを指定する模様。

使ったSSDでは512バイト、4096バイトの一般的なサイズのサポートとなっているが、チェックサム用領域8バイトを含んだ520バイトや4104バイトセクタなどに対応してる製品もあるらしい。

Namespace 1 Formatted LBA Sizeが4096に変わったことが分かる。

なお、LBAサイズの変更はOS側でハンドリングされてないようで、しばらくするとカーネルパニックするので直ぐに再起動すべし(FreeBSD 12.0Rp4現在)

> sudo nvmecontrol format -f 1 nvme0ns1
Decomo@freyja /u/h/Decomo> sudo smartctl -a /dev/nvme0
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE-p4 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZQLB960HAJR-00007
Serial Number:                      S437NY0KC00XYZ
Firmware Version:                   EDA5202Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 960,197,124,096 [960 GB]
Unallocated NVM Capacity:           0
Controller ID:                      4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          960,197,124,096 [960 GB]
Namespace 1 Utilization:            24,576 [24.5 KB]
Namespace 1 Formatted LBA Size:     4096
Local Time is:                      Thu Sep 19 10:38:21 2019 JST
Firmware Updates (0x17):            3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f):   Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     87 Celsius
Critical Comp. Temp. Threshold:     88 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +    10.60W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 -     512       0         0
 1 +    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        39 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    37,246,040 [19.0 TB]
Data Units Written:                 4,335,090 [2.21 TB]
Host Read Commands:                 2,298,018,184
Host Write Commands:                102,873,515
Controller Busy Time:               244
Power Cycles:                       425
Power On Hours:                     2,269
Unsafe Shutdowns:                   399
Media and Data Integrity Errors:    0
Error Information Log Entries:      14
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               39 Celsius
Temperature Sensor 2:               45 Celsius
Temperature Sensor 3:               52 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged
  • freebsd/freebsd_format_nvme_with_nvmecontrol.txt
  • 最終更新: 2020-03-10 11:06
  • by Decomo