====== FreeBSDのnvmecontrolコマンドでNVMe SSDをローレベルフォーマットしてみる ======
FreeBSDの''nvmecontrol''コマンドでNVMeデバイスをフォーマットしてみた記録。
別マシンでは読み込み3GB/sほど出るSSDがFreeBSDマシンでは1.6GB/sほどしか出ず、初期化したら速度回復すんじゃね？という安直な気持ちでformatしてみたら、色々と興味深かったのでメモがてら残しておく。
なお、初期化してみたけど速度は回復しなかった。というか、1.6GB/sはPCIe 2.0×4接続時の速度上限っぽい雰囲気なんだけど、間違いなくPCIe 3.0×4接続になってるし、PCIeスロット変えても変化無いしだし謎。
（2019-10-06 追記） \\
その後、どう足掻いても1.6GB/sを超えなかった方は、実際に使いだしたら2GB/s超えるようになったり、逆に元から3GB/s出てた方は遅くなったりして、NVMeなんもわからん。そもそもnvmecontrolのベンチマーク機能は、特定の領域をひたすらアクセスするだけの原始的なものらしく、結果に一喜一憂しても仕方ないと思い至った。参考値として見るに留めるのが吉。
===== 試した環境 =====
* FreeBSD 12.0-RELEASE-p4
* SAMSUNG MZQLB960HAJR-00007 (PM983 960GB)
* smartctl 7.0
===== 作業の記録 =====
==== 初期化前の状態 ====
> sudo smartctl -a /dev/nvme1
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE-p4 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: SAMSUNG MZQLB960HAJR-00007
Serial Number: S437NY0KC00XYZ
Firmware Version: EDA5202Q
PCI Vendor/Subsystem ID: 0x144d
IEEE OUI Identifier: 0x002538
Total NVM Capacity: 960,197,124,096 [960 GB]
Unallocated NVM Capacity: 0
Controller ID: 4
Number of Namespaces: 1
Namespace 1 Size/Capacity: 960,197,124,096 [960 GB]
Namespace 1 Utilization: 1,232,384,000 [1.23 GB]
Namespace 1 Formatted LBA Size: 512
Local Time is: Thu Sep 19 00:38:43 2019 JST
Firmware Updates (0x17): 3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f): Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size: 512 Pages
Warning Comp. Temp. Threshold: 87 Celsius
Critical Comp. Temp. Threshold: 88 Celsius
Namespace 1 Features (0x02): NA_Fields
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 10.60W - - 0 0 0 0 0 0
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 + 512 0 0
1 - 4096 0 0
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 41 Celsius
Available Spare: 100%
Available Spare Threshold: 10%
Percentage Used: 0%
Data Units Read: 36,935,426 [18.9 TB]
Data Units Written: 4,227,826 [2.16 TB]
Host Read Commands: 2,288,568,067
Host Write Commands: 96,108,724
Controller Busy Time: 241
Power Cycles: 417
Power On Hours: 2,260
Unsafe Shutdowns: 395
Media and Data Integrity Errors: 0
Error Information Log Entries: 11
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Temperature Sensor 1: 41 Celsius
Temperature Sensor 2: 48 Celsius
Temperature Sensor 3: 57 Celsius
Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged
今回初めて知ったんだけど、NVMeではnamespaceなる仕組みでデバイスを論理的に分割できるそうだ。要はパーティションのようなものだが、namespace毎に論理セクタサイズを変えたりできるらしい。
S.M.A.R.Tの中で気になったのは「Namespace 1 Utilization」という項目。名前の通り、当該namespaceの使用量を表している。デバイス側で使用量を持ってる事に驚いた。ファイルシステム側のファイル作成・削除と連動するのかは不明。なんとなく連動はしてない気はするが、これだけ高機能ならもしかしてもしかするのかも？
==== 名前空間をフォーマット ====
''nvmecontrol format 名前空間名''で、namespaceを初期化してみる。Namespace 1 Utilizationが14.4MBに減った。
> sudo nvmecontrol format nvme1ns1
> sudo smartctl -a /dev/nvme1
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE-p4 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: SAMSUNG MZQLB960HAJR-00007
Serial Number: S437NY0KC00XYZ
Firmware Version: EDA5202Q
PCI Vendor/Subsystem ID: 0x144d
IEEE OUI Identifier: 0x002538
Total NVM Capacity: 960,197,124,096 [960 GB]
Unallocated NVM Capacity: 0
Controller ID: 4
Number of Namespaces: 1
Namespace 1 Size/Capacity: 960,197,124,096 [960 GB]
Namespace 1 Utilization: 14,426,112 [14.4 MB]
Namespace 1 Formatted LBA Size: 512
Local Time is: Thu Sep 19 00:41:04 2019 JST
Firmware Updates (0x17): 3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f): Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size: 512 Pages
Warning Comp. Temp. Threshold: 87 Celsius
Critical Comp. Temp. Threshold: 88 Celsius
Namespace 1 Features (0x02): NA_Fields
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 10.60W - - 0 0 0 0 0 0
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 + 512 0 0
1 - 4096 0 0
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 42 Celsius
Available Spare: 100%
Available Spare Threshold: 10%
Percentage Used: 0%
Data Units Read: 36,935,426 [18.9 TB]
Data Units Written: 4,227,826 [2.16 TB]
Host Read Commands: 2,288,568,067
Host Write Commands: 96,108,724
Controller Busy Time: 241
Power Cycles: 417
Power On Hours: 2,260
Unsafe Shutdowns: 395
Media and Data Integrity Errors: 0
Error Information Log Entries: 11
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Temperature Sensor 1: 42 Celsius
Temperature Sensor 2: 48 Celsius
Temperature Sensor 3: 56 Celsius
Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged
==== デバイスをフォーマット ====
''nvmecontrol format NVMeデバイスファイル''でNVMeデバイス全体をフォーマットしてみる。名前空間が無くなっていることがわかる。
> sudo nvmecontrol format nvme1
> sudo smartctl -a /dev/nvme1
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE-p4 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: SAMSUNG MZQLB960HAJR-00007
Serial Number: S437NY0KC00XYZ
Firmware Version: EDA5202Q
PCI Vendor/Subsystem ID: 0x144d
IEEE OUI Identifier: 0x002538
Total NVM Capacity: 960,197,124,096 [960 GB]
Unallocated NVM Capacity: 0
Controller ID: 4
Number of Namespaces: 1
Namespace 1 Size/Capacity: 960,197,124,096 [960 GB]
Namespace 1 Utilization: 0
Namespace 1 Formatted LBA Size: 512
Local Time is: Thu Sep 19 00:42:30 2019 JST
Firmware Updates (0x17): 3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f): Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size: 512 Pages
Warning Comp. Temp. Threshold: 87 Celsius
Critical Comp. Temp. Threshold: 88 Celsius
Namespace 1 Features (0x02): NA_Fields
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 10.60W - - 0 0 0 0 0 0
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 + 512 0 0
1 - 4096 0 0
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 42 Celsius
Available Spare: 100%
Available Spare Threshold: 10%
Percentage Used: 0%
Data Units Read: 36,938,628 [18.9 TB]
Data Units Written: 4,227,826 [2.16 TB]
Host Read Commands: 2,288,580,574
Host Write Commands: 96,108,724
Controller Busy Time: 241
Power Cycles: 417
Power On Hours: 2,260
Unsafe Shutdowns: 395
Media and Data Integrity Errors: 0
Error Information Log Entries: 11
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Temperature Sensor 1: 42 Celsius
Temperature Sensor 2: 48 Celsius
Temperature Sensor 3: 57 Celsius
Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged
==== デバイスをSecure Erase ====
''nvmecontrol format -E NVMeデバイスファイル''でNVMeデバイスをUser Data Eraseしてみる。いわゆるSecure Eraseに相当するらしい。
> sudo nvmecontrol format -E nvme1
> sudo smartctl -a /dev/nvme1
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE-p4 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: SAMSUNG MZQLB960HAJR-00007
Serial Number: S437NY0KC00XYZ
Firmware Version: EDA5202Q
PCI Vendor/Subsystem ID: 0x144d
IEEE OUI Identifier: 0x002538
Total NVM Capacity: 960,197,124,096 [960 GB]
Unallocated NVM Capacity: 0
Controller ID: 4
Number of Namespaces: 1
Namespace 1 Size/Capacity: 960,197,124,096 [960 GB]
Namespace 1 Utilization: 0
Namespace 1 Formatted LBA Size: 512
Local Time is: Thu Sep 19 00:43:44 2019 JST
Firmware Updates (0x17): 3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f): Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size: 512 Pages
Warning Comp. Temp. Threshold: 87 Celsius
Critical Comp. Temp. Threshold: 88 Celsius
Namespace 1 Features (0x02): NA_Fields
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 10.60W - - 0 0 0 0 0 0
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 + 512 0 0
1 - 4096 0 0
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 44 Celsius
Available Spare: 100%
Available Spare Threshold: 10%
Percentage Used: 0%
Data Units Read: 36,945,030 [18.9 TB]
Data Units Written: 4,227,826 [2.16 TB]
Host Read Commands: 2,288,605,580
Host Write Commands: 96,108,724
Controller Busy Time: 241
Power Cycles: 417
Power On Hours: 2,260
Unsafe Shutdowns: 395
Media and Data Integrity Errors: 0
Error Information Log Entries: 11
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Temperature Sensor 1: 44 Celsius
Temperature Sensor 2: 50 Celsius
Temperature Sensor 3: 58 Celsius
Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged
==== LBAサイズを指定してフォーマット ====
''nvmecontrol format -f LBAサイズID 名前空間''でLBAサイズを指定して名前空間をフォーマットしてみる。LBAサイズIDはsmartctlのSupported LBA Sizesで表示されるIDを指定する模様。
使ったSSDでは512バイト、4096バイトの一般的なサイズのサポートとなっているが、チェックサム用領域8バイトを含んだ520バイトや4104バイトセクタなどに対応してる製品もあるらしい。
Namespace 1 Formatted LBA Sizeが4096に変わったことが分かる。
なお、LBAサイズの変更はOS側でハンドリングされてないようで、しばらくするとカーネルパニックするので直ぐに再起動すべし（FreeBSD 12.0Rp4現在）
> sudo nvmecontrol format -f 1 nvme0ns1
Decomo@freyja /u/h/Decomo> sudo smartctl -a /dev/nvme0
smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE-p4 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: SAMSUNG MZQLB960HAJR-00007
Serial Number: S437NY0KC00XYZ
Firmware Version: EDA5202Q
PCI Vendor/Subsystem ID: 0x144d
IEEE OUI Identifier: 0x002538
Total NVM Capacity: 960,197,124,096 [960 GB]
Unallocated NVM Capacity: 0
Controller ID: 4
Number of Namespaces: 1
Namespace 1 Size/Capacity: 960,197,124,096 [960 GB]
Namespace 1 Utilization: 24,576 [24.5 KB]
Namespace 1 Formatted LBA Size: 4096
Local Time is: Thu Sep 19 10:38:21 2019 JST
Firmware Updates (0x17): 3 Slots, Slot 1 R/O, no Reset required
Optional Admin Commands (0x000f): Security Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x001f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size: 512 Pages
Warning Comp. Temp. Threshold: 87 Celsius
Critical Comp. Temp. Threshold: 88 Celsius
Namespace 1 Features (0x02): NA_Fields
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 10.60W - - 0 0 0 0 0 0
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 - 512 0 0
1 + 4096 0 0
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 39 Celsius
Available Spare: 100%
Available Spare Threshold: 10%
Percentage Used: 0%
Data Units Read: 37,246,040 [19.0 TB]
Data Units Written: 4,335,090 [2.21 TB]
Host Read Commands: 2,298,018,184
Host Write Commands: 102,873,515
Controller Busy Time: 244
Power Cycles: 425
Power On Hours: 2,269
Unsafe Shutdowns: 399
Media and Data Integrity Errors: 0
Error Information Log Entries: 14
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Temperature Sensor 1: 39 Celsius
Temperature Sensor 2: 45 Celsius
Temperature Sensor 3: 52 Celsius
Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged
===== 参考サイト =====
* [[https://qiita.com/ken-yossy/items/ae42c06d8c9face76934|NVMe Revision 1.4: New commands and features (except for NVM Sets and its related features) - Qiita]]