vmware-samples / nvme Goto Github PK

see also
[(https://communities.intel.com/thread/97954?start=0&tstart=0)]

#esxcli storage core device vaai status get
t10.NVMe____INTEL_SSDPEDME400G4_*******__00000001
   VAAI Plugin Name:
   ATS Status: unsupported
   Clone Status: unsupported
   Zero Status: unsupported
   Delete Status: unsupported

Can you provide a mask example, or even a table, for the appropriate hex values to enable the various debug bits?

nvme-1.2.0.27-4vmw.550.0.0.1331820.x86_64.vib
6.0 U2
HP Microserver Gen8
INTEL NVME P3600

dmesg:

2016-05-22T10:07:33.017Z cpu0:35374)nvme:nvmeCoreLogError:333:command failed: 0x4302fea0d040.
2016-05-22T10:09:44.427Z cpu0:33196)nvme:nvmeCoreLogError:333:command failed: 0x4302fea0d540.
2016-05-22T10:11:45.263Z cpu0:32781)nvme:nvmeCoreLogError:333:command failed: 0x4302fea0da40.
2016-05-22T10:13:46.020Z cpu0:35374)nvme:nvmeCoreLogError:333:command failed: 0x4302fea0df40.
2016-05-22T10:15:46.521Z cpu0:35374)nvme:nvmeCoreLogError:333:command failed: 0x4302fea0e440.

After removing partitions and detach:
2016-05-22T10:38:56.275Z cpu1:34322 opID=6d0c3e71)ScsiDevice: 1741: Device t10.NVMe____INTEL_SSDPEDME400G4_____________________CVMD517401J4400AGN__00000001  has been turned off administratively.
2016-05-22T10:39:57.151Z cpu0:35374)nvme:nvmeCoreLogError:333:command failed: 0x4302fea12180.
2016-05-22T10:41:58.321Z cpu0:33198)nvme:nvmeCoreLogError:333:command failed: 0x4302fea12680.
2016-05-22T10:43:59.370Z cpu0:32781)nvme:nvmeCoreLogError:333:command failed: 0x4302fea12b80.
2016-05-22T10:45:59.979Z cpu0:35374)nvme:nvmeCoreLogError:333:command failed: 0x4302fea13080.
2016-05-22T10:48:00.980Z cpu0:39425)nvme:nvmeCoreLogError:333:command failed: 0x4302fea13580.
2016-05-22T10:50:01.576Z cpu0:39571)nvme:nvmeCoreLogError:333:command failed: 0x4302fea13a80.
2016-05-22T10:52:02.080Z cpu0:35371)nvme:nvmeCoreLogError:333:command failed: 0x4302fea13f80.

Here are the changes that had to be made to the linux kernal for apples nvme ssd in T2 based macs

http://lists.infradead.org/pipermail/linux-nvme/2019-July/025565.html
http://lists.infradead.org/pipermail/linux-nvme/2019-July/025566.html
http://lists.infradead.org/pipermail/linux-nvme/2019-July/025567.html (edited)

constant blinking when there no volumes or access
not blinking in POST or BIOS setup
When Intel driver loaded its solid amber, when VMWARE its blinking

Invalid controller activation sequences causes slow boot

In nvme_ctrlr.c:NvmeCtrlr_HwStart(), the controller is disabled, enabled, disabled again and finally enabled for good.
The first enable appears to be for QEMU, as the following comment appears before the code:

/*
* Note: on the Qemu emulator, simply write NVME_CC_ENABLE (0x1) to
* (regs + NVME_CC) is not enough to bring controller to RDY state.
* IOSQES and IOCQES has to be set to bring the controller to RDY
* state for the initial reset.
*/

This first enable sequence is done before the admin queu attributes are setup. According to the NVMe spec, enabling the controller before the Admin queue attributes are set will produce undefined results. On the HGST SN100 series SSDs, the results are the the controller enable is ignored. The driver will detect this as a timeout as CSTS.RDY is never set. The SN100 series drives hae a CAP.TO (Timeout) value indicating a 128 seconds.

The driver will continue and the device will come up, but the 128 second timeout delays the host boot. In hosts with multiple SN100 SSDs, the boot is delayed by by 128 times the number of devices.

I believe that copying the admin queue setup code from the next controller enable sequence to the first enable sequence will fix this.

i.e. the following code should be copied and inserted right after the qemu comment:

/* Set admin queue depth of completion and submission */
aqa = (sqInfo->qsize - 1) << NVME_AQA_SQS_LSB;
aqa |= (qinfo->qsize - 1) << NVME_AQA_CQS_LSB;

/* Set admin queue attributes */
Nvme_Writel(aqa, (regs + NVME_AQA));
Nvme_Writeq(qinfo->compqPhy, (regs + NVME_ACQ));
Nvme_Writeq(sqInfo->subqPhy, (regs + NVME_ASQ));