PCIM AXI W Channel hangs with wReady stuck at 0 about aws-fpga

[AWScsaralay]

Hi, Please note that the Shell-CL interface should be sync'ed to clk_main_a0 as stated in AWS_Shell_Interface_Specification.md#clocks.

Could you please confirm if your design is driving the PCIM interface based off clk_main_a0? If not, I recommend adding a clock domain crossing logic between your AXI4 driver running on 136MHz (Clock Group C, based on your metrics above) and the clk_main_a0 before feeding the PCIM interface.

Please let use know if you need any details or have questions.

Thanks!
Chakra

[tewujianke]

Thank you for the response! The design does have clock domain crossing from the clk_extra_c0 (CL clk) to clk_main_a0. xpm_fifo_axif is used to do this. We connect axi_in to our clk logic and axi_out to the shell, as shown in the below code snippet.

Another experiment I did was to remove the multi-clk domain design and drive both shell and CL with clk_main_a0; the same hanging behavior is observed at 136MHz, but it still worked fine with higher frequencies. We also noticed the design is doing b2b 64B read to the host (with max outstanding 32 req), and when it hangs, some read requests don't get responses. By inserting bubbles/gaps into the b2b read requests, the hanging would be gone. Any other advice would be much appreciated!

`module axi_asyncff (
input wclk,
input rclk,
axi_bus_t.master axi_in,
axi_bus_t.slave axi_out,
input rst_n
) ;

// xpm_fifo_axif: AXI Memory Mapped (AXI Full) FIFO
// Xilinx Parameterized Macro, version 2020.1

xpm_fifo_axif #(
.AXI_ADDR_WIDTH(64), // DECIMAL
.AXI_ARUSER_WIDTH(1), // DECIMAL
.AXI_AWUSER_WIDTH(1), // DECIMAL
.AXI_BUSER_WIDTH(1), // DECIMAL
.AXI_DATA_WIDTH(512), // DECIMAL
.AXI_ID_WIDTH(16), // DECIMAL
.AXI_LEN_WIDTH(8), // DECIMAL
.AXI_RUSER_WIDTH(1), // DECIMAL
.AXI_WUSER_WIDTH(1), // DECIMAL
.EN_RESET_SYNCHRONIZER(1),
.CDC_SYNC_STAGES(2), // DECIMAL
.CLOCKING_MODE("independent_clock"), // String
.ECC_MODE_RDCH("no_ecc"), // String
.ECC_MODE_WDCH("no_ecc"), // String
.FIFO_DEPTH_RACH(16), // DECIMAL
.FIFO_DEPTH_RDCH(16), // DECIMAL
.FIFO_DEPTH_WACH(16), // DECIMAL
.FIFO_DEPTH_WDCH(16), // DECIMAL
.FIFO_DEPTH_WRCH(16), // DECIMAL
.FIFO_MEMORY_TYPE_RACH("auto"), // String
.FIFO_MEMORY_TYPE_RDCH("auto"), // String
.FIFO_MEMORY_TYPE_WACH("auto"), // String
.FIFO_MEMORY_TYPE_WDCH("auto"), // String
.FIFO_MEMORY_TYPE_WRCH("auto"), // String
.PACKET_FIFO("false"), // String
.PROG_EMPTY_THRESH_RDCH(5), // DECIMAL
.PROG_EMPTY_THRESH_WDCH(5), // DECIMAL
.PROG_FULL_THRESH_RDCH(5), // DECIMAL
.PROG_FULL_THRESH_WDCH(5), // DECIMAL
.RD_DATA_COUNT_WIDTH_RDCH(5), // DECIMAL
.RD_DATA_COUNT_WIDTH_WDCH(5), // DECIMAL
.SIM_ASSERT_CHK(0), // DECIMAL; 0=disable simulation messages, 1=enable simulation messages
.USE_ADV_FEATURES_RDCH("0000"), // String
.USE_ADV_FEATURES_WDCH("0000"), // String
.WR_DATA_COUNT_WIDTH_RDCH(5), // DECIMAL
.WR_DATA_COUNT_WIDTH_WDCH(5) // DECIMAL
)
xpm_fifo_axif_inst (
.dbiterr_rdch(), // 1-bit output: Double Bit Error- Indicates that the ECC
// decoder detected a double-bit error and data in the FIFO core
// is corrupted.

.dbiterr_wdch(), // 1-bit output: Double Bit Error- Indicates that the ECC
// decoder detected a double-bit error and data in the FIFO core
// is corrupted.

.m_axi_araddr(axi_out.araddr), // AXI_ADDR_WIDTH-bit output: ARADDR: The read address bus gives
// the initial address of a read burst transaction. Only the
// start address of the burst is provided and the control
// signals that are issued alongside the address detail how the
// address is calculated for the remaining transfers in the
// burst.

.m_axi_arburst(), // 2-bit output: ARBURST: The burst type, coupled with the size
// information, details how the address for each transfer within
// the burst is calculated.

.m_axi_arcache(), // 2-bit output: ARCACHE: Indicates the bufferable, cacheable,
// write-through, write-back, and allocate attributes of the
// transaction.

.m_axi_arid(axi_out.arid), // AXI_ID_WIDTH-bit output: ARID: The data stream identifier
// that indicates different streams of data.

.m_axi_arlen(axi_out.arlen), // AXI_LEN_WIDTH-bit output: ARLEN: The burst length gives the
// exact number of transfers in a burst. This information
// determines the number of data transfers associated with the
// address.

.m_axi_arlock(), // 2-bit output: ARLOCK: This signal provides additional
// information about the atomic characteristics of the transfer.

.m_axi_arprot(), // 2-bit output: ARPROT: Indicates the normal, privileged, or
// secure protection level of the transaction and whether the
// transaction is a data access or an instruction access.

.m_axi_arqos(), // 2-bit output: ARQOS: Quality of Service (QoS) sent on the
// write address channel for each write transaction.

.m_axi_arregion(), // 2-bit output: ARREGION: Region Identifier sent on the write
// address channel for each write transaction.

.m_axi_arsize(axi_out.arsize), // 2-bit output: ARSIZE: Indicates the size of each transfer in
// the burst. Byte lane strobes indicate exactly which byte
// lanes to update.

.m_axi_aruser(), // AXI_ARUSER_WIDTH-bit output: ARUSER: The user-defined
// sideband information that can be transmitted alongside the
// data stream.

.m_axi_arvalid(axi_out.arvalid), // 1-bit output: ARVALID: Indicates that the master is driving a
// valid transfer. A transfer takes place when both ARVALID and
// ARREADY are asserted

.m_axi_awaddr(axi_out.awaddr), // AXI_ADDR_WIDTH-bit output: AWADDR: The write address bus
// gives the address of the first transfer in a write burst
// transaction. The associated control signals are used to
// determine the addresses of the remaining transfers in the
// burst.

.m_axi_awburst(), // 2-bit output: AWSIZE: The burst type, coupled with the size
// information, details how the address for each transfer within
// the burst is calculated.

.m_axi_awcache(), // 2-bit output: AWCACHE: Indicates the bufferable, cacheable,
// write-through, write-back, and allocate attributes of the
// transaction.

.m_axi_awid(axi_out.awid), // AXI_ID_WIDTH-bit output: AWID: Identification tag for the
// write address group of signals.

.m_axi_awlen(axi_out.awlen), // AXI_LEN_WIDTH-bit output: AWLEN: The burst length gives the
// exact number of transfers in a burst. This information
// determines the number of data transfers associated with the
// address.

.m_axi_awlock(), // 2-bit output: AWLOCK: This signal provides additional
// information about the atomic characteristics of the transfer.

.m_axi_awprot(), // 2-bit output: AWPROT: Indicates the normal, privileged, or
// secure protection level of the transaction and whether the
// transaction is a data access or an instruction access.

.m_axi_awqos(), // 2-bit output: AWQOS: Quality of Service (QoS) sent on the
// write address channel for each write transaction.

.m_axi_awregion(), // 2-bit output: AWREGION: Region Identifier sent on the write
// address channel for each write transaction.

.m_axi_awsize(axi_out.awsize), // 2-bit output: AWSIZE: Indicates the size of each transfer in
// the burst. Byte lane strobes indicate exactly which byte
// lanes to update.

.m_axi_awuser(), // AXI_AWUSER_WIDTH-bit output: AWUSER: The user-defined
// sideband information that can be transmitted alongside the
// data stream.

.m_axi_awvalid(axi_out.awvalid), // 1-bit output: AWVALID: Indicates that the master is driving a
// valid transfer. A transfer takes place when both AWVALID and
// AWREADY are asserted

.m_axi_bready(axi_out.bready), // 1-bit output: BREADY: Indicates that the master can accept a
// transfer in the current cycle.

.m_axi_rready(axi_out.rready), // 1-bit output: RREADY: Indicates that the master can accept a
// transfer in the current cycle.

.m_axi_wdata(axi_out.wdata), // AXI_DATA_WIDTH-bit output: WDATA: The primary payload that is
// used to provide the data that is passing across the
// interface. The width of the data payload is an integer number
// of bytes.

.m_axi_wlast(axi_out.wlast), // 1-bit output: WLAST: Indicates the boundary of a packet.
.m_axi_wstrb(axi_out.wstrb), // AXI_DATA_WIDTH/8-bit output: WSTRB: The byte qualifier that
// indicates whether the content of the associated byte of TDATA
// is processed as a data byte or a position byte. For a 64-bit
// DATA, bit 0 corresponds to the least significant byte on
// DATA, and bit 0 corresponds to the least significant byte on
// DATA, and bit 7 corresponds to the most significant byte. For
// example: STROBE[0] = 1b, DATA[7:0] is valid STROBE[7] = 0b,
// DATA[63:56] is not valid

.m_axi_wuser(), // AXI_WUSER_WIDTH-bit output: WUSER: The user-defined sideband
// information that can be transmitted alongside the data
// stream.

.m_axi_wvalid(axi_out.wvalid), // 1-bit output: WVALID: Indicates that the master is driving a
// valid transfer. A transfer takes place when both WVALID and
// WREADY are asserted

.prog_empty_rdch(), // 1-bit output: Programmable Empty- This signal is asserted
// when the number of words in the Read Data Channel FIFO is
// less than or equal to the programmable empty threshold value.
// It is de-asserted when the number of words in the Read Data
// Channel FIFO exceeds the programmable empty threshold value.

.prog_empty_wdch(), // 1-bit output: Programmable Empty- This signal is asserted
// when the number of words in the Write Data Channel FIFO is
// less than or equal to the programmable empty threshold value.
// It is de-asserted when the number of words in the Write Data
// Channel FIFO exceeds the programmable empty threshold value.

.prog_full_rdch(), // 1-bit output: Programmable Full: This signal is asserted when
// the number of words in the Read Data Channel FIFO is greater
// than or equal to the programmable full threshold value. It is
// de-asserted when the number of words in the Read Data Channel
// FIFO is less than the programmable full threshold value.

.prog_full_wdch(), // 1-bit output: Programmable Full: This signal is asserted when
// the number of words in the Write Data Channel FIFO is greater
// than or equal to the programmable full threshold value. It is
// de-asserted when the number of words in the Write Data
// Channel FIFO is less than the programmable full threshold
// value.

.rd_data_count_rdch(), // RD_DATA_COUNT_WIDTH_RDCH-bit output: Read Data Count- This
// bus indicates the number of words available for reading in
// the Read Data Channel FIFO.

.rd_data_count_wdch(), // RD_DATA_COUNT_WIDTH_WDCH-bit output: Read Data Count- This
// bus indicates the number of words available for reading in
// the Write Data Channel FIFO.

.s_axi_arready(axi_in.arready), // 1-bit output: ARREADY: Indicates that the slave can accept a
// transfer in the current cycle.

.s_axi_awready(axi_in.awready), // 1-bit output: AWREADY: Indicates that the slave can accept a
// transfer in the current cycle.

.s_axi_bid(axi_in.bid), // AXI_ID_WIDTH-bit output: BID: The data stream identifier that
// indicates different streams of data.

.s_axi_bresp(axi_in.bresp), // 2-bit output: BRESP: Indicates the status of the write
// transaction. The allowable responses are OKAY, EXOKAY,
// SLVERR, and DECERR.

.s_axi_buser(), // AXI_BUSER_WIDTH-bit output: BUSER: The user-defined sideband
// information that can be transmitted alongside the data
// stream.

.s_axi_bvalid(axi_in.bvalid), // 1-bit output: BVALID: Indicates that the master is driving a
// valid transfer. A transfer takes place when both BVALID and
// BREADY are asserted

.s_axi_rdata(axi_in.rdata), // AXI_DATA_WIDTH-bit output: RDATA: The primary payload that is
// used to provide the data that is passing across the
// interface. The width of the data payload is an integer number
// of bytes.

.s_axi_rid(axi_in.rid), // AXI_ID_WIDTH-bit output: RID: The data stream identifier that
// indicates different streams of data.

.s_axi_rlast(axi_in.rlast), // 1-bit output: RLAST: Indicates the boundary of a packet.
.s_axi_rresp(axi_in.rresp), // 2-bit output: RRESP: Indicates the status of the read
// transfer. The allowable responses are OKAY, EXOKAY, SLVERR,
// and DECERR.

.s_axi_ruser(), // AXI_RUSER_WIDTH-bit output: RUSER: The user-defined sideband
// information that can be transmitted alongside the data
// stream.

.s_axi_rvalid(axi_in.rvalid), // 1-bit output: RVALID: Indicates that the master is driving a
// valid transfer. A transfer takes place when both RVALID and
// RREADY are asserted

.s_axi_wready(axi_in.wready), // 1-bit output: WREADY: Indicates that the slave can accept a
// transfer in the current cycle.

.sbiterr_rdch(), // 1-bit output: Single Bit Error- Indicates that the ECC
// decoder detected and fixed a single-bit error.

.sbiterr_wdch(), // 1-bit output: Single Bit Error- Indicates that the ECC
// decoder detected and fixed a single-bit error.

.wr_data_count_rdch(), // WR_DATA_COUNT_WIDTH_RDCH-bit output: Write Data Count: This
// bus indicates the number of words written into the Read Data
// Channel FIFO.

.wr_data_count_wdch(), // WR_DATA_COUNT_WIDTH_WDCH-bit output: Write Data Count: This
// bus indicates the number of words written into the Write Data
// Channel FIFO.

.injectdbiterr_rdch(1'b0), // 1-bit input: Double Bit Error Injection- Injects a double bit
// error if the ECC feature is used.

.injectdbiterr_wdch(1'b0), // 1-bit input: Double Bit Error Injection- Injects a double bit
// error if the ECC feature is used.

.injectsbiterr_rdch(1'b0), // 1-bit input: Single Bit Error Injection- Injects a single bit
// error if the ECC feature is used.

.injectsbiterr_wdch(1'b0), // 1-bit input: Single Bit Error Injection- Injects a single bit
// error if the ECC feature is used.

.m_aclk(rclk), // 1-bit input: Master Interface Clock: All signals on master
// interface are sampled on the rising edge of this clock.

.m_axi_arready(axi_out.arready), // 1-bit input: ARREADY: Indicates that the master can accept a
// transfer in the current cycle.

.m_axi_awready(axi_out.awready), // 1-bit input: AWREADY: Indicates that the master can accept a
// transfer in the current cycle.

.m_axi_bid(axi_out.bid), // AXI_ID_WIDTH-bit input: BID: The data stream identifier that
// indicates different streams of data.

.m_axi_bresp(axi_out.bresp), // 2-bit input: BRESP: Indicates the status of the write
// transaction. The allowable responses are OKAY, EXOKAY,
// SLVERR, and DECERR.

.m_axi_buser(1'b0), // AXI_BUSER_WIDTH-bit input: BUSER: The user-defined sideband
// information that can be transmitted alongside the data
// stream.

.m_axi_bvalid(axi_out.bvalid), // 1-bit input: BVALID: Indicates that the master is driving a
// valid transfer. A transfer takes place when both BVALID and
// BREADY are asserted

.m_axi_rdata(axi_out.rdata), // AXI_DATA_WIDTH-bit input: RDATA: The primary payload that is
// used to provide the data that is passing across the
// interface. The width of the data payload is an integer number
// of bytes.

.m_axi_rid(axi_out.rid), // AXI_ID_WIDTH-bit input: RID: The data stream identifier that
// indicates different streams of data.

.m_axi_rlast(axi_out.rlast), // 1-bit input: RLAST: Indicates the boundary of a packet.
.m_axi_rresp(axi_out.rresp), // 2-bit input: RRESP: Indicates the status of the read
// transfer. The allowable responses are OKAY, EXOKAY, SLVERR,
// and DECERR.

.m_axi_ruser(1'b0), // AXI_RUSER_WIDTH-bit input: RUSER: The user-defined sideband
// information that can be transmitted alongside the data
// stream.

.m_axi_rvalid(axi_out.rvalid), // 1-bit input: RVALID: Indicates that the master is driving a
// valid transfer. A transfer takes place when both RVALID and
// RREADY are asserted

.m_axi_wready(axi_out.wready), // 1-bit input: WREADY: Indicates that the master can accept a
// transfer in the current cycle.

.s_aclk(wclk), // 1-bit input: Slave Interface Clock: All signals on slave
// interface are sampled on the rising edge of this clock.

.s_aresetn(rst_n), // 1-bit input: Active low asynchronous reset.
.s_axi_araddr(axi_in.araddr), // AXI_ADDR_WIDTH-bit input: ARADDR: The read address bus gives
// the initial address of a read burst transaction. Only the
// start address of the burst is provided and the control
// signals that are issued alongside the address detail how the
// address is calculated for the remaining transfers in the
// burst.

.s_axi_arburst(2'b0), // 2-bit input: ARBURST: The burst type, coupled with the size
// information, details how the address for each transfer within
// the burst is calculated.

.s_axi_arcache(4'b0), // 2-bit input: ARCACHE: Indicates the bufferable, cacheable,
// write-through, write-back, and allocate attributes of the
// transaction.

.s_axi_arid(axi_in.arid), // AXI_ID_WIDTH-bit input: ARID: The data stream identifier that
// indicates different streams of data.

.s_axi_arlen(axi_in.arlen), // AXI_LEN_WIDTH-bit input: ARLEN: The burst length gives the
// exact number of transfers in a burst. This information
// determines the number of data transfers associated with the
// address.

.s_axi_arlock(2'b0), // 2-bit input: ARLOCK: This signal provides additional
// information about the atomic characteristics of the transfer.

.s_axi_arprot(3'b0), // 2-bit input: ARPROT: Indicates the normal, privileged, or
// secure protection level of the transaction and whether the
// transaction is a data access or an instruction access.

.s_axi_arqos(4'b0), // 2-bit input: ARQOS: Quality of Service (QoS) sent on the
// write address channel for each write transaction.

.s_axi_arregion(4'b0), // 2-bit input: ARREGION: Region Identifier sent on the write
// address channel for each write transaction.

.s_axi_arsize(axi_in.arsize), // 2-bit input: ARSIZE: Indicates the size of each transfer in
// the burst. Byte lane strobes indicate exactly which byte
// lanes to update.

.s_axi_aruser(1'b0), // AXI_ARUSER_WIDTH-bit input: ARUSER: The user-defined sideband
// information that can be transmitted alongside the data
// stream.

.s_axi_arvalid(axi_in.arvalid), // 1-bit input: ARVALID: Indicates that the master is driving a
// valid transfer. A transfer takes place when both ARVALID and
// ARREADY are asserted

.s_axi_awaddr(axi_in.awaddr), // AXI_ADDR_WIDTH-bit input: AWADDR: The write address bus gives
// the address of the first transfer in a write burst
// transaction. The associated control signals are used to
// determine the addresses of the remaining transfers in the
// burst.

.s_axi_awburst(2'b0), // 2-bit input: AWBURST: The burst type, coupled with the size
// information, details how the address for each transfer within
// the burst is calculated.

.s_axi_awcache(4'b0), // 2-bit input: AWCACHE: Indicates the bufferable, cacheable,
// write-through, write-back, and allocate attributes of the
// transaction.

.s_axi_awid(axi_in.awid), // AXI_ID_WIDTH-bit input: AWID: Identification tag for the
// write address group of signals.

.s_axi_awlen(axi_in.awlen), // AXI_LEN_WIDTH-bit input: AWLEN: The burst length gives the
// exact number of transfers in a burst. This information
// determines the number of data transfers associated with the
// address.

.s_axi_awlock(2'b0), // 2-bit input: AWLOCK: This signal provides additional
// information about the atomic characteristics of the transfer.

.s_axi_awprot(3'd0), // 2-bit input: AWPROT: Indicates the normal, privileged, or
// secure protection level of the transaction and whether the
// transaction is a data access or an instruction access.

.s_axi_awqos(4'd0), // 2-bit input: AWQOS: Quality of Service (QoS) sent on the
// write address channel for each write transaction.

.s_axi_awregion(4'd0), // 2-bit input: AWREGION: Region Identifier sent on the write
// address channel for each write transaction.

.s_axi_awsize(axi_in.awsize), // 2-bit input: AWSIZE: Indicates the size of each transfer in
// the burst. Byte lane strobes indicate exactly which byte
// lanes to update.

.s_axi_awuser(1'b0), // AXI_AWUSER_WIDTH-bit input: AWUSER: The user-defined sideband
// information that can be transmitted alongside the data
// stream.

.s_axi_awvalid(axi_in.awvalid), // 1-bit input: AWVALID: Indicates that the master is driving a
// valid transfer. A transfer takes place when both AWVALID and
// AWREADY are asserted

.s_axi_bready(axi_in.bready), // 1-bit input: BREADY: Indicates that the slave can accept a
// transfer in the current cycle.

.s_axi_rready(axi_in.rready), // 1-bit input: RREADY: Indicates that the slave can accept a
// transfer in the current cycle.

.s_axi_wdata(axi_in.wdata), // AXI_DATA_WIDTH-bit input: WDATA: The primary payload that is
// used to provide the data that is passing across the
// interface. The width of the data payload is an integer number
// of bytes.

.s_axi_wlast(axi_in.wlast), // 1-bit input: WLAST: Indicates the boundary of a packet.
.s_axi_wstrb(axi_in.wstrb), // AXI_DATA_WIDTH/8-bit input: WSTRB: The byte qualifier that
// indicates whether the content of the associated byte of TDATA
// is processed as a data byte or a position byte. For a 64-bit
// DATA, bit 0 corresponds to the least significant byte on
// DATA, and bit 0 corresponds to the least significant byte on
// DATA, and bit 7 corresponds to the most significant byte. For
// example: STROBE[0] = 1b, DATA[7:0] is valid STROBE[7] = 0b,
// DATA[63:56] is not valid

.s_axi_wuser(1'b0), // AXI_WUSER_WIDTH-bit input: WUSER: The user-defined sideband
// information that can be transmitted alongside the data
// stream.

.s_axi_wvalid(axi_in.wvalid) // 1-bit input: WVALID: Indicates that the master is driving a
// valid transfer. A transfer takes place when both WVALID and
// WREADY are asserted

);

// End of xpm_fifo_axif_inst instantiation
endmodule
`