Hardware implementation of an OmniXtend Memory Endpoint/Lowest Point of Coherence.
Report Bug
·
Request Feature
OmniXtend is a protocol to transmit TileLink messages over Ethernet. The aim of the protocol is to create large fully coherent systems.
This repository contains a fully synthesizeable version of an OmniXtend 1.0.3 compatible memory endpoint. The endpoint supports TL-UL, TL-UH and TL-C type commands. In addition, the endpoint supports a proposed OmniXtend 1.1 standard with additional features and quality-of-life changes.
Features:
- AXI memory controllers (e.g., DDR 4/5, HBM).
- Full Tilelink 1.8.0 feature set.
- Variable length requests.
- Multiple TileLink messages per Ethernet frame.
- Written in Bluespec.
- Compiles to Verilog, usable in most Hardware tool flows.
View the OmniXtend 1.0.3 Specification and the TileLink 1.8.0 Specification for more information.
This repository contains additional tools for simulation:
host_software/omnixtend-rs
: OmniXtend library written in Rust implementing a requester.host_software/omnixtend-tui
: TUI application to interact with OmniXtend endpoints.host_software/bitload
: Load data onto an OmniXtend endpoint over Ethernet.host_software/config
: Read status registers and configure the endpoint over PCIe (For TaPaSCo designs only).
Depending on your use case you need to install some tools. This section details the different methods.
- Bluespec Compiler: bsc
- BSVTools: Provides the build system used by this project.
- (Optional: For simulation) Rust: Rustup recommended for installation.
- (Optional: For Xilinx FPGA/IP-XACT): Vivado
- Clone this repository with submodules
git clone --recursive https://github.com/westerndigitalcorporation/OmnixtendEndpoint.git
- Setup BSVTools
cd OmnixtendEndpoint
${BSVTOOLS_PATH}/bsvAdd.py
There are three types of simulations provided. The first runs simple tests of the endpoint using Bluespec with a Rust stimulus.
The second method uses Rust to open a raw socket. This method allows interaction with the endpoint similarly to one in hardware over Ethernet.
Both methods require compiled simulation libraries
cd rust_sim
cargo build --release
make
This method requires root access to create the raw sockets and the virtual Ethernet devices used to connect the endpoints/requesters.
./utils/run_socket.sh
By default, creates five virtual Ethernet devices and attaches the endpoint to veth0
. Attach your user software to veth1-4
. The endpoint listens on the MAC address assigned to veth0
, by default 04:00:00:00:00:00
.
The environment variable linkcount
controls how many virtual Ethernet interfaces to create.
An example setup using host_software/omnixtend-tui
is provided in utils/run_tui_tmux.sh
. Before running this script, ensure that omnixtend-tui is compiled using:
pushd host_software/omnixtend-tui
cargo build --release
sudo ../../utils/set_raw_cap.sh target/release/omnixtend-tui
popd
ox.mp4
make SIM_TYPE=VERILOG compile_top
This compiles the endpoint to Verilog files and places them in build/verilog
and the top level is in mkOmnixtendEndpoint.v
. This Verilog relies on primitives distributed with the Bluespec compiler.
The default configuration expects an AXI attached memory for storage. For simulation purposes a version with an integrated BRAM can be used instead:
make SIM_TYPE=VERILOG BRAM_SIM=1 compile_top
The BRAM is initialized using the Verilog function $readmemh
from the file memoryconfig.hex
.
With Vivado installed:
make SIM_TYPE=VERILOG compile_top
The IP-XACT contains all dependencies (e.g., BSC primitives) and is located in build/ip
.
The easiest way to create a bitstream for Xilinx based FPGA is using TaPaSCo. tapasco_job.json
is an example TaPaSCo job file for the Alveo U280 platform. TaPaSCo supports many additional platforms like the NetFPGA SUME or Bittware XUP-VVH.
- Install TaPaSCo according to the readme. Either by manually compiling or using their generated distribution packages. Source the TaPaSCo initialization script and ensure that
tapasco
is in your path. - Import an IP-XACT core. This example uses an OmniXtend 1.0.3 configuration from the releases section.
tapasco import releases/OmnixtendEndpoint_RES_15_RESTO_21_ACKTO_12_OX11_0_MAC_0_CON_8_MAXFRAME_1500_MAXTLFRAME_1_BRAMSIM_0.zip as 412 --skipEvaluation -p AU280
- Start bitstream generation using the job file.
tapasco --configFile tapasco_job.json
Depending on the target platform and available resources, the bitstream generation might take a while.
The generated bitstream contains one OmnixtendEndpoint connected to the default memory of the selected platform. PCIe exposes configuration and status registers. The tool host_software/config
supports interacting with TaPaSCo generated bitstreams.
The endpoint supports several configuration options:
Name | Use |
---|---|
BRAM_SIM |
Use embedded BRAM instead of external AXI memory. |
OX_11_MODE |
Set to 1 to enable OmniXtend 1.1 features. |
RESEND_SIZE |
Resend buffer size per connection. 2**RESEND_SIZE flits. |
RESEND_TIMEOUT_CYCLES_LOG2 |
Force resend after 2**RESEND_TIMEOUT_CYCLES_LOG2 cycles without a valid packet. |
ACK_TIMEOUT_CYCLES_LOG2 |
Send ACK only packet after 2**ACK_TIMEOUT_CYCLES_LOG2 cycles. |
OMNIXTEND_CONNECTIONS |
Number of parallel connections. |
MAXIMUM_PACKET_SIZE |
Maximum number of bytes in Ethernet packet. Set to 1500 for default. |
MAXIMUM_TL_PER_FRAME |
Maximum number of TileLink messages per Ethernet packet generated by this IP. Between 1 and 64. |
MAC_ADDR |
Default MAC address. Set as hexadecimal without any prefix, e.g., 040000000000 . |
SYNTH_MODULES |
Split out Bluespec modules into separate Verilog modules. Default creates a single Verilog file. Useful for some downstream tools that have a hard time processing the flattened design. |
Contributions are what make the open source community such an amazing place to be, learn, inspire, and create. We greatly appreciated any contributions you make.
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
Distributed under the Apache 2.0 license. See LICENSE
for more information.
This Readme is based on Best-README-Template.