seldridge / rocket-rocc-examples

Tests for example Rocket Custom Coprocessors

Collection of example libraries and test programs for the existing Rocket Custom Coprocessor (RoCC) accelerators for Rocket Chip.

Usage

Install the RISC-V toolchain and make sure that it's on your path. You need to pass the path to riscv-tools in as an argument to configure --with-riscvtools=$PATH_TO_RISCV_TOOLS.

autoconf
mkdir build && cd build
../configure --with-riscvtools=$PATH_TO_RISCV_TOOLS
make

This creates two classes of tests:

• bareMetal -- Bare metal tests like RISCV Tests
• pk -- Tests that use the Proxy Kernel

Bare Metal tests can be run directly on the emulator (for instructions on how to build this see the following section), e.g.:

emulator-freechips.rocketchip.system-RoccExampleConfig bareMetal/examples-bareMetal-p-accumulator

Proxy Kernel tests are ELFs and need the Proxy Kernel (or Linux). You must first patch the Proxy Kernel so that the XS bits allow access to the "extension" (the RoCC). You can change this manually or use the provided patch (patches/riscv-pk.patch):

cd $RISCV_PK_DIR
git apply $THIS_REPO_DIR/patches/riscv-pk.patch

mkdir build
cd build
../configure --prefix=$RISCV/riscv64-unknown-elf --host=riscv64-unknown-elf
make
make install

Following that, you can run Proxy Kernel tests, e.g., :

emulator-freechips.rocketchip.system-RoccExampleConfig pk pk/examples-pk-accumulator

Building a Rocket Chip Emulator

Build a rocket-chip emulator with the RoCC examples baked in and run the provided test program:

cd $ROCKETCHIP_DIR/emulator
make CONFIG=RoccExampleConfig

Expected Run Time

The included test should run for ~5 million cycles over a wall clock time of ~5 minutes.

> time $ROCKETCHIP_DIR/emulator-freechips.rocketchip.system-RoccExampleConfig -c pk pk/examples-pk-accumulator
[INFO] Write R[1] = 0xdead
[INFO] Read R[1]
[INFO]   Received 0xdead (expected 0xdead)
[INFO] Accum R[1] with 0xffffffffffffe042
[INFO] Read R[1]
[INFO]   Received 0xbeef (expected 0xbeef)
[INFO] Load 0xbad (virt: 0x0xfee9ac0, phys: 0x0x8ffffac0) via L1 virtual address
[INFO] Read R[1]
[INFO]   Received 0xbad (expected 0xbad)
Completed after 3278954 cycles

real	5m2.738s
user	5m0.262s
sys	0m1.179s

rocket-rocc-examples's Issues

About Load operation in this example

According to this example, we give the address of data[2] and load it into regfile(1), but I'm not sure how to define the length to load? I mean since we have the address of data[2] now, can we just like memcpy function in C , just copy data[2] to regfile(1) and data[3] to regfile(2) ?

csrrs instruction illegal with proxy kernel?

Hi,
I have a simple test for RoCC interrupt. The baremetal test works as expected. When I run it with proxy kernel, it error out with illegal instruction. I found out the illegal instruction is the csrrs instruction.
I am running with the patched proxy kernel.
Any idea to fix this? Thanks.

Lei

Building configuration failed

Hi, I've built toolchain for march rv32imac and mabi ilp32. My emulator is also 32 bit. Is there anyway to build configuration for 32 bit. I'm getting following error...

make[1]: riscv64-unknown-elf-gcc: Command not found
/home/sanket/rocc-eg/rocket-rocc-examples/build/../bareMetal/Makefile:24: recipe for target 'examples-bareMetal-p-accumulator' failed
make[1]: *** [examples-bareMetal-p-accumulator] Error 127

Please assist..
Thanks & Regards,

Why RoCC accelerator requires physical address when accessing L1D cache?

I was hacking the source code of Rocket SoC, and found that Rocket uses virtual address when having access to L1D cache(from the source code of nonblocking cache code).

But, from the following code:

As far as I know, the interface between L1D cache and Accelerator(co-processor) is same as the interface between core and L1D cache.

L1D cache accepts virtual address because it is virtually indexed and physically tagged. So, accelerator also has to use virtual address.

If L1D cache accepts physical address, the indexing of TLB will use physical address, which is non-sesnse.

The core uses virtual address in the 'store' and the 'load' instruction. But, why the virtual address needs to be translated into physical address before the accelerator accesses L1D cache?

test won't terminate

I did everything the Readme said, but when I ran the testaccumulator, it never terminated. I gave it 3 hours. Is this something anybody has come across and knows how to fix this? Is there any chance I would've had to be more patient?

In case anybody is wondering, I did not forget the pk, I ran:
./emulator-rocketchip-RoccExampleConfig pk ../../../rocket-rocc-examples/build/test-accumulator

Testing accumulator.c using a Linux kernel instead of pk

Hello,
First I would like to say thank you for this repo. I'm new to rocket chip, but this repo has helped me a lot with setting up custom RoCC extensions and testing them. My goal in the future is to eventually be able to run C files on the chip using libraries not supported by pk. What would the steps be to run the accumulator.c test file (to test my RoCC extension) on the rocket chip emulator without using pk, but using a Linux kernel and compiler instead? Is this possible using the emulator or would I have to use spike?
Thanks!

problem to build riscv-pk after patch

Thank you for the very useful repo. I'm new in RISC-V and I want to use rocc interface to evaluate my research. basically, I make a prototype for decimal multiplication that needs to evaluate through rocket accelerator. after study found your repo is an example using rocc interface. I follow your instruction and upto Bare Metal tests run directly on the emulator its okay if I execute the command

`This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 43783
*** PASSED *** Completed after 105192 cycles

real 0m8.390s
user 0m8.288s
sys 0m0.096s`

but after patch, if I want to build riscv-pk it shows the following error :

riscv64-unknown-elf-gcc -MMD -MP -Wall -Werror -D__NO_INLINE__ -mcmodel=medany -O2 -std=gnu99 -Wno-unused -Wno-attributes -fno-delete-null-pointer-checks -DBBL_PAYLOAD=\"bbl_payload\" -DBBL_LOGO_FILE=\"bbl_logo_file\" -I. -I../pk -I../bbl -I../softfloat -I../dummy_payload -I../machine -I../util -c ../machine/minit.c riscv64-unknown-elf-ar rcv libmachine.a fdt.o mtrap.o minit.o htif.o emulation.o muldiv_emulation.o fp_emulation.o fp_ldst.o uart.o finisher.o misaligned_ldst.o mentry.o fp_asm.o r - fdt.o r - mtrap.o r - minit.o r - htif.o r - emulation.o r - muldiv_emulation.o r - fp_emulation.o r - fp_ldst.o r - uart.o r - finisher.o r - misaligned_ldst.o r - mentry.o r - fp_asm.o riscv64-unknown-elf-ranlib libmachine.a riscv64-unknown-elf-gcc -nostartfiles -nostdlib -static -o pk pk.o -L. -lpk -lmachine -lsoftfloat -lutil -lgcc -T ../pk/pk.lds /path/to/install/riscv/toolchain/lib/gcc/riscv64-unknown-elf/7.2.0/../../../../riscv64-unknown-elf/bin/ld: ./libmachine.a(minit.o): can't link hard-float modules with soft-float modules /path/to/install/riscv/toolchain/lib/gcc/riscv64-unknown-elf/7.2.0/../../../../riscv64-unknown-elf/bin/ld: failed to merge target specific data of file ./libmachine.a(minit.o) collect2: error: ld returned 1 exit status Makefile:302: recipe for target 'pk' failed make: *** [pk] Error 1
please help me to complete the porcess Thanks

Illegal instruction error for accumulator example

cd build
make
mkdir -p bareMetal
make -C bareMetal -f /home/abdullah/Workspace/Chisel-practice/rocket-chip/rocket-rocc-examples/build/../bareMetal/Makefile abs_top_srcdir=/home/abdullah/Workspace/Chisel-practice/rocket-chip/rocket-rocc-examples/build/.. XLEN=64 PREFIX=examples-bareMetal src_dir=/home/abdullah/Workspace/Chisel-practice/rocket-chip/rocket-rocc-examples/build/../bareMetal RISCVTOOLS=/home/abdullah/Workspace/Chisel-practice/rocket-chip/riscv-tools
make[1]: Entering directory '/home/abdullah/Workspace/Chisel-practice/rocket-chip/rocket-rocc-examples/build/bareMetal'
make[1]: Nothing to be done for 'default'.
make[1]: Leaving directory '/home/abdullah/Workspace/Chisel-practice/rocket-chip/rocket-rocc-examples/build/bareMetal'
mkdir -p pk
make -C pk -f /home/abdullah/Workspace/Chisel-practice/rocket-chip/rocket-rocc-examples/build/../pk/Makefile abs_top_srcdir=/home/abdullah/Workspace/Chisel-practice/rocket-chip/rocket-rocc-examples/build/.. XLEN=64 PREFIX=examples-pk src_dir=/home/abdullah/Workspace/Chisel-practice/rocket-chip/rocket-rocc-examples/build/../pk RISCVTOOLS=/home/abdullah/Workspace/Chisel-practice/rocket-chip/riscv-tools
make[1]: Entering directory '/home/abdullah/Workspace/Chisel-practice/rocket-chip/rocket-rocc-examples/build/pk'
make[1]: Nothing to be done for 'default'.
make[1]: Leaving directory '/home/abdullah/Workspace/Chisel-practice/rocket-chip/rocket-rocc-examples/build/pk'

spike pk examples-pk-accumulator
[INFO] Write R[1] = 0xdead
z 0000000000000000 ra 00000000000101e8 sp 000000007f7e9af0 gp 000000000001c910
tp 0000000000000000 t0 0000000000010560 t1 0000000000040000 t2 0000000000000000
s0 000000007f7e9b40 s1 0000000000000000 a0 000000000000001b a1 000000000000dead
a2 0000000000000001 a3 0000000000000000 a4 0000000000000000 a5 0000000000000001
a6 000000000000000f a7 0000000000000040 s2 0000000000000000 s3 0000000000000000
s4 0000000000000000 s5 0000000000000000 s6 0000000000000000 s7 0000000000000000
s8 0000000000000000 s9 0000000000000000 sA 0000000000000000 sB 0000000000000000
t3 000000000000000a t4 0000000000000000 t5 0000000000000000 t6 0000000000000000
pc 00000000000101f4 va 0000000000c5f50b insn 00c5f50b sr 8000000200046020
An illegal instruction was executed!

Hi, thanks for providing this git. I was trying to run the example I got this error. is it fault due to
ROCC_INSTRUCTION(XCUSTOM_ACC, y, data, rocc_rd, k_DO_WRITE);

Add examples for all available Rocket RoCCs

• Translator
• Character Count

Assertion `buf != MAP_FAILED' failed.

After building a rocket-chip emulator with the RoCC examples baked in,i can't run the test"time emulator-freechips.rocketchip.system-RoccExampleConfig -c pk ./build/test-accumulator" and "emulator-freechips.rocketchip.system-RoccExampleConfig pk pk/examples-pk-accumulator"

the error show:

This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 37782
emulator-freechips.rocketchip.system-RoccExampleConfig: ../fesvr/elfloader.cc:27: std::map<std::__cxx11::basic_string<char>, long unsigned int> load_elf(const char*, memif_t*, reg_t*): Assertion `buf != MAP_FAILED' failed.

real	0m0.983s
user	0m0.928s
sys	0m0.048s

but i can run "emulator-freechips.rocketchip.system-RoccExampleConfig bareMetal/examples-bareMetal-p-accumulator" and "emulator-freechips.rocketchip.system-RoccExampleConfig pk hello".The first show nothing,and the second show the right output

xcustom.h compile error

First I would like to thank you for your great example to interact with ROCC, it has been really helpful!!!

So I have been using your accumulator example, but when I compile the code, I got an error saying there is "lvalue required as left operand of assignment" in line (https://github.com/IBM/rocc-software/blob/022e8a51d6ea91e1dfa82c78478904a3d9986b9c/src/xcustom.h#L56).

When I modified it using "==" everything works fine, but I didn't get a return value from accelerator, could you tell me what would be my source of error here?

Thank you very much!

Regards,
Dong

Do_load instruction for 32 bit.

Hi,
I've implemented 32 bit rocket chip with rocc example, but in accumulator example while accessing data through hella cache interface using do_load instruction. The io_mem_response_valid signal remains high for two clock cycle so data in reg file is overwritten by data of next memory location.

May be memory response interface default setting to transfer 64 byte or else. please assist me. how to change burst size?

Thanks & Regards,
Sanket

after pk patch, It seems program hang forever

[1]+ Stopped ./emulator-rocketchip-RoccExampleConfig ./pk ./rocket-rocc-examples/build/test-accumulator

real 11m56.553s
user 0m0.000s
sys 0m0.000s

An illegal instruction was executed!

I did apply the patch and rebuild pk (configure, make and make install).

I have a problem when running the Proxy Kernel test with the emulator:

$ ./emulator-freechips.rocketchip.system-RoccExampleConfig pk ../rocket-rocc-examples/build/pk/examples-pk-accumulator
This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 44165
[INFO] Write R[1] = 0xdead
z 0000000000000000 ra 00000000000101e8 sp 000000000fee9ad0 gp 000000000001d6a8
tp 0000000000000000 t0 0000000000010560 t1 0000000000040000 t2 0000000000000000
s0 000000000fee9b20 s1 0000000000000000 a0 000000000000001b a1 000000000000dead
a2 0000000000000001 a3 0000000000000000 a4 0000000000000000 a5 0000000000000001
a6 000000000000001f a7 0000000000000040 s2 0000000000000000 s3 0000000000000000
s4 0000000000000000 s5 0000000000000000 s6 0000000000000000 s7 0000000000000000
s8 0000000000000000 s9 0000000000000000 sA 0000000000000000 sB 0000000000000000
t3 000000000000000a t4 0000000000000000 t5 0000000000000000 t6 0000000000000000
pc 00000000000101f4 va 0000000000c5f50b insn 00c5f50b sr 8000000200046020
An illegal instruction was executed!
*** FAILED *** via dtm (code = -1, seed 1531194881) after 3611144 cycles

However, the baremetal test works fine:

./emulator-freechips.rocketchip.system-RoccExampleConfig ../rocket-rocc-examples/build/bareMetal/examples-bareMetal-p-accumulator
This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 45235

Using spike, I got this:

$ spike ../rocket-rocc-examples/build/bareMetal/examples-bareMetal-p-accumulator
*** FAILED *** (tohost = 668)

$ spike pk ../rocket-rocc-examples/build/pk/examples-pk-accumulator
[INFO] Write R[1] = 0xdead
z 0000000000000000 ra 00000000000101e8 sp 000000007f7e9ad0 gp 000000000001d6a8
tp 0000000000000000 t0 0000000000010560 t1 0000000000040000 t2 0000000000000000
s0 000000007f7e9b20 s1 0000000000000000 a0 000000000000001b a1 000000000000dead
a2 0000000000000001 a3 0000000000000000 a4 0000000000000000 a5 0000000000000001
a6 000000000000001f a7 0000000000000040 s2 0000000000000000 s3 0000000000000000
s4 0000000000000000 s5 0000000000000000 s6 0000000000000000 s7 0000000000000000
s8 0000000000000000 s9 0000000000000000 sA 0000000000000000 sB 0000000000000000
t3 000000000000000a t4 0000000000000000 t5 0000000000000000 t6 0000000000000000
pc 00000000000101f4 va 0000000000c5f50b insn 00c5f50b sr 8000000200046020
An illegal instruction was executed!

Any ideas?

Best regards,
Xuhao

"Illegal Instruction Executed" when running the pk test

Hello,

I apologize for all the comments I have been posting. I have been stuck on this issue for a while and can't seem to get my head wrapped around it. Whenever I try to run the accumulator.c file using the proxy kernel, I get the illegal instruction error.

What I have done:
Basically I have just gone through this repository's setup instructions including the proxy kernel patching. I didn't run into any problems there. I went into the pk directory and ran the makefile. I then tried running the accumulator test using the command below:

~/rocket-chip/emulator/emulator-freechips.rocketchip.system-RoccExampleConfig -c pk ./-accumulator

What I get:
The output of running this command is shown below:

[INFO] Write R[1] = 0xdead
z 0000000000000000 ra 00000000000101e8 sp 000000000fee9b00 gp 000000000001c910
tp 0000000000000000 t0 0000000000010560 t1 0000000000040000 t2 0000000000000000
s0 000000000fee9b50 s1 0000000000000000 a0 000000000000001b a1 000000000000dead
a2 0000000000000001 a3 0000000000000000 a4 0000000000000000 a5 0000000000000001
a6 000000000000000f a7 0000000000000040 s2 0000000000000000 s3 0000000000000000
s4 0000000000000000 s5 0000000000000000 s6 0000000000000000 s7 0000000000000000
s8 0000000000000000 s9 0000000000000000 sA 0000000000000000 sB 0000000000000000
t3 000000000000000a t4 0000000000000000 t5 0000000000000000 t6 0000000000000000
pc 00000000000101f4 va 0000000000c5f50b insn 00c5f50b sr 8000000200046020
An illegal instruction was executed!
*** FAILED *** (code = -1, seed 1514843766) after 3949206 cycles

I am a noob when it comes to this stuff so I could be making a mistake. If I am, I apologize for wasting your time. I have just been stuck on this for quite a while and am at a loss.

Thanks and happy holidays!

Complie custom instructions error

I want to use custom instructions,macro to pass rs1_ and rs2_ as immediates. When I complie them,it generate errors.

Connecting RoCC to system

Hi @seldridge,

This might be off topic (let me know if this is, I will close this issue then), but I am stuck in connecting RoCC to system level. I've got one accelerator (accumulator example) instantiated as per instruction available here: (https://github.com/freechipsproject/rocket-chip/blob/7e75d63ba6b4c1b50aaaf920e1c693ef6acf51d7/src/main/scala/tile/LazyRoCC.scala).

If I understand it correctly, this instantiates the accelerator inside rocket tile, which is wrapped by rocket coreplex, which in turn wrapped by system.

I am just wondering if there is any way to connect the RoCC interface to system level? I am assuming we need some kind of traits like this (https://github.com/freechipsproject/rocket-chip/blob/7e75d63ba6b4c1b50aaaf920e1c693ef6acf51d7/src/main/scala/system/ExampleRocketSystem.scala#L13)? Thanks!

Best,
Dong

There is no test-accumulator

I did everything the Readme said,when i did the last step"time $ROCKETCHIP_DIR/emulator-freechips.rocketchip.system-RoccExampleConfig -c pk ./build/test-accumulator",there is no file"test-accumulator"at all.

so after run the command,it show:

~/桌面/project/rocket-rocc-examples$ time emulator-freechips.rocketchip.system-RoccExampleConfig -c $RISCV/riscv64-unknown-elf/bin/pk ./build/test-accumulator

This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 33726
couldn't open ELF program: ./build/!
*** FAILED *** via dtm (code = -1, seed 1522044587) after 1469412 cycles

real	3m14.550s
user	3m13.520s
sys	0m0.484s

I did the pk test "emulator-freechips.rocketchip.system-RoccExampleConfig pk pk/examples-pk-accumulator",it works well:

This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 35611
[INFO] Write R[1] = 0xdead
[INFO] Read R[1]
[INFO]   Received 0xdead (expected 0xdead)
[INFO] Accum R[1] with 0xffffffffffffe042
[INFO] Read R[1]
[INFO]   Received 0xbeef (expected 0xbeef)
[INFO] Load 0xbad (virt: 0x0xfee9b00, phys: 0x0x8ffffb00) via L1 virtual address
[INFO] Read R[1]
[INFO]   Received 0xbad (expected 0xbad)

I think i'm just not find the file "test-accumulator" in floder“build”

Build failed

Sorry I am totally new to this. When I run "make", it shows the error "/home/x/chipyard/toolchains/riscv-tools/rocket-rocc-examples/build/../bareMetal/accumulator.S:1:10: fatal error: riscv_test.h: No such file or directory."
1 | #include "riscv_test.h"
Which "riscv_test.h" does this accumulator.S use?