riscv-software-src / riscof Goto Github PK

The documentation in for target_run uses inconsistent formatting for the 0 and 1 values.
https://github.com/riscv-software-src/riscof/blame/master/docs/source/plugins.rst#L206-L211

Further from the Python code it is unclear whether this is supposed to be a string or an integer.
But this might just be my lack of understanding of how ini files are handled in Python.
https://github.com/riscv-software-src/riscof/blame/master/docs/source/plugins.rst#L259

Hi!

Can you confirm if you have tested the test bclr-01.S (https://github.com/riscv-non-isa/riscv-arch-test/blob/main/riscv-test-suite/rv32i_m/B/src/bclr-01.S) with Sail 0.5 as reference model? When I try to run it, the test runs indefinetly showing the following message repeatedly

The hex value "1241489331" refers to the instruction "bclr".

In the Github repo of Sail I found an issue showing that "bclr" should have been incorporated riscv/sail-riscv#116. Yet, this dates to the 5 Dec, 2021, while the latest release of Sail is 0.5 from the 15 Jun, 2020.

Regards

How would I use riscof to validate the SweRV RH2 core from Western Digital?
https://github.com/chipsalliance/Cores-SweRV-EH2

I expect that I have to use an EDA tool, like Cadence Xcelium, but I have no idea how to connect riscof to it.
Maybe I am blind, but the documentation mentions the ability to validate RTL code many times but does never state how to do it.

Currently, I have a working Riscof setup with spike and sail_model.

When running riscof testlist and riscof run suite and env is required.
I fixed it by first running riscof arch-test
then running above comands with
--suite=riscv-arch-test --env=riscv-arch-test/riscv-test-suite/env

Which seems to be running fine. but i receive the following errors..

`
......

INFO | [--root--]: Generating database for suite: /home/user/riscof/riscv-arch-test
WARNING | [--riscof.dbgen--]: /home/user/riscof/riscv-arch-test/riscv-test-env/v/entry.S:ISA not specified.
WARNING | [--riscof.dbgen--]: /home/user/riscof/riscv-arch-test/riscv-target/OpenHW/device/cv32e40p/handler.S:ISA not specified.
WARNING | [--riscof.dbgen--]: /home/user/riscof/riscv-arch-test/riscv-target/ibex/device/rv32imc/handler.S:ISA not specified.
WARNING | [--riscof.dbgen--]: /home/user/riscof/riscv-arch-test/riscv-target/ri5cy/device/rv32imc/handler.S:ISA not specified.
INFO | [--root--]: Database File Generated: /home/user/riscof/riscof_work/database.yaml
.....
`

And when running riscof run ....

`
WARNING | [--riscof.utils--]: In file included from /home/user/riscof/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cadd-01.S:19:
/home/user/riscof/riscv-arch-test/riscv-test-suite/env/arch_test.h:14: warning: "TEST_CASE_1" redefined
#define TEST_CASE_1

: note: this is the location of the previous definition
In file included from /home/user/riscof/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/caddi-01.S:19:
/home/user/riscof/riscv-arch-test/riscv-test-suite/env/arch_test.h:14: warning: "TEST_CASE_1" redefined
#define TEST_CASE_1

: note: this is the location of the previous definition
In file included from /home/user/riscof/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/caddi16sp-01.S:19:
/home/user/riscof/riscv-arch-test/riscv-test-suite/env/arch_test.h:14: warning: "TEST_CASE_1" redefined
#define TEST_CASE_1

etc.
`

According to https://riscof.readthedocs.io/en/stable/installation.html#running-riscof riscof testlist doesn't need the arguments --suite and --env but this seems to be incorrect since the command riscof testlist --config=config.ini results in this output:

error: the following arguments are required: --suite, --env
usage: riscof testlist [-h] [--work-dir PATH] [--config PATH] --suite PATH
                       --env PATH

optional arguments:
  --config PATH    The Path to the config file. [Default=./config.ini]
  --env PATH       The Path to the custom env directory.
  --suite PATH     The Path to the custom suite directory.
  --work-dir PATH  The Path to the work-dir.
  -h, --help       show this help message and exit

But as you see the help text is a bit of a mix, both listing them as required and optional.

I'm following the installation guide, but there seems to be dependencies missing form the instructions.

The step apt-get install device-tree-compiler as mentioned at https://github.com/riscv/riscv-isa-sim#build-steps is missing.

Currently the riscof run command starts always by creating a database of the suite, generating a test-list from it, and then running the tests. However the database generation and test-list generation are quite time consuming functions. While debugging bugs one would like to skip these everytime and simply start from the test-list itself.

Proposal:
To the run command add 2 more sub args: --dbfile and --testlist. These arguments along with the --suite will all be mutually exclusive. Following is what is done for each args:

• --suite: run database generation, generate test-list and run tests on target and reference
• --dbfile: generate test-list and run tests on target and reference (assumes the dbfile is correct and updated)
• --testfile: directly run tests on the target and reference (assumes the testlist has been correctly generated)

Only one of the above should be enabled at any point.

One other subargs could be to run the tests only on the reference or the target and then quit (before performing the signature comparisons) This can be achieved by having --no-ref-run and --no-dut-run sub args to the run command

The following dependencies cause issues while installing/running riscof.

• 3.1.17 of GitPython has been yanked from PyPi. Support for python 3.6 has been dropped in the latest release.
• python 3.6.0 and Jinja 3.0.1 are incompatible with each other. A solution is to hardcode Jinja version to 3.0.0 or conditionally pick up 3.0.0 as a part of setup. The latter is preferred.

when running test cases, flow was hanging at "INFO | Runing Tests on Reference Model"

what's going on? what else can I　ｄｏ？

When trying to riscof validateyaml --config=config.ini(here) with the template ini file generated by riscof setup --dutname=spike (here) I get the following error:

     INFO | [--riscv_config.checker--]: Load Schema /home/bo/.local/lib/python3.6/site-packages/riscv_config/schemas/schema_isa.yaml
Traceback (most recent call last):
  File "/home/bo/.local/bin/riscof", line 11, in <module>
    sys.exit(execute())
  File "/home/bo/.local/lib/python3.6/site-packages/riscof/main.py", line 211, in execute
    isa_file = checker.check_isa_specs( isa_file, work_dir, True)

   [...]

  File "/home/bo/.local/lib/python3.6/site-packages/ruamel/yaml/reader.py", line 249, in check_printable
    'special characters are not allowed',
ruamel.yaml.reader.ReaderError: unacceptable character #x0080: special characters are not allowed
  in "/home/bo/.local/lib/python3.6/site-packages/riscv_config/schemas/schema_isa.yaml", position 15493

This corresponds to the ’ at ~/.local/lib/python3.6/site-packages/riscv_config/schemas/schema_isa.yaml:493:
default: The mstatus register keeps track of and controls the hart’s current

My installed python version is:

> python3 --version
Python 3.6.9

If this riscv-non-isa/riscv-arch-test#189 proposal is implemented, the RVTEST_ISA macro parser, the database yaml and possibly the test-list generation functions will have to be modified accordingly.

The template code (riscof_model.py) contains some benign but possibly bogus Python:

    def __init__(self, *args, **kwargs):
        sclass = super().__init__(*args, **kwargs)

        # [...]

        return sclass

The __init__ function is not supposed to return anything, on pain of TypeError. Quoth the data model docs:

Because new() and init() work together in constructing objects (new() to create it, and init() to customize it), no non-None value may be returned by init(); doing so will cause a TypeError to be raised at runtime.

I am guessing the threatened TypeError isn't emitted unless we're using mypy (which we aren't). However, it's possible there's something gnarlier happening behind the scenes and this code is intentional.

Overall, though, I think this is a bug. Because it's in template code, it tends to proliferate:

https://github.com/stnolting/neorv32-riscof/blob/main/plugin-neorv32/riscof_neorv32.py#L63
https://github.com/olofk/serv/blob/main/verif/plugin-sail_cSim/riscof_sail_cSim.py#L30

...which makes it worth tidying up.

Thank you for all your hard work.

Need a way to tell what version of riscof is installed.

This can be implemented either via the --help option or adding a new --version option.

Please be aware that the following command in the Quickstart is wrong :
riscof --verbose info arch-tests --clone
It should be replace by :
riscof --verbose info arch-test --clone
Best regards

Hi,

I am currently applying Riscof on a core compiled with Verilator. I have found inspiration from #10, which in general has helped me a lot. Thank you for the extended answer in this issue.

In the above issue it is written "Currently test selection is only based on the ISA of the core for the most part (...)"? Furthermore, in the quickstart guide and chromite example, the only configured CSR is misa. So, my question is, if any of you can elaborate on this line? Will it effect my test results if I leave out standard CSR's which have been configured, custom CSR's and other harts from the isa-yaml file? Or is this "only" important if you are using Spike as a DUT?

I hope, I make sense :)

Regards
Don-Haugaard

The generated config.ini contains absolute paths, there are two issues with this:

1. The file is not portable to a different folder without changing the paths from absolute to relative,
2. If the file is committed to a public repo, some private information about the developer could be exposed (username).

since the framework and the tests are now in separate repos, it seems like a convenient feature to have riscof clone the correct repository from github. This way, a user simply needs to install riscof and run a subcommand to clone the correct version of the riscv-arch-test repo.

This should be implemented as a separate subcommand (maybe riscof test-clone). This sub-command shuold be able to :

• clone the riscv-arch-test repo and checkout the latest tag
• clone the riscv-arch-test repo and cehckout a specif version by tag or commit
• update an existing cloned repo to a specific tag/version

Some enterprise systems/environment may typically disable/restrict remote access or git commands, and thus this command should be completely optional and should not have any dependencies on the rest of the framework. This should be treated as an optional feature.

It is currently assumed that the hartid being tested is 0. This should be an argument in the cli to enable testing harts with other hartid values.

Hey,

I'm trying to run the riscof with my design but what I'm observing is that sometimes the tool does not wait for my simulation to run/end or either it doesn't compile the test program, thus it calls the simulator with an inexistent my.elf file. Then if I enter in the folder and run the target manually, it compiles and run the test program as expected generating the proper signature. This was working before with the old setup of riscv-arch-tests. Also, as you can check in the picture it fails in the compilation of some tests with the redefinition of some macros.

In the image below for instance, the signature was being dumped correctly (right- reference) when the tool just stopped the sim run as you can check in the left hand side.

It would be helpful to add the timeout option to section 4.7. I couldn't find it elsewhere in the docs, and I was getting a timeout error compiling on a slower server. It showed up in a Google search.

Is there way to get complete command/options list for riscovf? current --help prints only few commands options.
Is there also option to run a specific test only?

Hey guys,

thanks for nice work with RISCOF framework!, I'd like ask you if there's a way to use spike isa-sim or another RV simulator as a reference model that supports misaligned lw/sw. I discovered recently that there's a way to enable misaligned support on Sail RISC-V C-simulator but the problem is that it seems to not be fully working as per ticket down below. If there's an alternative for it'd be really great.

riscv/sail-riscv#156

I've attached my report to show that only misaligned-sw are currently failing with this model....
report.html.zip

• Shallow clones can be used while cloning the arch-test repo as the history and other branches are irrelevant for running the tests.
• Update link to arch-test repo.

As per the spec "The D extension depends on the base single-precision instruction subset F", which requires both F and D to be passed as part of the ISA string in-order to execute the double precision tests. So when we run a case with flen = 64, the current code always passed it as 32 and the statement under elif won't get executed at all for the D extension tests.

I want to do compliance test for Swerv EH1 Core using riscof. But I got following error after installing riscof and running "riscof --help"

Please guide and resolve this issue.

regards

How to give path in config.ini file, relative to config.ini location: A path starting from the point where config.ini is stored. Can you give an example?

Also Is it possible to give paths having environment variables?

Currently there isn't any support in riscof to feed in the flen to isac. Something similar as below would fix the issue.

                coverage_cmd = 'riscv_isac --verbose info coverage -d \
                        -t {0}.log --parser-name c_sail -o coverage.rpt  \
                        --sig-label begin_signature  end_signature \
                        --test-label rvtest_code_begin rvtest_code_end \
                        -e ref.elf -c {1} -x{2} -f{3} {4};'.format(\
                        test_name, ' -c '.join(cgf_file), self.xlen,self.flen, cov_str)

But again the question is from where do we get this flen in riscof, from config file is one option we may consider.

Since the framework and tests are now in different repos, the RISCOF run report must also include the tag/version of the riscv-arch-test repo.

To achieve this, during the riscof run command, we will check

• if the suite directory is a git repo
• if the repo is indeed the riscv-arch-test repo
• There are no changes/pending commits to the repo
• then collect the tag that has been checked out.

If any of these checks fail, the report will indicate that a custom a suite was used to run the tests.

I am going through the entire documentation preparing spelling, grammar, consistency fixes.
This are all small/trivial/cosmetic changes.

What would be the preferred capitalization of the phrase in the issue title? I encountered both versions.

The default template (Templates/setup/model/riscof_model.py) creates a Makefile and invokes it. The net result is a command along the lines of

make -j16 -f Makefile.DUT-thing TARGET0 TARGET1 TARGET2 TARGET3 TARGET4 TARGET5 TARGET6 TARGET7 TARGET8 TARGET9 TARGET10 TARGET11 TARGET12 TARGET13 TARGET14 TARGET15 TARGET16 TARGET17 TARGET18 TARGET19 TARGET20 TARGET21 TARGET22 TARGET23 TARGET24 TARGET25 TARGET26 TARGET27 TARGET28 TARGET29 TARGET30 TARGET31 TARGET32 TARGET33 TARGET34 TARGET35 TARGET36 TARGET37 TARGET38 TARGET39 TARGET40 TARGET41 TARGET42 TARGET43 TARGET44 TARGET45 TARGET46 TARGET47 TARGET48 TARGET49 TARGET50 TARGET51 TARGET52 TARGET53 TARGET54 TARGET55 TARGET56 TARGET57 TARGET58 TARGET59 TARGET60 TARGET61 TARGET62 TARGET63 TARGET64 TARGET65 TARGET66 TARGET67 TARGET68 TARGET69 TARGET70 TARGET71 TARGET72 TARGET73 TARGET74 TARGET75 TARGET76 TARGET77 TARGET78 TARGET79 TARGET80

Perfect - we run tests in parallel. However, because make is not invoked with the "-k" option, a failing test prevents subsequent tests from being queued and executed. For these tests, no signature file is written and the following step fails as follows:

INFO | Running Tests on Reference Model.
INFO | Initiating signature checking.
ERROR | Signature file : /path/to/DUT-thing.signature does not exist

This brings down the runner and no reports are generated, which is not intended behaviour.

The template should invoke make with the "-k" option, which tells make to continue queueing and executing unrelated jobs after a failure. The makeUtil class (riscof/utils.py) does not provide a mechanism to pass this option along, but it can be added directly to the template as follows:

--- a/riscof/Templates/setup/model/riscof_model.py
+++ b/riscof/Templates/setup/model/riscof_model.py
@@ -117,7 +117,7 @@ class dutname(pluginTemplate):
 
       # set the make command that will be used. The num_jobs parameter was set in the __init__
       # function earlier
-      make.makeCommand = 'make -j' + self.num_jobs
+      make.makeCommand = 'make -k -j' + self.num_jobs
 
       # we will iterate over each entry in the testList. Each entry node will be refered to by the
       # variable testname.

I have tried this in my version of the template script and it seems like a definite improvement: when a test fails, make continues anyways and a report is generated.

As part of the migration from gitlab to github the following needs to be done on a high priority:

• replace gitlab ci with github actions for auto release
• remove hosted test-suite, cgf files and database files
• make the suite and env args required for all sub commands
• remove the generate_standard database command
• update docs with correct and new links to riscof and riscof-plugins

Tasks not required on this repo but necessary for migration:

• copy the spike-plugin to riscv-isa-sim directory
• copy the sail-plugin to riscv-sail directory

In the following documentation section:

https://github.com/riscv-software-src/riscof/blob/69b57e14a01b8e1eb5249ca757f3d08a1a6e2c72/docs/source/installation.rst#cloning-the-architectural-tests

...the command

$ riscof --verbose info arch-tests --clone

should be

$ riscof --verbose info arch-test --clone

(without the plural on "arch-test")

I have read the custom plugins of the document
but still not figure out how to use riscof with verilator
where verilator does not takes binary files

should i create a custom riscof Makefile and modify the run command ?
i also haven't figured out where to create the makefile >_>

sorry for being dumb

I get this message in several situations where IMO my ISA string should be accepted.
Some examples:
RV32IMCZicsr_Zifencei_Zba_Zbb_Zbc_Zbs_Zkn_Zks_Zkt_Xfoo (Xfoo not recognized)
RV32IMCNZicsr_Zifencei (says U must be with N, RV32IMCNUZicsr_Zifencei is accepted but there is no requirement to place U in the ISA string AFAIK)
RV32IMC_Zicsr_Zifencei
rv32imczicsr_zifencei

So the ISA-string parsing doesn't work more than works.
It is extremily bad that I must have two different strings / some kind of converter between GNU-GCC-accepted and riscof formats (e.g. GNU GCC rejects any CAPITAL letter).

When trying to follow the quickstart guide as in: https://riscof.readthedocs.io/en/stable/installation.html# the riscof run command at the very end fails.

I've followed the installation step by step on a clean Ubuntu 20.04 LTS install in Windows Subsystem for Linux with the following directories and decisions made with every step succeeding:

1. Installed and used pyenv for managing my Python versions in ~/.pyenv and activated the shell
2. Installed RISCOF via git
3. Installed the RISCV-GNU toolchain with ./configure install prefix path ~/riscof
4. Cloned Spike in ~/riscv-sims/riscv-isa-sim and ./configure install prefix path ~/riscv-sims/spike
5. Cloned Sail in ~/riscv-sims/sail-riscv
6. In folder ~/risctest/spike, ran the riscof setup --dutname=spike command
7. Changed the ini file to include the path to riscv_sim_RV32
8. Cloned the architectural tests without problems, and validated the yaml and generated the test list without problem

Then, finally running in the same directory riscof run --config=config.ini --suite=riscv-arch-test/riscv-test-suite/ --env=riscv-arch-test/riscv-test-suite/env fails with a Python AttributeError: 'Token' object has no attribute 'test'.

It does not open up a HTML page with the test results, and just exits.
Seemingly, the tool does simulate and collects the test signature for both the reference and the dut. The tool also seemingly runs perfect up to that point, and logs the info messages to the shell no problem.
A full stacktrace can be found below:

user@COMPUTER:~/risctest/spike$ riscof run --config=config.ini --suite=riscv-arch-test/riscv-test-suite/ --env=riscv-arch-test/riscv-test-suite/env    INFO | ****** RISCOF: RISC-V Architectural Test Framework 1.23.2 *******
    INFO | using riscv_isac version : 0.8.0
    INFO | using riscv_config version : 2.11.1
    INFO | Reading configuration from: /home/user/risctest/spike/config.ini
    INFO | Preparing Models
    INFO | Input-ISA file
    INFO | Loading input file: /home/user/risctest/spike/spike/spike_isa.yaml
    INFO | Load Schema /home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/riscv_config/schemas/schema_isa.yaml
    INFO | Processing Hart: hart0
    INFO | Initiating Validation
    INFO | No errors for Hart: 0 :)
    INFO | Initiating post processing and reset value checks.
    INFO | Dumping out Normalized Checked YAML: /home/user/risctest/spike/riscof_work/spike_isa_checked.yaml
    INFO | Input-Platform file
    INFO | Loading input file: /home/user/risctest/spike/spike/spike_platform.yaml
    INFO | Load Schema /home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/riscv_config/schemas/schema_platform.yaml
    INFO | Initiating Validation
    INFO | No Syntax errors in Input Platform Yaml. :)
    INFO | Dumping out Normalized Checked YAML: /home/user/risctest/spike/riscof_work/spike_platform_checked.yaml
    INFO | Generating database for suite: /home/user/risctest/spike/riscv-arch-test/riscv-test-suite
    INFO | Database File Generated: /home/user/risctest/spike/riscof_work/database.yaml
    INFO | Env path set to/home/user/risctest/spike/riscv-arch-test/riscv-test-suite/env
    INFO | Running Build for DUT
    INFO | Running Build for Reference
    INFO | Selecting Tests.
    INFO | Running Tests on DUT.
    INFO | Running Tests on Reference Model.
    INFO | Initiating signature checking.
    INFO | Following 90 tests have been run :

    INFO | TEST NAME                                          : COMMIT ID                                : STATUS
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cadd-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/caddi-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/caddi16sp-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/caddi4spn-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cand-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/candi-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cbeqz-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cbnez-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cebreak-01.S : 0e77784916ed9c07842883ba6c62db2555a8335f : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cj-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cjal-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cjalr-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cjr-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cli-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/clui-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/clw-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/clwsp-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cmv-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cnop-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cor-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cslli-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/csrai-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/csrli-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/csub-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/csw-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cswsp-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/C/src/cxor-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/add-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/addi-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/and-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/andi-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/auipc-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/beq-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/bge-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/bgeu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/blt-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/bltu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/bne-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/fence-01.S : ff9358edda915263426801f485b5be24f788862c : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/jal-01.S : ff9358edda915263426801f485b5be24f788862c : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/jalr-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/lb-align-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/lbu-align-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/lh-align-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/lhu-align-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/lui-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/lw-align-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/or-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/ori-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/sb-align-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/sh-align-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/sll-01.S : 7f99888d992294a61b2a6bc8da4d8e3a9db0a3b8 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/slli-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/slt-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/slti-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/sltiu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/sltu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/sra-01.S : 7f99888d992294a61b2a6bc8da4d8e3a9db0a3b8 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/srai-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/srl-01.S : 7f99888d992294a61b2a6bc8da4d8e3a9db0a3b8 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/srli-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/sub-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/sw-align-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/xor-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/I/src/xori-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/M/src/div-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/M/src/divu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/M/src/mul-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/M/src/mulh-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/M/src/mulhsu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/M/src/mulhu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/M/src/rem-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/M/src/remu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/Zifencei/src/Fencei.S : ff9358edda915263426801f485b5be24f788862c : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/ebreak.S : 0e77784916ed9c07842883ba6c62db2555a8335f : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/ecall.S : 2c53665403e472c95f81e23bf4351ee5c8bd0990 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-beq-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-bge-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-bgeu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-blt-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-bltu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-bne-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-jal-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-lh-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-lhu-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-lw-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-sh-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign-sw-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign1-jalr-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
    INFO | /home/user/risctest/spike/riscv-arch-test/riscv-test-suite/rv32i_m/privilege/src/misalign2-jalr-01.S : 6ba8712973dade81eaf1182d1050e5799a7ef2a0 : Passed
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.6.0/bin/riscof", line 33, in <module>
    sys.exit(load_entry_point('riscof==1.23.2', 'console_scripts', 'riscof')())
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/riscof/cli.py", line 335, in run
    template = Template(report_template.read())
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/environment.py", line 1195, in __new__
    return env.from_string(source, template_class=cls)
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/environment.py", line 1092, in from_string
    return cls.from_code(self, self.compile(source), gs, None)
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/environment.py", line 749, in compile
    source = self._parse(source, name, filename)
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/environment.py", line 606, in _parse
    return Parser(self, source, name, filename).parse()
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/parser.py", line 1038, in parse
    result = nodes.Template(self.subparse(), lineno=1)
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/parser.py", line 1012, in subparse
    add_data(self.parse_tuple(with_condexpr=True))
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/parser.py", line 725, in parse_tuple
    args.append(parse())
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/parser.py", line 523, in parse_expression
    return self.parse_condexpr()
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/parser.py", line 528, in parse_condexpr
    expr1 = self.parse_or()
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/parser.py", line 543, in parse_or
    left = self.parse_and()
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/parser.py", line 552, in parse_and
    left = self.parse_not()
  File "/home/user/.pyenv/versions/3.6.0/lib/python3.6/site-packages/jinja2/parser.py", line 560, in parse_not
    if self.stream.current.test("name:not"):
AttributeError: 'Token' object has no attribute 'test'

If anybody has an idea how to solve this issue, or if you need more information about my file structure, do let me know!

Getting this error when running RISCOF run for riscv-test-suite/rv64i_m/I/src/sll-01.S which has
RVTEST_ISA("RV64i")

  File "/release/ventana/vt1/venv/cb1d784/lib/python3.8/site-packages/riscof/framework/main.py", line 200, in run
    results = test.run_tests(dut, base, ispec['hart0'], pspec, work_dir, cntr_args)
  File "/release/ventana/vt1/venv/cb1d784/lib/python3.8/site-packages/riscof/framework/test.py", line 412, in run_tests
    test_list, test_pool = generate_test_pool(ispec, pspec, work_dir, cntr_args[0])
  File "/release/ventana/vt1/venv/cb1d784/lib/python3.8/site-packages/riscof/framework/test.py", line 341, in generate_test_pool
    isa = prod_isa(ispec['ISA'],db[file]['isa'])
  File "/release/ventana/vt1/venv/cb1d784/lib/python3.8/site-packages/riscof/framework/test.py", line 285, in prod_isa
    prefix = match[0][0]
IndexError: list index out of range

We probably need to make these two regexp case insensitive

        match = re.findall("(?P<prefix>RV(64|128|32)(I|E))",entry)
        prefix = match[0][0]
        exts = isa_set(re.sub("RV(64|128|32)(I|E)","",entry))

Hello everyone,
I am trying to build a simple risc-v processor that implements the RV32IZicsr instruction set (and only M mode). I integrated the riscof framework for testing purposes, and discovered that the unprivileged instructions are all passing the tests, but many misaligned tests are failing. Specifically:
lb-align-01.S, lbu-align-01.S, lh-align-01.S, lhu-align-01.S, lw-align-01.S, sb-align-01.S, sh-align-01.S, misalign-beq-01.S, misalign-bge-01.S, misalign-bgeu-01.S, misalign-blt-01.S, misalign-bltu-01.S, misalign-bne-01.S, misalign-jal-01.S, misalign-lh-01.S, misalign-lhu-01.S, misalign-lw-01.S, misalign-sh-01.S, misalign-sw-01.S, misalign2-jalr-01.S.

I browsed the issues and discovered that this has been reported here riscv-non-isa/riscv-arch-test#186 .
But they claim that riscof will have solved these issues. So I have the following questions.

1. On misaligned loads and stores the risc-v manual says (Sec:2.6) that: Loads and stores where the effective address is not naturally aligned to the referenced datatype have behavior dependent on the EEI. I intend to run the FreeRTOS operating system on the processor, which doesn't handle misaligned loads/stores in its trap handler. So my question is, do I have to handle misaligned loads/stores? And if I do have to handle them, does that mean that the FreeRTOS port is broken? Here is a link to the FreeRTOS port https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/main/portable/GCC/RISC-V/portASM.S

2. If the answer to the previous question is no (load/stores don't need to be handled), then is perhaps my riscof configuration wrong, or incomplete?

3. Since I don't support compressed instructions, I have to throw an address misaligned on jump and branch instructions that aren't 4 byte aligned. My question is, do the tests assume that this exception is implemented (i.e. not fatal)? and is this a requirement of the spec? or is it a drawback of the testing framework?

4. If the answer to the previous question is that the spec does not require handling of misaligned jumps/branches then, is my riscof configuration wrong or incomplete?

Here is my riscof configuration for the core:

hart_ids: [0]
hart0:
  ISA: RV32IZicsr
  physical_addr_sz: 32
  User_Spec_Version: '2.3'
  supported_xlen: [32]
  misa:
   reset-val: 0x40000100
   rv32:
     accessible: true
     mxl:
       implemented: true
       type:
           warl:
              dependency_fields: []
              legal:
                - mxl[1:0] in [0x1]
              wr_illegal:
                - Unchanged
     extensions:
       implemented: true
       type:
           warl:
              dependency_fields: []
              legal:
                - extensions[25:0] bitmask [0x0001104, 0x0000000]
              wr_illegal:
                - Unchanged

and for the platform:

mtime:
  implemented: true
  address: 0xbff8
mtimecmp:
  implemented: true
  address: 0x4000
nmi:
  label: nmi_vector
reset:
  label: reset_vector

1. • The current html report must be named: <instanceID>-<test-date>.html . Where <test-date> is in YYYY-MM-DD format using GMT as the time zone. <instanceID> can be a concatenation of the vendor and device strings from the isa yaml. Use underscore for delimiter in concatenation.
2. • mvendorid and mimpids from the isa yaml also need to be captured in the report as well as part of the top table
3. • we also need to capture the toolchain and version used by the DUT and the version of the reference model in the report. This is going to be somewhat tricky to get. I propose the initialize function of the DUT/Reference plugin can return this information to RISCOF which directly goes into the report. Other ideas are welcomed.

I followed the riscof installation quickstart guide and when I ran the tests using the following command some tests passed and others failed

riscof run --config=config.ini --suite /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m --env /home/owaisfarooq/riscv-arch-test/riscv-test-suite/env/

riscof version: RISC-V Architectural Test Framework., version 1.23.2
python3 version: 3.8.10

    INFO | ****** RISCOF: RISC-V Architectural Test Framework 1.23.2 *******
    INFO | using riscv_isac version : 0.8.0
    INFO | using riscv_config version : 2.10.1
    INFO | Reading configuration from: /home/owaisfarooq/riscof_test/config.ini
    INFO | Preparing Models
    INFO | Input-ISA file
    INFO | Loading input file: /home/owaisfarooq/riscof-plugins/spike_simple/spike_simple_isa.yaml
    INFO | Load Schema /home/owaisfarooq/.local/lib/python3.8/site-packages/riscv_config/schemas/schema_isa.yaml
    INFO | Processing Hart: hart0
    INFO | Initiating Validation
    INFO | No errors for Hart: 0 :)
    INFO | Initiating post processing and reset value checks.
    INFO | Dumping out Normalized Checked YAML: /home/owaisfarooq/riscof_test/riscof_work/spike_simple_isa_checked.yaml
    INFO | Input-Platform file
    INFO | Loading input file: /home/owaisfarooq/riscof-plugins/spike_simple/spike_simple_platform.yaml
    INFO | Load Schema /home/owaisfarooq/.local/lib/python3.8/site-packages/riscv_config/schemas/schema_platform.yaml
    INFO | Initiating Validation
    INFO | No Syntax errors in Input Platform Yaml. :)
    INFO | Dumping out Normalized Checked YAML: /home/owaisfarooq/riscof_test/riscof_work/spike_simple_platform_checked.yaml
    INFO | Generating database for suite: /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m
    INFO | Database File Generated: /home/owaisfarooq/riscof_test/riscof_work/database.yaml
    INFO | Env path set to/home/owaisfarooq/riscv-arch-test/riscv-test-suite/env
    INFO | Running Build for DUT
    INFO | Running Build for Reference
    INFO | Selecting Tests.
    INFO | Running Tests on DUT.
    INFO | Running Tests on Reference Model.
    INFO | Initiating signature checking.
    INFO | Following 116 tests have been run :

    INFO | TEST NAME                                          : COMMIT ID                                : STATUS
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cadd-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/caddi-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/caddi16sp-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/caddi4spn-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/caddiw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/caddw-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cand-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/candi-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cbeqz-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cbnez-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cebreak-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cj-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cjalr-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cjr-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cld-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cldsp-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cli-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/clui-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/clw-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/clwsp-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cmv-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cnop-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cor-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csd-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csdsp-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cslli-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csrai-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csrli-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csub-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csubw-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csw-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cswsp-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cxor-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/add-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/addi-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/addiw-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/addw-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/and-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/andi-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/auipc-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/beq-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/bge-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/bgeu-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/blt-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/bltu-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/bne-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/fence-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/jal-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/jalr-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lb-align-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lbu-align-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/ld-align-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lh-align-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lhu-align-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lui-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lw-align-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lwu-align-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/or-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/ori-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sb-align-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sd-align-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sh-align-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sll-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/slli-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/slliw-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sllw-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/slt-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/slti-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sltiu-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sltu-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sra-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/srai-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sraiw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sraw-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/srl-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/srli-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/srliw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/srlw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sub-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/subw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sw-align-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/xor-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/xori-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/div-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/divu-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/divuw-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/divw-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/mul-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/mulh-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/mulhsu-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/mulhu-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/mulw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/rem-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/remu-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/remuw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/remw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/Zifencei/src/Fencei.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/ebreak.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/ecall.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-beq-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-bge-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-bgeu-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-blt-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-bltu-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-bne-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-jal-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-ld-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-lh-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-lhu-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-lw-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-lwu-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-sd-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-sh-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-sw-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign1-jalr-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign2-jalr-01.S : -                                        : Failed
    INFO | Test report generated at /home/owaisfarooq/riscof_test/riscof_work/report.html.
    INFO | Opening test report in web-browser

The normalisation of cgf in ISAC is a resource intensive process which currently dominates the time taken to report coverage. The normalisation can be performed by riscof and the output of the normalised cgf can be written down in the work directory. This file can then used to compute coverage in the plugin and the arguments passed to the plugins can reflect this file path.

In Quickstart section 3.7 there is CLI:

$ riscof --verbose info arch-tests --clone

When executed it returns an error:

$ riscof --verbose info arch-tests --clone
Usage: riscof [OPTIONS] COMMAND [ARGS]...
Try 'riscof --help' for help.

Error: No such command 'arch-tests'.

This needs to happen here : https://github.com/riscv/riscof/blob/f4ceecc87af6c98baff9886043690352c03f3d10/riscof/framework/test.py#L273-L276

This place (and some more) calls raise that calls exit.
Exit exits before raise raises.
This is very strange construction to me, especially raise on success.
What are intentions of it?

riscof runs normally for me with GCC 11.x installed. It's slow but usable.

I've been trying to upgrade to GCC 12.2 (specifically 2023.01.31).

riscof appears to hang at
INFO | Running Tests on Reference Model.

Looking into riscof_work, I see that the reference runs have become extremely slow. In particular, I get to rv32i_m/D/src/fmadd.d_b15-01.S. Generating the objdump takes 4 minutes. It's similar on the other FMA b_15 variants.

Is this a known issue?

I'm interested in compiling the bit manipulation tests. Is there an older version of gcc that is adequate?

Thank you,

David

-rw-rw-r-- 1 harris harris 355094731 Feb 7 16:04 fmadd.d_b15-01.log
-rwxrwxr-x 1 harris harris 15646896 Feb 7 16:00 ref.elf
-rw-rw-r-- 1 harris harris 174017680 Feb 7 16:04 ref.elf.objdump
-rw-rw-r-- 1 harris harris 3672216 Feb 7 16:04 Reference-sail_c_simulator.signature

Are ZB* bitmanip extensions supported? If yes, how do I specify supported ZB* extensions in isa.yaml

The testlist docs suggest using the command

riscof testlist --config=config.ini

https://riscof.readthedocs.io/en/stable/testlist.html

However, --suite and --env are now required arguments as well. The example should be updated to match.

$ riscof testlist --help
INFO | ****** RISCOF: RISC-V Architectural Test Framework 1.24.1 *******
INFO | using riscv_isac version : 0.11.0
INFO | using riscv_config version : 2.13.1
Usage: riscof testlist [OPTIONS]

Generate the test list for the given DUT and suite.

Options:
--suite PATH Path to the custom Suite Directory. [required]
--env PATH Path to the env directory for the suite. [required]
--config PATH The Path to the config file. [Default=./config.ini]
--work-dir PATH Path to the work directory. [Default = ./riscof_work]
--help Show this message and exit.

In the installation instructions, rename "arch-tests" to "arch-test".

$ riscof --verbose info arch-tests --clone

All riscv tests have RVTEST_ISA macro fro which the isa string is extracted to be passed as -march option to the compiler.
The rv64i_m/C/src/cebreak-01.S has RVTEST_ISA("RV64IC")
but it also defines rvtest_mtrap_routine which includes CSRRW instruction. Our new compiler (riscv64-unknown-elf-gcc) needs the -march option to have Zicsr to recognize these csrrw instructions.
Is there a way to append to the RVTEST_ISA string ?

Hi,

I have followed your quickstart guide to setup and run RISCOF (version 1.23) with Spike as DUT, but ran into issues as RISCOF fails to compile two tests (riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S, riscv-arch-test/riscv-test-suite/rv32i_m/F/src/fsw-align-01.S). These tests are put into the generated test list by the suggested "riscof testlist" command. I have installed the RISCV-GNU toolchain and plugin models according to the quickstart guide, so they should be the correct version.

The error messages I get when running "riscof run --config=config.ini --suite=riscv-arch-test/riscv-test-suite/ --env=riscv-arch-test/riscv-test-suite/env" are:

riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S: Assembler messages:
riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S:37: Error: unrecognized opcode flw f18,0x200(x26)' riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S:37: Error: unrecognized opcode fsreg f18,0(x15)'
(...)
riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S:161: Error: unrecognized opcode fsreg f16,128(x15)' riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S:165: Error: unrecognized opcode flw f2,-0x800(x12)'
riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S:165: Error: unrecognized opcode `fsreg f2,136(x15)'

When I peek in the Makefile at the failing target I see the following command for the flw test:

TARGET27 : @cd (...)/riscoftest/riscof_work/rv32i_m/F/src/flw-align-01.S/dut; riscv32-unknown-elf-gcc -march=rv32i -static -mcmodel=medany -fvisibility=hidden -nostdlib -nostartfiles -g -T (...)/riscoftest/spike/env/link.ld -I (...)/riscoftest/spike/env/ -I (...)/riscoftest/riscv-arch-test/riscv-test-suite/env (...)/riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S -o my.elf -DTEST_CASE_1=True -DXLEN=32 -mabi=ilp32 ; spike --isa=rv32imc +signature=(...)/riscoftest/riscof_work/rv32i_m/F/src/flw-align-01.S/dut/DUT-spike.signature +signature-granularity=4 my.elf;

I wondered why the test is compiled with a machine architecture (-march=rv32i) that does not include floats. When I change this to "-march=rv32if" half of the error codes disappear, i.e. all "(...) Error: unrecognized opcode `flw (...)":

riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S: Assembler messages:
riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S:37: Error: unrecognized opcode fsreg f18,0(x15)' riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S:41: Error: unrecognized opcode fsreg f28,8(x15)'
(...)
riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S:129: Error: unrecognized opcode fsreg f19,64(x15)' riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S:133: Error: unrecognized opcode fsreg f22,72(x15)'
riscoftest/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S:137: Error: unrecognized opcode `fsreg f8,80(x15)'

I have also tried to run RISCOF version 1.22.1, but this instead had the issue of the two tests simulating infinitely - but they compile. The interactive debugger in Spike shows that the simulation is caught in an infinite loop after simulating the first floating point instruction contained in either of the tests:

(...)
core 0: exception trap_instruction_access_fault, epc 0x00000000
core 0: tval 0x00000000
:
core 0: exception trap_instruction_access_fault, epc 0x00000000
core 0: tval 0x00000000
:
core 0: exception trap_instruction_access_fault, epc 0x00000000
core 0: tval 0x00000000
:
core 0: exception trap_instruction_access_fault, epc 0x00000000
core 0: tval 0x00000000
:
core 0: exception trap_instruction_access_fault, epc 0x00000000
core 0: tval 0x00000000
(...)

The make target used to run the particular tests does not inform Spike that the DUT supports float instructions. One of those targets is seen below:

.PHONY : TARGET27 TARGET27 : @cd (...)/riscoftest/riscof_work/rv32i_m/F/src/flw-align-01.S/dut; riscv32-unknown-elf-gcc -march=rv32if -static -mcmodel=medany -fvisibility=hidden -nostdlib -nostartfiles -g -T (...)/spike/env/link.ld -I (...)/spike/env/ -I (...)/riscv-arch-test/riscv-test-suite/env (...)/riscv-arch-test/riscv-test-suite/rv32i_m/F/src/flw-align-01.S -o my.elf -DTEST_CASE_1=True -DXLEN=32 -DFLEN=32 -mabi=ilp32 ; spike --isa=rv32imc +signature=(...)/riscof_work/rv32i_m/F/src/flw-align-01.S/dut/DUT-spike.signature +signature-granularity=4 my.elf;

Specifically, it is the option "spike --isa=rv32imc", which every test applies, that seems to cause the problem. If I run the target with a different ISA "--isa=rv32imdc" the test simulates correctly. Yet, reading the script used to generate the Makefile (spike/riscof_spike.py) shows that ISA support for float instructions never is considered for Spike's DUT.

I have also tried to run RISCOF version 1.21.0, but this also terminates with an error. The two tests causing me the issues was added to the riscv-arch-test repository at September 10.

Have you experienced any issues like this?

XLEN's default setting is 64. Where to set supported_xlen to 64? I got thi following issue:
ERROR | riscv32-unknown-elf-objdump: executable not found. Please check environment setup

I am intended to run test with 64 and only settup the riscv64-unknown-elf-xxx