riscv-software-src / riscv-isac Goto Github PK

riscv_isac/plugins/internaldecoder.py:
instrObj.instr_name = 'c.ldsp' instrObj.rd = (rd, 'f') instrObj.rs1 = (2, 'x')
the code mentioned above may cause rd coverage collection failed. instruction c.ldsp does not have 'f' type register.

Unable to execute any tests over isac on the coverage mode. Have tried execution fclass test, but without success. The age old buffer unpack issue looks not fixed yet on the latest isac version.

![Screenshot from 2022-08-24 17-16-08](https://user-images.githubusercontent.com/98386162/186413759-07e7abf0-2f25-4622-9f97-2a130f6b
Isac_Issue.log
21ed.png)

Attached are the logs out of the run done.

Issue Description: Pythons eval function takes the optional arguments as dictionary variable locals, which contains the local variables and it's values, which eventually be used to evaluate the expression that we are passing (in this context it is the cover point expression with the combination of sign, exponent and mantissa values) as referenced here. This cannot be replaced by an ordinary dictionary variable (instr_vars) with it's keys and values. Those keys and it's values are not considered as a variable and it's values, hence isac failed to evaluate the given cover-points, this resulted in ignoring the val_comb of the tests that we run.

Possible Solution: Fix for this issue is to have the local variable dictionaries passed as is (or a copy of it), and it should be incrementally updated with all the necessary variables required, rather emptying the dictionary depends on where we invoke the function define_sem.

CC: @neelgala @allenjbaum @pawks

The documentation of the FP coverpoint generator functions (riscv_isac.fp_dataset.ibm_b1 and friends) are documented without specific knowledge of the instructions they generate for (see https://riscv-isac.readthedocs.io/en/stable/code.html#module-riscv_isac.fp_dataset).

However, the implementation is full of hard-coded assumptions that enforce different behavior for instructions without good reasons (e.g. if opcode.split('.')[0] in ["fadd","fsub","fmul","fdiv","fsqrt","fmadd","fnmadd","fmsub","fnmsub","fcvt","fmv"]:).

The result is, that the generators are useless for new instructions (Q, Zfh, Zfinx, Zfa, etc.). And adding these new instructions into fp_dataset.py is pointless, because there is no need to do so.

fp_dataset.py should be re-implemented with the following ideas in mind:

• generators should not know anything about FP instructions
• specific behavior should be controlled via function parameters, which are properly documented
• constant should be defined on package level, not repeatedly in functions
• seed functionality needs to be streamlined (that would also make it consistent)
• code can be easily written to be compatible with Zfh, F, S, or Q by using simple abstractions (e.g. maxnorm = get_maxnorm(flen))
• if code is copy-and-pasted from function to function, then it should be moved into its own function

Successfully built and installed, but failed to check riscv_isac --help :

➜  ~ riscv_isac --help
Traceback (most recent call last):
  File "~/.pyenv/versions/3.12.0/lib/python3.12/importlib/metadata/__init__.py", line 397, in from_name
    return next(cls.discover(name=name))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/.pyenv/versions/latest/bin/riscv_isac", line 33, in <module>
    sys.exit(load_entry_point('riscv-isac==0.18.0', 'console_scripts', 'riscv_isac')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/.pyenv/versions/latest/bin/riscv_isac", line 22, in importlib_load_entry_point
    for entry_point in distribution(dist_name).entry_points
                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "~/.pyenv/versions/3.12.0/lib/python3.12/importlib/metadata/__init__.py", line 861, in distribution
    return Distribution.from_name(distribution_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/.pyenv/versions/3.12.0/lib/python3.12/importlib/metadata/__init__.py", line 399, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for riscv-isac

Does anybody meet the same error messages?

Currently, the coverpoints defined in the csr_comb node are evaluated for every instruction. But these coverpoints are not bound to an instruction and may evaluate to true over multiple instructions. Therefore, such coverpoints must instead be evaluated only when a CSR update is seen in the Sail log. And for this to work, the architectural state must be updated before evaluating the coverpoints. The DPR logic will also need to be modified accordingly to handle signature updates for such coverpoints.

Summary

The data propagation reports track the impact of an instruction (which matches the coverpoint) on the signature. However, each coverpoint hit might have to cause multiple signature updates and current implementation assumes a single update(word/dword) only. The deliverables for this task are the updates to track multiple signature updates and detect signature overwrites. The logic will also have to account for any updates to the memory region by other memory instructions(like fsw,fsd) to detect successfull propagation of results to the memory. An additional parameter(possibly a cli option) will be necessary to indicate the expected number of signature updates per instance. The DPR logic will also need to be updated to detect overwrites in the signature region.

Basic Example

The following is the signature update macro for the F tests:

#define RVTEST_SIGUPD_F(_BR,_R,_F,...)\
  .if NARG(__VA_ARGS__) == 1;\
    FSREG _R,_ARG1(__VA_ARGS__,0)(_BR);\
    SREG _F,_ARG1(__VA_ARGS__,0)+REGWIDTH(_BR);\
    .set offset,_ARG1(__VA_OPT__(__VA_ARGS__,)0)+(REGWIDTH+REGWIDTH);\
  .endif;\
  .if NARG(__VA_ARGS__) == 0;\
    FSREG _R,offset(_BR);\
    SREG _F,offset+REGWIDTH(_BR);\
    .set offset,offset+(REGWIDTH+REGWIDTH);\
  .endif;

There are 2 updates to the signature region in this macro. The first by a FSREG i.e fsw/fsd instruction and
the second by the SREG i.e sw/sd instruction. Currently only the update by the SREG
instruction is detected. The logic will have to be updated to detect the FSREG update and throw an
error if it does not occur in the execution trace.

"rvopcodesdecoder" has a few issues:

• it imports constants without using it's relative path. this results in import errors when isac is imported as a python package.
• it does not log an error message prior to raising a SystemExit.
• it does not support passing the path to opcodes_dir.

@anuani21

This issue is related to the PR #72

While running the coverage using the risc-v isac command, I get the following error:

Traceback (most recent call last):
  File "/usr/local/bin/riscv_isac", line 33, in <module>
    sys.exit(load_entry_point('riscv-isac', 'console_scripts', 'riscv_isac')())
  File "/home/hammad/.local/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/hammad/.local/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/hammad/.local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/hammad/.local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/hammad/.local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/hammad/mcause_check/new/virtual_memory_implementation/riscv-isac/riscv_isac/main.py", line 149, in coverage
    isac(output_file,elf,trace_file, window_size, preprocessing(expand_cgf(cgf_file,int(xlen),int(flen),log_redundant), header_file, cgf_macro), parser_name, decoder_name, parser_path, decoder_path, detailed, test_label,
TypeError: isac() missing 1 required positional argument: 'procs'

The issue I saw is that you have used the inxFlg in the isac.py:

But, it is not used in the main.py:

I am unable to understand whether the issue is on my end or in the merged PR. Please correct me if I am misunderstanding this.

Dear All,
I have written a basic test in which I am causing illegal instruction exception in M-mode. Here is the test.
basic.txt
This exception will update mcause register and will display it in the signature (which is happening correctly). I have written the following coverpoints for this test.

mcause:
config:
- check ISA:=regex(.*I.Zicsr.); def rvtest_mtrap_routine=True; def rvtest_strap_routine=True
mnemonics:
csrrw: 0
csrrs: 0
csr_comb:
mcause & 0x01 == 0x01 : 0
mcause & 0x02 == 0 x02: 0
mcause & 0x03 == 0x03 : 0

The coverpoint (mcause & 0x02 == 0x02 : 0) should have been passed due to the illegal instruction exception but ISAC is showing it in the report.

rvopcodesdecoder decodes reg operands and immediate values for all instructions. it does not do so for compressed instructions.

Here's an example of how the compressed instructions look post-decode:

{'instr': 17304, 'instr_name': 'c.lw', 'instr_addr': 2147487848, 'rd': None, 'rs1': None, 'rs2': None, 'rs3': None, 'imm': None, 'zimm': None, 'csr': None, 'shamt': None, 'succ': None, 'pred': None, 'rl': None, 'aq': None, 'rm': None, 'reg_commit': ('x', '14', '0x000000000000000a'), 'csr_commit': None, 'mnemonic': None, 'is_rvp': False, 'rs1_nregs': 1, 'rs2_nregs': 1, 'rs3_nregs': 1, 'rd_nregs': 1}

here's the commit for that instruction (in my program, based on spike):

instr: 17304 addr: 0x80001068 instr_name: c.lw reg_commit: ('x', '14', '0x000000000000000a')

here's what is actually supposed to happen (example of lw):

{'instr': 67250179, 'instr_name': 'lw', 'instr_addr': 2147488558, 'rd': (16, 'x'), 'rs1': (4, 'x'), 'rs2': None, 'rs3': None, 'imm': 64, 'zimm': None, 'csr': None, 'shamt': None, 'succ': None, 'pred': None, 'rl': None, 'aq': None, 'rm': None, 'reg_commit': ('x', '16', '0x0000000000000007'), 'csr_commit': None, 'mnemonic': None, 'is_rvp': False, 'rs1_nregs': 1, 'rs2_nregs': 1, 'rs3_nregs': 1, 'rd_nregs': 1}

isac's decoder currently does this (link):

            for arg in args[:-1]:
                if arg == 'rd':
                    treg = reg_type
                    if any([instr_name.startswith(x) for x in [
                            'fcvt.w','fcvt.l','fmv.s','fmv.d','flt','feq','fle','fclass']]):
                        treg = 'x'
                    temp_instrobj.rd = (int(get_arg_val(arg)(mcode), 2), treg)
                if arg == 'rs1':
                    treg = reg_type
                    if any([instr_name.startswith(x) for x in [
                            'fsw','fsd','fcvt.s','fcvt.d','fmv.w','fmv.l']]):
                        treg = 'x'
                    temp_instrobj.rs1 = (int(get_arg_val(arg)(mcode), 2), treg)
                if arg == 'rs2':
                    treg = reg_type
                    temp_instrobj.rs2 = (int(get_arg_val(arg)(mcode), 2), treg)
                if arg == 'rs3':

here's the actual list of variable fields from riscv-opcodes -- being looked up in the arg_lut:

c.lw :: ['rd_p', 'rs1_p', 'c_uimm7lo', 'c_uimm7hi', 'rv_c']

isac's decoder needs to be modified to take into account, these compressed field names, and decode them appropriately from here(link). here's an example code implementation (not compiled) for starters:

                 if 'rd' in arg:
                    treg = reg_type
                    if any([instr_name.startswith(x) for x in [
                            'fcvt.w','fcvt.l','fmv.s','fmv.d','flt','feq','fle','fclass']]):
                        treg = 'x'
                    temp_instrobj.rd = (int(get_arg_val(arg)(mcode), 2), treg)
                if 'rs1' in arg:
                    treg = reg_type
                    if any([instr_name.startswith(x) for x in [
                            'fsw','fsd','fcvt.s','fcvt.d','fmv.w','fmv.l']]):
                        treg = 'x'
                    temp_instrobj.rs1 = (int(get_arg_val(arg)(mcode), 2), treg)
                    # print(f'{instr_name} rs1: {temp_instrobj.rs1}')
                if 'rs2' in arg:
                    treg = reg_type
                    temp_instrobj.rs2 = (int(get_arg_val(arg)(mcode), 2), treg)

the change is in the if condition where i replaced the check for arg == 'rd' with 'rd' in arg.
this change reflects the correct register operand in the decoded artifacts:

{'instr': 17304, 'instr_name': 'c.lw', 'instr_addr': 2147487848, 'rd': (6, 'x'), 'rs1': (7, 'x'), 'rs2': None, 'rs3': None, 'imm': None, 'zimm': None, 'csr': None, 'shamt': None, 'succ': None, 'pred': None, 'rl': None, 'aq': None, 'rm': None, 'reg_commit': ('x', '14', '0x000000000000000a'), 'csr_commit': None, 'mnemonic': None, 'is_rvp': False, 'rs1_nregs': 1, 'rs2_nregs': 1, 'rs3_nregs': 1, 'rd_nregs': 1}

immediates have not been fixed in this example.
there might be a minor quibble in the get_instr function in the same rvopcodesdecoder when it comes to compressed instructions. this will need to be checked as well.

Since #8, there has been a dependency on pluggy which is not listed in install_requires / requirements.txt.

On the other hand, as best as I can tell, pytest is only required to run tests (which is reasonable), yet is present in install_requires instead of only in tests_require, thus requiring every riscv_isac user to have it on their computer (which is not). It would be better to move it to the latter.

riscv_isac/InstructionObject.py defines the following lists:

• unsgn_rs1...instructions that have an unsigned rs1 value
• unsgn_rs2...instructions that have an unsigned rs2 value
• f_instrs_pref...floating-point instructions (without postfixes)

The idea is that the API gets the instruction in the form of a string (e.g. fadd.s or xor), and the code makes decisions based on the instruction.

However, instead of having yet another unmaintained list of instructions that is out of date within a few weeks, the code should be rewritten to use proper (keyword) arguments. In this particular case the name of these parameters are already in the name of the lists above.

Many of the privilege tests need to be able to express conditions which denote a transition in the value of the csrs. To do so, we can define a map variable which contains information about the csr updates as a result of the current information. This variable can then be used to express the condition for transition as follows:

# Coverpoint to test toggle of the D bit in misa.extensions field from 1 to 0.
misa[3] == 1 and misa in csr_update and csr_update['misa'][3] == 0

A particular csr write can be triggered due to one of 4 conditions.

1. CSR operation instruction
2. Trap due to interrupt or exception
3. Side effect of a write to a different csr
4. Side effect of a instruction(like fld/fsd cause a change in the FS bit of mstatus).

Further extension of the feature:

It might also be beneficial to tag each csr update with the reason for such an update. It is possible that at a particular instance multiple csrs are updated due to a mixture of different reasons( like mstatus is updated due to 4 but at the same time an interrupt is taken which causes an update in mstatus too (due to 2)). This might allow us to explicitly specify coverpoints which test for the occurrence of a particular exception or interrupt too.

Thoughts @allenjbaum @neelgala ?

Many of the csrs happen to deal with memory addresses. It is not feasible to define absolute coverpoints for these csrs. Hence the need for memory variables which derive their values based on the instruction stream at runtime. The following variables are needed.

• depa: The physical address of the data access.
• depa_align: The alignment of the physical address i.e the 2bits in the LSB.
• deva: The virual address of the data access. This is the effective address of the access as defined by the states being consumed in the instruction i.e in the absence of translation eva==epa.
• deva_align: Currently the alignment of the address of the data access is expressed in ea_align. This needs to be renamed.
• dptw[stage]a: The address being accessed by the ptwalk stage(depending on the levels of ptwalk needed) in the translation of effective data address from va to pa. In the presence of hypervisor the stage becomes a 2 digit number with the first showing the stage in conversion of va to hpa and the second showing hpa to pa.
• iepa: The physical address of the instruction access.
• iepa_align: The alignment of the physical address i.e the 2bits in the LSB of the instruction address.
• ieva: The virual address of the instruction access. This is the effective address of the access as defined by the states being consumed in the instruction i.e in the absence of translation eva==epa.
• ieva_align: LSB 2 bits of the va of instruction.
• iptw[stage]a: The address being accessed by the ptwalk stage(depending on the levels of ptwalk needed) in the translation of effective instruction address from va to pa. In the presence of hypervisor the stage becomes a 2 digit number with the first showing the stage in conversion of va to hpa and the second showing hpa to pa.

All of these can simply be parsed from the SAIL log since it dumps implicit accesses.

I am using RISCV-ISAC to generate the add instruction coverage. I found that some checkpoints in CGF are 0, but it is obvious from the assembly file that this constraint is satisfied. Is this normal?

The riscv_isac was installed by source code. The spike version is 1.1.1-dev. The gcc version is "riscv32-unknown-elf-gcc (g2ee5e430018) 12.2.0".
I used the following command to generate the instruction coverage.
I first cloned arch-test using RISCOF and then created the default configuration.

riscof --verbose info arch-test --clone
riscof setup --refname=sail_cSim --dutname=spike

Compile:

riscv32-unknown-elf-gcc -march=rv32i -static -mcmodel=medany -fvisibility=hidden -nostdlib -nostartfiles -T ./sail_cSim/env/link.ld         -I ./sail_cSim/env/ -I ./riscv-arch-test/riscv-test-suite/env -mabi=ilp32 ./riscv-arch-test/riscv-test-suite/rv32i_m/I/src/add-01.S -o ref.elf -DTEST_CASE_1=True -DXLEN=32

Generate trace:

spike --log-commits --isa=rv32imc +signature=Reference-spike.signature +signature-granularity=4 ref.elf > add-01.log 2>&1

Generate coverage:

riscv_isac --verbose info coverage -d -t add-01.log --parser-name spike -o coverage.rpt  -e ref.elf -c ./riscv-arch-test/coverage/dataset.cgf -c ./riscv-arch-test/coverage/rvi.cgf -x32 -l add

From the Coverage report, I see that the constraint, "rs1_val == -2147483648", has a count of 0. However, the string can be searched directly from the assembly file, which indicates there is at least one test case that satisfies this constraint.

I have packed all the necessary files. If I make a mistake, please let me know. Thanks.
Test.tar.gz

Summary

The libopcodes library is generated as an aritifact of gcc build. The library supports API calls which decode a given instruction encoding. A python plugin(which implements the pluggy decoder plugin specification as defined here) can be interfaced with the C library using python bindings. This would greatly reduce developmental and maintenance overheads for new extensions. A sample decode plugin can be found here.

Summary

Coverage computation in ISAC is an iterative process where all coverpoints are evaluated for every instruction. This causes an exponential increase in coverage computation time as the number of coverpoints increases. The following strategies may be used to decrease the time taken to measure coverage.

• Removing hit coverpoints from the working-cgf - For architectural testing, it is enough if a coverpoint is hit just once and has an impact on the signature. To leverage this, coverpoints can be dropped once hit to avoid evaluating it for all the subsequent instructions. A CLI option should be added to enable/disable this strategy.

• Launching multiple parallel threads to collect coverage parallelly - Coverage computation is an embarrassingly parallel workload. The set of available coverpoints can be partitioned across multiple sets and each set can be evaluated in parallel to reduce total time taken. A cli option may be added to make the number of parallel threads lauched configurable. Both strategies can be combined together to reduce coverage calculation time further.

Problem

All coverpoints are currently defined over the opcode field in the cgf. Since isac deals with the mnemonics of the instruction rather than the actual encoding, dealing with pseudo-instructions needs additional thought. The same instruction instance of the base op should satisfy/trigger the coverpoints of both the base op and the pseudo instruction for accurate coverage reporting.

Potential Solution

To support this, the nodes in the cgf will have to undergo a overhaul. The opcode field can be renamed to mnemonics and 2 new fields base_op and p_op_cond are added to the covergroups. The base_op and p_op_cond are optional fields which specify the base operation and the condition over the different fields of the instruction, which when satisfied results in the instruction being recognised as the pseudo op. For example, zext.h is a pseudo op of pack in RV64 and packw in RV32 with the rs2 field hardwired to x0. The nodes for zext.h will be expressed as follows:

zext.h_32:
    config: 
      - check ISA:=regex(.*RV32.*B.*)
      - check ISA:=regex(.*RV32.*Zbb.*)
    mnemonics: 
      zext.h: 0
    base_op: packw
    p_op_cond: rs2 == x0
    ...

zext.h_64:
    config: 
      - check ISA:=regex(.*RV64.*B.*)
      - check ISA:=regex(.*RV64.*Zbb.*)
    mnemonics: 
      zext.h: 0
    base_op: pack
    p_op_cond: rs2 == x0
    ...

Schema

• The p_op_cond node is relevant only if the base_op node has been defined. Otherwise both the nodes remain empty.
• The mnemonics node is allowed to have multiple entries only if the base_op node is empty. This introduces a restriction that a pseudo-op cannot be combined with other instructions into a single covergroup.

Changes required

• This change will need an overhaul of all the cgfs defined so far( a simple replace of opcode with mnemonics).
• Changes in ISAC and CTG to support this.

Caveats

• Pseudo-ops cannot be combined with other instructions in a covergroup
• Decoder always detects an encoding as the relevant base-op(i.e pack even if the encoding satisfies conditions for zext.h).
• Coverpoints are defined over the fields of the base_op always.

I followed steps from Quickstart. And I get a error when try to compute coverage from Architecture Test Suite

$ riscv_isac --verbose info coverage -d -t add-01.log --parser-name c_sail --decoder-name internaldecoder -o coverage.rpt --sig-label begin_signature end_signature --test-label rvtest_code_begin rvtest_code_end -e ref.elf -c dataset.cgf -c rvi.cgf -x 32 -l add
    INFO | ****** RISC-V ISA Coverage 0.18.1 *******
    INFO | Copyright (c) 2020, InCore Semiconductors Pvt. Ltd.
    INFO | All Rights Reserved.
    INFO | Start Test Label: rvtest_code_begin @ 0x800000f8
    INFO | End Test Label  : rvtest_code_end @ 0x8000329c
    INFO | Start Signature Label: begin_signature @ 0x80006110
    INFO | End Signature Label  : end_signature @ 0x80006a50
 WARNING | FLEN is set to 32. Commit values in the log will be terminated to 32 bits irrespective of their original size.
    INFO | Writing out updated cgf : /home/pager/Desktop/32work/riscof_work/rv32i_m/I/src/add-01.S/ref/ref.cgf
Traceback (most recent call last):
  File "/home/pager/.local/bin/riscv_isac", line 33, in <module>
    sys.exit(load_entry_point('riscv-isac', 'console_scripts', 'riscv_isac')())
  File "/usr/lib/python3/dist-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/pager/Desktop/riscv-isac/riscv_isac/main.py", line 136, in coverage
    isac(output_file,elf,trace_file, window_size, expand_cgf(cgf_file,int(xlen),int(flen),log_redundant), parser_name, decoder_name, parser_path, decoder_path, detailed, test_label,
  File "/home/pager/Desktop/riscv-isac/riscv_isac/isac.py", line 40, in isac
    rpt = cov.compute(trace_file, test_name, cgf, parser_name, decoder_name, detailed, xlen, flen, test_addr, dump, cov_labels, sig_addr, window_size, no_count, procs)
  File "/home/pager/Desktop/riscv-isac/riscv_isac/coverage.py", line 1468, in compute
    dump_file.write(ruamel.yaml.round_trip_dump(rcgf, indent=5, block_seq_indent=3))
  File "/home/pager/.local/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1284, in round_trip_dump
    error_deprecation('round_trip_dump', 'dump')
  File "/home/pager/.local/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
    raise AttributeError(s, name=None)
AttributeError: 
"round_trip_dump()" has been removed, use

  yaml = YAML()
  yaml.dump(...)

instead of file "/home/pager/Desktop/riscv-isac/riscv_isac/coverage.py", line 1468

    dump_file.write(ruamel.yaml.round_trip_dump(rcgf, indent=5, block_seq_indent=3))

Im not sure which step goes wrong. ths samples cgfs from riscv-ctg

riscv_isac package is not compatible with ruamel.yaml version > 0.18.0 (current latest version is 0.18.6) because of the deprecated method:

AttributeError: 
"round_trip_dump()" has been removed, use

  yaml = YAML()
  yaml.dump(...)

It could be nice to update the dependency on the ruamel.yaml version
ruamel.yaml>=0.16.0,<=0.18.0

If this makes sense, I'll be happy to propose a PR

The DPR computation involves tracking the updates to the signature region. This feature can be extended to obtain the label of the relevant test case from the elf and annotate the signature file with the appropriate instruction label. It will also be useful if this feature had a stand alone CLI and API interface for use with other tools.

(Link to original discussion)

• fcsr & rm in coverpoints - The current coverpoints define effective rm fields i.e insts always have 7 and fcsr has the necessary rm. Future coverpoints will try to test the frm and rm fields individually to see if the rm field of the instruction over-rides the frm value at run time. To support this and the highlighted issue with fflags, the coverpoints will be modified to have conditions defined on the fcsr value along with the rm field of the instruction.
• Nan prefix in coverpoints - Add a new variable in the coverpoints which is defined as the 32 bit msb(in FD) of the value in the register. This is relevant because for most of the floating-point instructions if this prefix is not all 1's, the input is a canonical NaN. Hence the coverpoints should capture this particular aspect while describing cases for fadd.s in RV*FD. Current coverpoints will just have a condition(nan_prefix == 0xffffffff) appended to it.​
• DPR update - Current DPRs only test whether 1 signature update per instruction which hits coverpoints is present. This will be modified to vary from instruction to instruction by maintaining a mapping which tells the number of stores to the memory for each case. ​
• fcvt.*.* coverpoints (Caught by @davidharrishmc) - The coverpoints for the fcvt.a.b instructions currently assume that the target format is b whereas the target format is a. These need to be reworked. The current tests have a wrong coverage definition and hence provide no value. All outputs are NaNs.

The nan boxed value returned by sail is 0x7FF8000000000000 which is assumed to be valid with respect to the spec "Any operation that writes a narrower result to an f register must write all 1s to the uppermost FLEN−n bits to yield a legal NaN-boxed value". But the generated rs_nan_prefix value 0xffffffff (by riscof) is not matching up the extracted nan prefix from the sail log which is 0x0

This has to addressed for each of the fcvt function given below, as the nan boxed value are different for each one of them. Above value are given as an example from fcvt.d.s logs.

fcvt.d.s
fcvt.d.w
fcvt.d.wu

The condition of invoking the define_sem function should not be of the registers starting with f. The solution has to be generalized rather hard-coding the register's alphabets.

P.S: Ideally these changes/revisions in the framework should've taken place after Z_inx PR is merged.

Currently the fields in the instruction object are set by the decoder exclusively. Based on the instruction (p extension/floating point) additional (derived) fields are used in the coverpoints to describe coverage requirements accurately.

Currently these are defined in an ad-hoc manner in the coverage function. This causes repetitive effort across ctg and isac to define and evaluate coverage for such instructions. Re-architecting the field definitions and coverage evaluation as member functions in the instruction object will localize coverage logic and enable per-instruction customization while avoiding duplication. This would ease adding support for future extensions.

For rs3 the functions has to be defined with respect to sign and unsigned values.

The normalisation functions used to abstract representations of coverpoints can potentially produce duplicate coverpoints. However, the coverpoints are stored in dictionaries; hence only unique coverpoints are retained. It is desirable to identify and optimise these to reduce the time taken to normalise a CGF. In order to do this, the duplicate coverpoints need to be printed out in the generated logs (enabled by a switch).

Summary

The riscv-opcodes repository enumerates the instruction encodings of RISCV and contains scripts to convert them to several formats. These can be used to automatically decode the instruction encodings with minimal effort. A python plugin (which implements the pluggy decoder plugin specification as defined here) can leverage the same to ease overheads for new extensions. A sample decode plugin can be found here.

Hello,
I am trying to use this software to extract coverage. But failed. Does anybody can upload an example to try.

• Coverpoints to test nan boxing behaviour (hand coded tests in most cases) - These coverpoints will mostly be described on the nan_prefix described in point 2 above. The minimal required bin is the walking1s.
• F Memory operations test - Memory ops should propagate NaN payloads in case of incorrect NaN boxing too. These will need specific scenarios and test cases. The coverpoints will also have to capture these scenarios precisely. The same coverpoints will also be applicable for the fmv instructions. ​
• Misaligned tests for F memory ops - The floating point memory ops require coverpoints and tests which check for the misaligned effective addresses like their integer counter parts. ​