Tracking RISC-V Debug Spec v0.11 testing

[en-sc]

This issue is intended to track passrate of riscv-tests/debug testsuite on SiFive HiFive1 A01

Version	Report
e51f869	#1063 (comment)

[en-sc]

@TommyMurphyTM1234, @JanMatCodasip, #1046 and #1054 are merged, so most tests should be passing now. Can you please run the testsuite and report status here?

[TommyMurphyTM1234]

To keep things more concise than in other issues recently I used the hifive1_excludes.yaml file to exclude tests that are currently deemed inapplicable for the HiFive1 target:

# OpenOCD version
openocd -v
Open On-Chip Debugger 0.12.0+dev-03752-ge51f8695e (2024-04-26-21:57)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html

# HiFive1 RAM based tests
./gdbserver.py targets/SiFive/HiFive1.py --exclude-tests hifive1_excludes.yaml

...

:::::::::::::::::::::::::::[ ran 73 tests in 270s ]:::::::::::::::::::::::::::
29 tests returned not_applicable
43 tests returned pass
1 tests returned fail
   DisconnectTest > logs/20240427-011939-HiFive1-DisconnectTest.log

# HiFive1 flash based tests
./gdbserver.py targets/SiFive/HiFive1-flash.py --exclude-tests hifive1_excludes.yaml

...

:::::::::::::::::::::::::::[ ran 73 tests in 249s ]:::::::::::::::::::::::::::
37 tests returned not_applicable
35 tests returned pass
1 tests returned fail
   DisconnectTest > logs/20240427-012429-HiFive1Flash-DisconnectTest.log

So, DisconnectTest is the only unexpected failure and is already covered by this issue:

• Incorrect "valid" read of menvcfg CSR from HiFive1 board #1058

[TommyMurphyTM1234]

Does this need to stay open or can it be closed given that my testing certainly suggests that PRs #1046 and #1054 have addressed the issue of unexpected failures/exceptions (other than with DisconnectTest which seems to be a separate issue)?

[en-sc]

Does this need to stay open ...

I'd like to leave this issue open and track the status of riscv-tests/debug on RISC-V Debug Spec 0.11 here.

[TommyMurphyTM1234]

Ok, thanks @en-sc.

For what it's worth, and as mentioned above, this issue triggered by the DisconnectTest is the only unexpected failure that I'm seeing at the moment when running the debug test suite against the SiFive HiFive1 board and may be down to a bug in OpenOCD's 0.11 support:

• Incorrect "valid" read of menvcfg CSR from HiFive1 board #1058

However, I still have a TODO to look at the existing HiFive1 exclusion list to see if any of the tests there result in failures/exceptions due to issues with the tests or OpenOCD itself and should be capable of working correctly:

• debug: Fix tests so they pass/not_applicable on HiFive1 riscv-software-src/riscv-tests#488