Supervisor Domains Privileged ISA Extension

SMMTT is a new isolation mechanism proposed for RISC-V platforms. It has several benefits over PMP when considering multi-tenant security use cases, especially in the case of confidential computing. This repository holds the WIP implementations of SMMTT in various projects, mainly QEMU and OpenSBI. The hope is that many of these patches will eventually be upstreamed into their respective open source projects.

This work is being done as part of a Linux Foundation Mentorship project, specifically the Supervisor Domains Priv. ISA Extension Emulation project.

Repository structure

This repository mainly consists of a set of Makefiles, helper scripts, and submodules containing the respective subprojects being integrated. An explanation of most files and directories is below.

• mk/: A variety of Makefiles, containing both helpers for building submodules (linux.mk, opensbi.mk, qemu.mk, and tests.mk) as well as runtime utility targets (run.mk, utils.mk). The build helpers are relatively straightforward, but the run helpers do have some nuance (discussed below).
• scripts/: Various utility scripts.
• gdb/: Helpers for loading symbol files when debugging, both using the run helpers as well as CLion.
• templates/: XML templates for generating run configurations in CLion.
• shared/include/smmtt_defs.h: A shared header used in both OpenSBI and QEMU containing bit encodings for SMMTT functionality. Much easier to do this than try to keep the implementations synchronized manually.

Each submodule includes a couple different branches, with a consistent structure. These include, in typical rebase order:

• master/main: the current latest software version as pulled from upstream.
• to-upstream: patches I have written that have been submitted for review
• feature/*: feature branches for individual, separate high-level features. Right now, these include mpxy for the beta message passing implementation in OpenSBI/QEMU and smmtt

Build instructions

Simply type make in this directory to build release versions of all relevant binaries. The build system additionally recognizes the following environment variables:

• DEBUG: if set, usually to 1, produce debug builds instead
• VERBOSE if set, usually to 1, verbosely compile all subprojects. This is useful for importing the project into an IDE.

You can also build the opensbi{32,64}, linux{32,64}, tests{32,64}, and qemu targets individually to build just those subcomponents.

Debugging

The run helpers (defined in mk/run.mk) implement a large number of targets for running and debugging various parts of the system. There are two variants of the run targets, which are both useful for slightly different purposes.

• Boot targets: these are of the form {tests,linux}-{max,smmtt}{32,64}, and simply test whether the simulator can boot at all. To run these, use a command like make run-tests-max32 or similar. These tests are also used to make sure we don't inadvertently break running with PMP rather than SMMTT.
  • The first part {tests,linux} selects which project to boot (either linux or kvm-unit-tests)
  • The second part {max,smmtt} selects which CPU to boot with (either the default QEMU PMP CPU, or the new QEMU CPU with SMMTT features).
  • The third part selects whether to run a 32-bit or 64-bit test.
• Unit test targets: these are of the form unittests-smmtt{32,64}-smmtt{34,46,56}-{disallow,allow-rw,allow-rx,allow-rwx} and are primarily used to make sure that the SMMTT implementation functions correctly in each configuration. These can be invoked with make run-unittests-smmtt32-smmtt34-disallow or similar.
  • The first part selects whether to run a 32-bit or 64-bit test.
  • The second part selects the specific SMMTT mode to use. Valid choices are smmtt34 for 32-bit, and smmtt{46,56} for 64-bit.
  • The third part selects the permissions to test, for a region of each size.

Additionally, various debugging helpers are implemented. For each of the run targets described above, the run- part of the make target can be substituted with:

• dbg-: Run the specified target in QEMU debug mode, which can then be attached to with a RISC-V cross GDB.
• connect-: Connect to a QEMU debug mode system, typically launched in another shell with a dbg- target.
• qemudbg-: Debug QEMU itself with the host debugger.
• alldbg-: Debug both QEMU with the host debugger and the application with the cross debugger at the same time.
• dumpdtb-: Dump the device tree generated by this run target.

Unit tests

The unit test targets work by instantiating one region of each supported size (i.e., 1G, 2/4M (depending on bitness), and 4K) and initializing them in SMMTT with the specified permission (disallow, rw, rx, rwx). The unit tests currently make use of MPXY to spin up kvm-unit-tests in a secondary domain. The primary domain then transfers control over to the secondary one via message passing. This was implemented as to facilitate testing of memory donation and mapping invalidation flows, but this was not completed during this mentorship project.

The secondary domain runs all of the tests. For each test region, kvm-unit-tests/riscv/smmtt.c will check to see whether permissions actually work as expected. It does so by first setting up exception handlers for load accesses, store accesses, execution accesses, and illegal instructions. Then, it will attempt to access each page of each region and measure whether that page is readable, writable, and executable. Finally, the code checks whether the measured permissions are the same as the expected permissions.

Architecture

This subsection documents various architectural choices made in this SMMTT implementation.

Table memory

Especially for 64-bit systems, several megabytes of memory are typically required for SMMTT tables. This is a larger amount of memory than what OpenSBI typically deal with, especially since it typically needs to be dynamically allocated. Therefore, this SMMTT implementation requires the presence of a reserved-memory node named smmtt-tables in the device tree that explicitly specifies a physical memory region which should contain the tables. The run.mk target generator handles this in the qemu-flags macro, but we typically require platforms to arrange for this node themselves.

Patches

Below is a summary of all the patches developed for this project, and their current state. For patches marked Pending, some more integration effort is needed to clean these up before upstreaming. All changes can be found in the submodules of this main project.

Project	Patch	Status
OpenSBI	lib: sbi: Heap improvements for SMMTT	Merged
	lib: utils: fdt_domain: Make opensbi-domain optional in CPU specification	Merged
	Prepare memregion handling for future isolation primitives	Awaiting review
	Implement SMSDID	Pending
	Add actual memory size to memregion rather than maximum	Pending
	Core SMMTT implementation	Pending
	Invalidation instruction	Not done yet
QEMU	Add support for generating OpenSBI domains in the device tree	Rejected
	Implement SMSDID	Pending
	Core SMMTT implementation	Pending
	Invalidation instruction	Not done yet
kvm-unit-tests	Correctly handle reserved memory	Pending

Specification Coverage

The following parts of the SMMTT spec are implemented:

Section	Name	Implemented	Notes
3	SMSDID	Partially	mmtp fully implemented
3.1	msdcfg	No (Not planned)
3.2	MFENCE.SPA	No (Not planned)
3.3	MINVAL.SPA	No (Planned)	This is the invalidation instruction described above
4	SMMTT	Yes
5, 6, 7, 8	-	No (Not planned)