Clarify what makes AAP API and app structure requirements for interoperability

[atsushieno]

It is a draft of a new documentation.

---

For an audio plugin format, it seems obvious that the format should be built on API without implementation. AAP of course wants to be in that party. In AAP it was never regarded as possible. We need an implementation of Service to be bound, and therefore org.androidaudioplugin.AudioPluginService has to exist anyways.

However, as we could build AAP hosts and plugins in different versions and they could still be queried and instantiated whilst the actual implementations can be different, there are rooms for interoperability between different implementations. It is also expected, taken as diverged "AAP ecosystems" in different package namespaces. Therefore in theory, it should be possible!

By defining the API and the normative Android application structure, some fundamental changes could be made possible without breaking interoperability, for example:

• Rewrite the overall core library using LV2 (directly) or CLAP libraries and helpers
• Make some room for WASI based implementation (DSP in wasm + Web UI + WASI everything else = everything in web technology! We can go ahead of NDK)

In other words, we can "layerize" current state for swift migration across AAP ecosystems.

Throughout this documentation, we will figure out how we improve interoperability.

AAP Protocol: the basic principles

This applies to the AAP protocols of at least V3 or any later versions, until the principle itself is revised.

As long as the same protocol version is used, a host and a plugin should be interoperable. The interoperability is "considered" at release time, not at every commit.

At this state, we do not really have any verification tools that assure interoperability across versions, so things could happen and there could be interoperability bugs. They would be either fixed in later versions, or given up. To maximize interoperability we might keep using the same protocol version across multiple AAP implementation versions whilst such an issue exists (then the interoperability expectation will be back in the next protocol version bump).

AudioPluginService, AIDL, and aap_metadata.xml

The most strongly binding component is org.androidaudioplugin.AudioPluginService with some suffixed versioned name (such as org.androidaudioplugin.AudioPluginService.V3#Plugins) and its related AIDL. They must be queryable as Android Services.

By defining that, the actual procedures that make use of this AIDL also have to be uniform across implementations, and allowed to expand only in backward-compatible manner (e.g. add optional steps between existing steps).

The same goes to the content for aap_metadata.xml in theory. It can be extended in backward-compatible manner.

the AAP protocol in AIDL

AAP AIDL defines a handful of methods that build up a set of commands. Some commands consist of more than one methods in the AIDL. Either of the host or the client holds the connection status, and the available commands are restricted to some subset. The following list describes it:

State	transition	avaiable commands
INITIAL	the initial state	"instantiation"
UNPREPARED	after "instantiation"	"extension", "preparation"
INACTIVE	after "preparation" or "deactivation"	"activation", "extension", "offline rendering", "termination"
ACTIVE	after "activation"	"deactivation", "realtime processing", "termination"
TERMINATED	after "termination"	(it is the last state)

Some extensions such as port-config must be performed at UNPREPARED state (each extension defines when it can be performed).

Each "command" is bound to the methods in AAP AIDL protocol as follows:

Command	AAP AIDL method invocations
"instantiation"	one beginCreate(), zero or more addExtension(), one endCreate()
"extension"	extension()
"preparation"	one beginPrepare(), zero or more prepareMemory(), one endPrepare()
"activation"	activate()
"deactivation"	deactivate()
"offline rendering"	process()
"realtime processing"	process()
"termination"	destroy()

NOTE: Current implementation accidentally accepts extension() calls at ACTIVE state, but it should be allowed only at INACTIVE state. At ACTIVE state, extension messaging should be performed as in UMP system exclusive messages.

(Port connection can be established only once; Host application connects plugin channels, but that happens within the host application itself. Connection between host and plugin is not affected. There will be extraneous connections, but it that won't hurt much.)

Extensibility

Both hosts and plugins make use of extensions. They do not directly interact with each other but they will have to work with AAPXS.

It is ultimately up to each developer of host or plugin that which AAPXS implementations they integrate (if they have choices). For interoperability, we only care about them at the protocol level, where only (1) extension() and/or MIDI 2.0 Universal SysEx and (2) the packet format (defined in each AAPXS service) matter.

Extensions should be provided in C API/ABI.

(The way how each AAPXS is implemented, and whether they are independent of the rest of the core API or not, do not really matter at this protocol level, as it is not about host/plugin interoperability.)

Miscellaneous rules that applies to the protocol level

• AAP client host will have to be able to handle multiple services and multiple instances of same or different plugins for each service.
• AAP plugin service will have to be able to serve mutliple plugins, and be able to handle connections from multiple clients that could request multiple instances of the same or different plugins.
• There is only one event input port, and there is only one event output port. Both flows MIDI 2.0 UMP stream.
• Every plugin has to parse MIDI 2.0 input at process() function and dispatch those extension()-equivalent Universal SysEx messages.
• Since V3, process() takes the actual frames in the audio buffer to process.
• Instance ID must be unique within a live process. Since it is possible that multiple hosts connect to the same AudioPluginService, the instance ID should be unique across the connections.
• Plugins should report any parameter changes that occurred on the plugin back to the host unless the change is sent by the UI. It may contain parameter changes triggered by the non-UI part of the host as well, but that should be distinct.

aap_metadata.xml and GUI: inquiry and instancing

An experiment is ongoing around GUI support. The details are described as a documentation, but in short regarding this interop context, it has to be specified on aap_metadata.xml and can instantiate either an in-host-process Web UI, or an in-plugin-process native UI through GUI extension to control the UI (create/show/hide/etc.).

The Web UI needs to interoperate with host, and therefore its protocol should be regarded as part of interoperability promises and needs clear documentation on how the host's JavaScript interface object (currently AAPInterop) is defined.

(Other than that there is no specification on interoperability. The host is responsible to send whatever user operated on the UI to the plugin and reflect MIDI outputs from the plugin to the UI i.e. everything happens within the host. And on native UI, everything happens within the plugin itself.)

Since GUI support is at very early stage, we will keep bringing in breaking changes for a while.