OMMX Artifact is a specification for packaging and distributing OMMX Messages and associated metadata over the OCI Artifact defined in OCI Image specification 1.1.0 and OCI Distribution specification 1.1.0.
The problem that metadata should solve is the reference between data.
-
Even if the schema of the data is defined, the moment it is saved to a file or DB and separated from the application, the relationship between the data is lost.
- For example, within the solver process, it is clear which Instance the Solution is for, but after saving to a file or DB, it is no longer apparent.
-
Wouldn't it be better to always save the Solution and Instance together in a file?
- This simply increases the amount of data to be saved. Considering cases where many Solutions are generated by changing parameters for a single Instance, saving all Instances is inefficient. Moreover, as the amount of data increases, the overhead of exchanging that data also becomes significant.
-
Therefore, a reference mechanism is introduced.
- Reference refers to sufficient information to obtain the data even without the actual data.
- If the data is not shared between machines, the path in the file system can serve as a reference, but if it is shared over the Internet, a URI is needed. Introducing a reference function is closely tied to how to retain and obtain the data.
OMMX Artifact is an OCI Artifact of media type application/org.ommx.v1.artifact
.
OCI Artifact is represented as an OCI Image manifest.
OMMX Artifact is a collection of config
, layers
, and annotations.
config
is a JSON blob with the following media types:application/org.ommx.v1.config+json
layers
consists of the following blobs:application/org.ommx.v1.solution
blob with the following annotations:org.ommx.v1.solution.instance
: (digest) The corresponding instance of the solutionorg.ommx.v1.solution.solver
: (JSON) The solver information which generated the solution as a JSONorg.ommx.v1.solution.parameters
: (JSON) Solver parameters used to generate the solution as a JSONorg.ommx.v1.solution.start
: (RFC3339) The start time of the solution as a RFC3339 stringorg.ommx.v1.solution.end
: (RFC3339) The end time of the solution as a RFC3339 string
application/org.ommx.v1.instance
blob with the following annotations:org.ommx.v1.instance.title
: (Free string) The title of this instanceorg.ommx.v1.instance.created
: (RFC3339) When this instance was created
application/vnd.numpy
: NumPy's ndarray with NPY formatapplication/vnd.apache.parquet
: DataFrame with Parquet format- And other blobs with appropriate media types. The media type SHOULD be registered in the IANA media type registry.
- Users can add arbitrary annotation to arbitrary layer.
org.ommx.user.
prefix is reserved for user-defined annotations.
- Users can add arbitrary annotation to arbitrary layer.
- Annotations in manifest:
- TBA
Note that other annotations listed above are also allowed.
The key may not start with org.ommx.v1.
, but must be a valid reverse domain name as specified by OCI specification.