How to correctly point to refs with JSON Schema emitter, when generated as single file.

[Devon-White]

I am currently attempting to generate a JSON schema from TypeSpec and then use said Schema with a Monaco editor for JSON validation.

I couldn't get the behavior of the schema validation to work properly when split into multiple files, so I started to generate the schemas into a single file. Upon doing this I noticed a couple of things.

• The generated file didn't seem to have a top definition, instead all definitions are put inside defs
• When pointing to other schema definitions when generated in a single file, it doesn't seem to properly append the entire schema path, leading it to definitions being not found.
  • Example - Notice that refs are not appending #/$defs/<definition>:

    {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "$id": "swml.schema.json",
    "$defs": {
        "NoArgSWMLMethod": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "NoArgSWMLMethod.json",
            "type": "string",
            "enum": [
                "answer",
                "hangup",
                "record",
                "stop_record_call",
                "denoise",
                "stop_denoise",
                "receive_fax",
                "return",
                "stop_tap"
            ]
        },
        "SWMLMethod": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "SWMLMethod.json",
            "anyOf": [
                {
                    "$ref": "Answer.json"
                },
                {
                    "$ref": "Denoise.json"
                },
                {
                    "$ref": "NoArgSWMLMethod.json"
                }
            ]
        },

I was hoping to be pointed in the right direction on how to handle these issues.

My current setup looks like such:

/typespec/swml/
│
├── main.tsp
│
└── methods/
    ├── main.tsp
    ├── answer/
    │   └── main.tsp
    └── denoise/
        └── main.tsp

---

typespec/swml/main.tsp

import "@typespec/json-schema";
import "./Methods";

using TypeSpec.JsonSchema;
using SWML.Methods;

@jsonSchema
namespace SWML {

  model Section {
    main: SWMLMethod[];
    ...Record<SWMLMethod[]>;


  }

  model SWMLObject {
    version: "1.0.0";
    sections: Section
  }
}

---

typespec/swml/Methods/main.tsp

import "@typespec/json-schema";
import "./answer";
import "./denoise";

using TypeSpec.JsonSchema;
using SWML.Methods;

@jsonSchema
namespace SWML.Methods {

    // Union type for methods that do not take any arguments
    enum NoArgSWMLMethod {
        "answer",
        "hangup",
        "record",
        "stop_record_call",
        "denoise",
        "stop_denoise",
        "receive_fax",
        "return",
        "stop_tap";
    }

    @jsonSchema
    union SWMLMethod {
        Answer,
        Denoise,
        NoArgSWMLMethod

    }
}

---

typespec/swml/Methods/answer/main.tsp

import "@typespec/json-schema";

using TypeSpec.JsonSchema;


model Answer {
    @doc("Answers an incoming call.")
    answer: {
        @doc("Maximum time in seconds to wait for an answer. Cannot be less than `7` seconds. Defaults to 14400 seconds.")
        @minValue(7)
        max_duration?: integer;

        @doc("Comma-seperated string of codecs to offer.\n Valid codecs are: [`PCMU,PCMA,G722,G729,AMR-WB,OPUS,VP8,H264`]")
        codecs?: string;
    }
}

---

typespec/swml/methods/denoise

model Denoise {
    denoise: {}
}

The entire JSON schema that is generated from the above setup:

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "$id": "swml.schema.json",
    "$defs": {
        "NoArgSWMLMethod": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "NoArgSWMLMethod.json",
            "type": "string",
            "enum": [
                "answer",
                "hangup",
                "record",
                "stop_record_call",
                "denoise",
                "stop_denoise",
                "receive_fax",
                "return",
                "stop_tap"
            ]
        },
        "SWMLMethod": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "SWMLMethod.json",
            "anyOf": [
                {
                    "$ref": "Answer.json"
                },
                {
                    "$ref": "Denoise.json"
                },
                {
                    "$ref": "NoArgSWMLMethod.json"
                }
            ]
        },
        "Answer": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "Answer.json",
            "type": "object",
            "properties": {
                "answer": {
                    "type": "object",
                    "properties": {
                        "max_duration": {
                            "type": "integer",
                            "minimum": 7,
                            "description": "Maximum time in seconds to wait for an answer. Cannot be less than `7` seconds. Defaults to 14400 seconds."
                        },
                        "codecs": {
                            "type": "string",
                            "description": "Comma-seperated string of codecs to offer.\n Valid codecs are: [`PCMU,PCMA,G722,G729,AMR-WB,OPUS,VP8,H264`]"
                        }
                    },
                    "description": "Answers an incoming call."
                }
            },
            "required": [
                "answer"
            ]
        },
        "Denoise": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "Denoise.json",
            "type": "object",
            "properties": {
                "denoise": {
                    "type": "object",
                    "properties": {}
                }
            },
            "required": [
                "denoise"
            ]
        },
        "Section": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "Section.json",
            "type": "object",
            "properties": {
                "main": {
                    "type": "array",
                    "items": {
                        "$ref": "SWMLMethod.json"
                    }
                }
            },
            "required": [
                "main"
            ],
            "additionalProperties": {
                "type": "array",
                "items": {
                    "$ref": "SWMLMethod.json"
                }
            }
        },
        "SWMLObject": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "SWMLObject.json",
            "type": "object",
            "properties": {
                "version": {
                    "type": "string",
                    "const": "1.0.0"
                },
                "sections": {
                    "$ref": "Section.json"
                }
            },
            "required": [
                "version",
                "sections"
            ]
        }
    }
}

[bterlson]

The root cause here is that our emitter is producing JSON Schema 2020-12 and VSCode supports (I think) draft 7 or so? I am guessing you are emitting a single file via the bundleId option, which produces a bundle as specified in 2020-12 and which unfortunately VSCode doesn't support.

One thing that might work - if you have a root object for your schemas, you can put the @jsonSchema decorator directly on that root object, and then any references to other schemas will be inlined and use a ref format that should be compatible. So it would look something this.

[Devon-White]

The root cause here is that our emitter is producing JSON Schema 2020-12 and VSCode supports (I think) draft 7 or so? I am guessing you are emitting a single file via the bundleId option, which produces a bundle as specified in 2020-12 and which unfortunately VSCode doesn't support.

One thing that might work - if you have a root object for your schemas, you can put the @jsonSchema decorator directly on that root object, and then any references to other schemas will be inlined and use a ref format that should be compatible. So it would look something this.

Appreciate the advise!

I went ahead and followed these instructions. I did the following with the above TS setup:

• removed all namespace
• declared @jsonSchema on the root object
• removed emitAllRefs from config file

Upon doing this, I noticed i was having an issue still. I investigated the generated schema file and noticed that two defs were defined. One at the root and one underneath Section.

Upon removing the bundleId this behavior stopped, and I was able to generate the schema as intended.

I imagine this is how bundleId was meant to be used, so certain schemas are bundled under certain defs.

Appreciate your help on this! :)

[drpump]

I am guessing you are emitting a single file via the bundleId option, which produces a bundle as specified in 2020-12 and which unfortunately VSCode doesn't support.

AJV (JSON schema validator) also doesn’t accept the emitter generated bundle (TypeSpec 0.5.9). I’ve pasted example tsp, generated bundle and AJV compile error below. It chokes on the references, trying to resolve them to files. I’ve tried hacking the ids, refs and property names, but the only thing I could get to work was explicit fragment refs (e.g. ”$ref”: “#/$defs/Foo”).

Just generally, I’ve lots of trouble with explicit $id and $ref resolution with JSON schema. Not sure if there’s a problem with the JSON schema bundling spec interpretation of relative reference resolution in RFC-3986, or just that the tooling is lagging behind.

My current workaround is to write anonymous schemas (no $Id) with relative file or absolute URL cross-references in $ref properties, then apply the bundle function of the json-ref-parser tool to package in a bundle. The tool replaces bundled schema cross references with fragment references (e.g. #/$defs/Foo). It would be nice if the emitter had an option to generate something similar. Perhaps a fragmentRefs or safeRefs option.

Note that anonymous schemas also resolve lots of versioning issues that can arise from using an explicit $Id. Generally we want to package our schemas and publish with a package version number. Injecting versioning into the $Id fields when packaging is problematic a best, and not doing it leads to ambiguity. Versioning at the individual schema level is unmaintainable.

My sample TypeSpec code that demonstrates the issue with the emitter:

import "@typespec/json-schema";
using JsonSchema;

namespace example.bundle;

@jsonSchema
model Bundle {
  foo: Foo;
  bar: Bar;
}

model Foo {
  foo: integer;
}

model Bar {
  bar: string;
}

Generated bundle:

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "$id": "bundle-schema.json",
    "$defs": {
        "Bundle": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "Bundle.json",
            "type": "object",
            "properties": {
                "foo": {
                    "$ref": "Foo.json"
                },
                "bar": {
                    "$ref": "Bar.json"
                }
            },
            "required": [
                "foo",
                "bar"
            ]
        },
        "Foo": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "Foo.json",
            "type": "object",
            "properties": {
                "foo": {
                    "type": "integer"
                }
            },
            "required": [
                "foo"
            ]
        },
        "Bar": {
            "$schema": "https://json-schema.org/draft/2020-12/schema",
            "$id": "Bar.json",
            "type": "object",
            "properties": {
                "bar": {
                    "type": "string"
                }
            },
            "required": [
                "bar"
            ]
        }
    }
}

AJV compilation error:

    JSONParserError: Error opening file "{cwd}/schemas/test/Foo.json"

Replace {cwd} with your home dir.

[drpump]

Just briefly, I also tested the simple example published on the JSON schema site. Same error.