Using insta for schema generation and updates

[konstin]

In our projects, we're generating json schema output for our configuration. We want to check that this json is up-to-date when running tests, with an additional option to update it when it isn't - basically what assert_json_snapshot does with nicer UX. The only difference is that we need the output file to be put somewhere else, we call it {project}.schema.js and it lives in the project root with no header. Is supporting this kind of output in scope for insta?

[max-sixty]

I think this falls under the second item in #353 (comment) — i.e. a plain file with no metadata which is compared to the generated text

[mitsuhiko]

Would you want to "move" the entire snapshot or just also store it elsewhere in addition?

[konstin]

I'd move it to avoid duplicating the git diff

[mitsuhiko]

@konstin I have been thinking about this a bit. Does what I write in that comment make sense? #456 (comment)

[konstin]

I do like the flat macro we have now, but honestly happy about whatever lets me configure the filename while showing diffs in cargo test. Being able to create and pass around snapshots independently of the assert would be a cool feature though.

[xamgore]

Hi! Another use-case for insta. Let me show you.

It's quite convenient that the whole business logic between the server and a client is defined with GraphQL API. Thus, it makes sense to test it via calling GraphQL handlers one by one, driving the system to a point, where we can check the resulting state. How to check?

1. Ask the system to return some data and compare it with previously generated snapshot
2. Annotate handlers and fields by test matchers, like this:
   

I'm implementing a testing library for async-graphql. Each test case is defined by .graphql file. You copy-paste a test function in rust, and that's it.

➕ good meaningful snapshots' filenames
➕ panic error points to the test function
➖ have to keep the functions' names and scripts' names in sync
➖ have to create a rust function for each test scenario written in GraphQL, as we don't use loops (and loops are not really good friends with assert_json_snapshot)

#[tokio::test]
async fn test_something_is_done() {
    let api = Api::prepare().await; // run a database
    let gql = include_str!("./queries/test_something_is_done.graphql");
    let res = api.execute_script(gql).await;
    insta::assert_json_snapshot!(res);
}

So far so good. After 100+ tests in a file, there are certain downsides, like unreferenced snapshots, need for granular snapshots (per query/per mutation), etc.

That's why I'm in the middle of implementing the second bullet, where test matchers are, like @shouldBeNull or @shouldThrow. The most sophisticated are @shouldMatchSnapshot, @shouldThrowErrorMatchingSnapshot and @shouldMatchInlineSnapshot({ json }) (out of scope).

The idea is simple: let's take some piece of GraphQL query, annotate it with @shouldMatchSnapshot and somehow put it in a separate snapshot file. I'll write down some problems I've encountered while implementing it:

• insta has a way to assert_snapshot, though it prints to stderr and panics. As I write a testing library, I have to collect all those errors at the first step, run the following GraphQL queries at the second step, print beautiful errors at the last step. And only now we are free to panic.
• insta has a way to define a path and all those structures like ToolConfig and SnapshotAssertionContext through helpers. Insta's configuration is heavy, yeah, there are a lot of things one could tune. And as a developer of a testing library, I probably would like to keep those configuration powered. The more I reuse existing code the better. But it's not always possible, due to pub(crate) or not pub modifiers. That's why I've started to copy-paste some things from the crate.

Some nasty experimental code I had to write, which is ok for now

let mut settings = insta::Settings::clone_current();
settings.set_snapshot_path(format!("{}/src/__tests/snapshots", env!("CARGO_MANIFEST_DIR")));

settings.bind(|| {
// https://stackoverflow.com/a/54432441
let old_hook = std::panic::take_hook();
std::panic::set_hook(Box::new(|_info| {}));

let result = std::panic::catch_unwind(|| {
    insta::_macro_support::assert_snapshot(
        insta::_macro_support::AutoName.into(),
        &transform(&value),
        env!("CARGO_MANIFEST_DIR"),
        &function_name,
        module_path,
        &assertion_file,
        assertion_line as u32,
        &expr,
    )
    .unwrap()
});

std::panic::set_hook(old_hook);

match result {
    Ok(()) => Ok(()),
    Err(_) => {
        let err = String::from("Snapshot doesn't match");
        Err(MatcherError::new(_ctx.item.pos, err))
    }
}
})?

If you check Jest (js) source code, you may find that each function has a module. Like, you may take matchers, or snapshot generators, or runners, or smth else. Probably, if insta could become not only a helpful tool but a library — it would be perfect for creating own testing solutions.

Anyway, thanks for the hard work! 👍🏻

[max-sixty]

I think #610 can do a lot of this, because it can print plain snapshots with the extension we want.

It was (edit: wasn't!) exactly defined for this, so let's leave this open and see how it goes. It's still experimental, so we have the chance to make small changes if it would allow a wider scope.

In particular @xamgore — I thought your comment was insightful and clearly you've put a lot of thought into this. Would be interested in your feedback / whether #610 can fill most of those gaps.

[xamgore]

Hey, thanks for calling out 😌 I'll review the code this Sunday and come back with the feedback.