Support for Recording Thumbnails/Preview Images

[jacobagilbert]

Summary

It is often useful to have a quick preview of the contents of a recording such as a spectrogram of the data or other graphical representation of what is contained within a dataset. Including explicit support for this within SigMF would enable users to more effectively manage navigating their data. This originated from discussion in #221.

Implementation Details

This feature MUST be optional. It is ideal that there be some flexibility in the size of the spectrogram so that sufficient content can be included, though at the same time this is intended to be a preview type feature and should be limited on some level to prevent misuse. This feature SHOULD NOT serve as an alternate data storage mechanism. Its also possible that we should support a fixed icon-size image (128x128?) and a larger but bounded preview image. Thinking out loud here...

There are a few ways I see this data could be included, and it is possible we could support more than one though we need to be careful about complexity:

• As a separate file (PNG? JPG? should probably specify one for the sake of preventing this from introducing lots of complexity). This is nice since its readily viewed by any tool, but becomes yet another file that needs to be shipped around. Using TAR archives helps but makes it difficult to use external tools to quickly view data.
• Encoded directly into the metadata file (e.g.: base64). This has the advantage of ensuring it is always conveyed with the metadata, but some obvious disadvantages as its going to introduce a bunch of junk into the JSON.
• Others?

Other Comments

It would be desirable for this to also include tooling to work through a set of files and generate spectrograms (or PSD) icons for datasets.

[Teque5]

If a thumbnail was included, it seems like it should either be alongside the data like how I do this:

• whatever.sigmf-data
• whatever.sigmf-meta
• whatever.webp (512x512) lossy image

Or it could be kept in the sigmf-data file and we could use the header_bytes field to ignore that portion of the header. This is how it is done with many audio formats. id3 tags support dozens of pngs embedded in the audio file as header data.

[777arc]

My opinion is that when it comes to managing a large number of sigmf recordings, you are going to want a separate piece of software anyway, like a web or desktop app tailored to sigmf, could be a simple table or a maps-based interface who knows. But that tool is what should create the spectrogram thumbnails imo. If you only have a handful of sigmf files, just open them in Inspectrum or GNU Radio, and nothing is stopping someone from adding a myrecording.webp with a spectrogram to recordings they share with others. The question is, what is the benefit in standardizing that .webp file, and I think the kind of software that would benefit from having it be a standard format is also the kind that should just create it on its own. People just downloading a dataset and looking at the spectrograms on their desktop don't need them to be a standard name/size/etc they aren't going to care, it's the code that reads in those files that would benefit, but it's only another dozen lines for that code to create the spectrogram itself. That way it can make decisions like whether it's a spectrogram of the whole file or just the first million samples, the aspect ratio to use, the resolution, etc.

That being said, I wouldn't be against the specs having a very simple rule saying "if a .webp or .png of the same base filename is found in the same directory, it is assumed to be a spectrogram thumbnail with frequency on the horizontal axis and time on the vertical, associated with that recording", simply because, what else would it be? PSD maybe, but spectrogram includes PSD so its more useful anyway, at least as a thumbnail.

[jacobagilbert]

This is a very reasonable take Marc, appreciate the input.