Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What is the best test image for integration testing? #1236

Open
armintaenzertng opened this issue Jul 10, 2023 · 4 comments
Open

What is the best test image for integration testing? #1236

armintaenzertng opened this issue Jul 10, 2023 · 4 comments

Comments

@armintaenzertng
Copy link

As I'm currently updating the SPDX output of tern, I noticed that this part of the code does not seem to be tested (please correct me if I'm wrong and overlooked something).
To make sure that the old and new SPDX generators actually produce the same output, I'd like to include at least an integration test for the workflow starting from an Image and resulting in an SPDX JSON file.

Now, what would be a good test image for this endeavor? Are there any existing test images that I could use?
The SPDX output I get with tern report -x scancode -i golang:1.12-alpine is 120k lines long, which sounds like overkill, but I'm not quite sure which parts of the image could be dropped while maintaining full test coverage, or if this actually covers everything.

@rnjudge
Copy link
Contributor

rnjudge commented Jul 10, 2023

@armintaenzertng For a general image that is subject to change like any other image, there's photon:3.0 which is a VMware minimal, single layer image. We also host a docker image in the "vmware" namespace that we use in tests (vmware/tern@sha256:20b32a9a20752aa1ad7582c667704fda9f004cc4bfd8601fac7f2656c7567bb4) that is just tern packaged as a container which would not change since this image is not updated. Note that there is some long term stability concerns around the tern test image as discussed here.

@armintaenzertng
Copy link
Author

Thanks for the pointers, @rnjudge! :)
Unfortunately, neither of these examples results in any files in the output SPDX, so we would have a blind spot there if we used them for testing. Maybe a slightly more complex example would be better?

@rnjudge
Copy link
Contributor

rnjudge commented Jul 11, 2023

File information will only happen if you use Scancode. No file output with Scancode for those images?

@armintaenzertng
Copy link
Author

I used tern report -x scancode -i photon:3.0 -f spdxjson -o photon_test.json.
No file information to be found in photon_test.json.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants