Add "Dockerized" information on packages

[mattpolzin]

In the same vein as whether or not a package offers an executable or library target, knowing that it is dockerized could be quite helpful (a check for Dockerfile at the root of the package would usually be enough).

[daveverwer]

Thanks for this Matt!

So I'm quite new to using docker and actually I'm not even sure what a Dockerfile in the context of a package repository would mean. I'd love to be educated though!

Can you give me an example of a package that does what you're suggesting we monitor for here, which will allow me to think about how this information could be exposed on the package page.

[mattpolzin]

Can you give me an example of a package that does what you're suggesting we monitor for here

Absolutely! I’ll start by filling in some context you may already have.

Context

A “Dockerized” app is anything that has a Docker image available and can therefore be run using the Docker daemon on any platform for which a Docker daemon exists (notably, Mac OS, Windows, and a handful of Linux platforms). It’s a nice way to set an executable up to be easily run on any platform because technically it will always be running on the same platform everywhere.

A Dockerfile is the basis for such an executable because it is the roadmap Docker follows to create a “Docker image” that can be downloaded by the Docker daemon and run anywhere. Much of the time if you see a Dockerfile in the wild, the command docker build . (the dot really is part of the invocation) from the directory containing the Dockerfile will build the Docker image. In practice, there is more nuance to the build context, but I don’t think we need to worry about that here.

Finally, this means that if a Swift Package repository (really, a git repository of any kind) contains a Dockerfile (i.e. a file with the name “Dockerfile”) then it is a strong signal that there exists (hopefully documented, often the simple invocation I mentioned above) a way to build the package into a Docker image that can be downloaded and executed by Docker daemons everywhere.

As a corollary to all of this, it probably is only relevant to consider packages that have an executable target to be “Dockerized” but who am I to judge a non-executable package for having something I can’t think of to gain by offering up a Docker image of some kind?

Examples

My package OpenAPIDiff offers a library, an executable, and also a Docker image. In fact, the README explains usage of the executable via docker run because I have uploaded a build of the Docker image to Docker Hub and therefore the given docker run command will work from anywhere a Docker daemon has been installed as long as an internet connection is available to download the Docker image (no need to have Swift available, no need to build the Swift project).

swift-doc has a library, an executable, and a Dockerfile. It’s README does not explicitly call out using swift-doc via Docker, but if you know what to look for, you can see that (a) building and running the Docker image will execute the package’s executable (and by default it’ll print the help) and (b) the Docker image is the basis for swift-doc’s support of a GitHub action that builds documentation.

Final thoughts

These two examples are a pretty good representation because one of them explicitly calls out the Dockerized usage and the other does not. My gut is that much of the time the average person who has not used Docker for some other purpose will not really care whether a Swift package is Dockerized, but if you’re like me (and I’m not alone I feel confident saying), then you’ve built out a few docker stacks in your day and knowing that a Dockerfile exists means a particular package author has already done some of my work for me if I need to add their package’s executable to a stack I am building out or an automated process like a GitHub Action or other CI integration.

[Sherlouk]

Really appreciate the write up there, learned a fair bit! Are there any restrictions on what can be Dockerized, or is it a case that with the right motivation any package could be?

[mattpolzin]

Technically, yes, there are restrictions on what can be Dockerized. Practically speaking, though, I think the answer is “no.”

This is a bit of a potential rabbit hole so I’ll leave it at this for now:

Docker is a hypervisor layer over which a shared Linux kernel is used by all running Docker containers, each of which was started with a particular Docker image, and each of which has its own sandboxed environment based on any of the readily available and popularized Linux distributions. You can think of a Dockerfile as simply setting up a Linux environment and then providing Docker with “the thing that should be executed when the Linux environment has been set up.”

All of that is to say that if your goal is to set up a Linux environment on any of Docker’s supported platforms and then do anything (pretty much) then your application can be Dockerized. Because Swift is supported on several Linux distributions also supported as bases for Docker images, any Swift package supporting those Linux platforms has at least the opportunity to build itself and run itself in a Docker container (with instructions provided by a Dockerfile).

[finestructure]

One considerable restriction is that the package has to compile on Linux :) So in this regard "dockerised" is indeed a strong signal - but (at least for me) mainly because it means a package is Linux compatible!

[mattpolzin]

There’s actually a lot that the Swift Package Index could check on in this area, but I think the biggest return on investment is just checking whether a Dockerfile exists or not.

Other layers of granularity include but are not limited to:

• Checking whether the Dockerfile builds successfully (a bit hairy because some Dockerfiles might need to be run from a context other than the current directory or even run from a context provided by an accompanying docker-compose file).
• Checking whether the package’s Docker image is hosted on Docker Hub (not a requirement for being a useful Dockerfile, just a convenience in some cases, and also not something I am sure there is an obvious one size fits all answer for).
• Only considering packages with both an executable target and a Dockerfile to be Dockerized (in practice, probably a reasonable rule but I can imagine the possibility of a package with no executable target still finding utility in Docker for something).

[daveverwer]

This makes so much more sense in terms of executables rather than libraries :) When you originally suggested it I just couldn't understand what sense it would make for a library. Thank you for the extensive context Matt.

[finestructure]

I can see the appeal of having “dockerised” as an attribute but now that we’ve got Linux compatibility, I wonder if this even needed anymore. Any package that is Linux compatible can be dockerised.

On the other hand there might be packages that have a Dockerfile for test purposes but aren’t Linux compatible. This is probably rare, but still, there’s potential for mixed/conflicting signals.

I have to admit that personally I don’t find a package’s Dockerfile all that useful and in the past the only reason I looked at them was to judge Linux compatibility or to find out which OS level dependencies where needed.

@mattpolzin , would you agree that Linux compatibility is good enough or does “dockerisation” add much (anything?) beyond that for you?

[mattpolzin]

My experience has been pretty different, but admittedly that is more from the perspective of building out infrastructure than selecting packages to include as dependencies in my project.

To me, the question is whether you want the package index to be library-centric or to also be a source of information about the executables vended by Swift packages. Dave noted this important distinction in a comment above.

A package being compatible with Linux generally just means it builds for Linux. On the other hand, a package having a Dockerfile generally indicates to me that the author has written up a use case for the package as an executable because I've never seen a Dockerfile that just builds the package and calls it a day -- usually, it's going to run something core to the package's purpose as-a-service.

At this point it's worth briefly noting the difference I see between a package (a) being Linux compatible and having an executable target or (b) having a Dockerfile. It's subtle, but a package need not have an executable target to have a Dockerfile that uses the package and runs something useful (although I have to imagine that is an uncommon occurrence). So, more likely, a Dockerfile is in addition to an executable target and the difference is that a successfully building Dockerfile gives me the ability to use the executable anywhere without even knowing much about Docker as long as the README gives me a run command to execute.

Anyway, I think I have to admit at the end of the day that the value add here is limited and I agree probably smaller now that you build packages against Linux as part of compatibility checks already. Just offering a small reason why it does add a little something that might become more and more useful if Swift becomes more commonly used in server or utility settings.

[finestructure]

Ah yes, that makes sense to take executables into account. Would it be fair to say: Linux compatibility + executable target == dockerisable (if not outright dockerised)? You mention the case of a Dockerfile without an executable target being uncommon - but I can't actually come up with a use case for that. Perhaps as a base image for other images?

In any case, the main problem I see with taking Dockerfiles as an input is that unless we did some sort of inspection I think Linux compatibility + executable is a much more reliable signal. Maybe the way to go would be:

Dockerised == has Dockerfile && Linux compatible && executable target

because even if the Dockerfile has nothing to do with the executable, you're almost guaranteed to be able to create one that does.

As you say, that might become a more interesting piece of information as Linux/Server-side Swift gains traction.

[mattpolzin]

Personally I would steer away from calling something "Dockerized" or "Docker compatible" unless it does indeed have a Dockerfile. I think it's easy for any of us who have a lot of docker experience to see how to roll something new into a Dockerfile, but for someone with less exposure to Docker the gap between having a Package.swift with an executable target and having a functioning Docker image is much bigger.

Just speaking for myself, what I would find useful is a little lozenge indicating presence of a Dockerfile in the same repository as the package. As you note, the other important signals are already well covered (you already say if the package has an executable target and you already say what platforms it builds against). The presence or absence of something like a Docker whale icon would be great to let me know "this package's author has composed a Dockerfile for their package" which in turn tells me that I can at least see an example setup and maybe even find the prebuilt Docker image on Docker Hub.

[finestructure]

Yes, I meant to only show this if there is a Dockerfile. What I meant was that in the pathological case that the Dockerfile is actually unrelated to the package the signal isn't entirely wrong, because one could be created in principle.

An icon would be nice for that. Also, with the help of the .spi.yml file, we could authors allow to link to Dockerhub. I'd find that very useful.

[mattpolzin]

Actually, here's a slightly different perspective I just thought of.

If someone is very familiar with Docker, they are not going to confuse the absence of a Dockerfile with the inability to easily build out a Dockerfile of their own because a package is Linux compatible and has an executable.

On the other hand, if a person is not very familiar with Docker, the existence of a Dockerfile might be exactly what they need to be comfortable spending more time looking into a particular package because they are in early stages of experimentation and cannot spare the time to go down any rabbit holes (learning how to write a Dockerfile) at this point.

[mattpolzin]

we could authors allow to link to Dockerhub

This would be really great.