Replies: 5 comments 4 replies
-
|
@mtrmac PTAL |
Beta Was this translation helpful? Give feedback.
-
|
I found a solution. Read containers/image#1413 it seems that the underlying library is modifying the digests. If you use |
Beta Was this translation helpful? Give feedback.
-
Yes, but it will also push uncompressed data, creating a much larger image and affecting consumers. (But then again the compression is CPU and time-intensive, so in some situations that might be the right trade-off.) So that’s not how it is intended to work (also we don’t generally want mandatory
Looking at the logs, it has not, AFAICS, uploaded anything at all: it’s all just HEAD requests to confirm that the data is already present on the destination (which must be confirmed because many registries reject manifests referring to locally-nonexistent data), and a final PUT of the manifest to upload the metadata (and update the tag); that PUT is basically exactly as costly as a GET to realize that w don’t need to issue a PUT. All the copy-avoiding optimizations seem to run pretty much perfectly. So, yes, if there are 6 layers, there are going to be 6*N HTTP round-trips. That’s not obviously avoidable. There might well be some efficiency improvements here, e.g.
So far, most of the effort has been spent on avoiding actual copies, because they are quite a few orders of magnitude slower than these small-scale metadata-only requests. I.e. a second here or there has typically been completely lost in the time necessary to upload any changed layer. I suppose we might be getting in a place where the big things have actually been solved and we now can/should sweat out the small sub-second times — that’s a good news, in some sense. |
Beta Was this translation helpful? Give feedback.
-
Hi, that's not correct. I see some |
Beta Was this translation helpful? Give feedback.
-
|
No. There should be two HEAD checks per layer, one with the uncompressed digest, with “not present”, and one with the cached compressed value, finding a match. None of the logs contain any attempts to actually upload the data. The 11 high-level “Copying blob” progress logs match the 11 “already exists” notes. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, good to know. Why is Isn't it possible to check the HEAD check in parallel to the build process? Wouldn't it makes sense to have a combined command like |
Beta Was this translation helpful? Give feedback.
-
|
I guess it’s only because it is doing one less HEAD roundtrip per layer. The build/push code is basically not at all tuned for hundreds-of-millisecond-scale efficiencies/inefficiencies. And the complexity of combining build (commit?) and push, as proposed, along with the 6 parallel registry clients, is, at a first-glance, completely unviable. Getting the cache logic (that saves dozens of seconds/minutes) correct and manageable is hard enough already. |
Beta Was this translation helpful? Give feedback.
-
|
BTW why does all of this matter, at a high level? What’s the purpose of building+pushing exactly the same image, and caring how fast it takes? Not building an image is always going to be faster than building an image. |
Beta Was this translation helpful? Give feedback.
-
|
We have a pretty simple dockerfile and utilize the container storage cache and blob info cache to speed up builds. I recognized on subsequent "cached builds" that pushing is slower than building the image. This seems pretty inefficient, especially if you don't have ideal network conditions. We spent 2-4s just for "checking". |
Beta Was this translation helpful? Give feedback.
-
Hi, I experienced an incorrect behavior of
buildah push. All my docker instructions have been cached, but on every run, the podman wants to push something to the registry. That process, depending on the network takes several seconds. This feels not correct.Here are the debug logs of
podman pushIf you have any ideas, please share your thoughts. I can provide more information when necessary.
Beta Was this translation helpful? Give feedback.
All reactions