Error pulling gcr.io/go-containerregistry/crane:debug

[baby-gnu]

Hello.

I just setup the docker-registry-proxy which is working fine in most case but I have a strange issue about invalid URL prefix

docker-registry-proxy logs for `docker pull gcr.io/go-containerregistry/crane:debug`

{"access_time":"25/Oct/2021:07:17:01 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/","request_type":"unknown","status":"401","bytes_sent":"84","upstream_response_time":"0.076","host":"gcr.io","proxy_host":"gcr.io","upstream":"74.125.133.82:443"}
{"access_time":"25/Oct/2021:07:17:01 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/token","request_type":"unknown","status":"200","bytes_sent":"217","upstream_response_time":"0.476","host":"gcr.io","proxy_host":"gcr.io","upstream":"74.125.133.82:443"}
{"access_time":"25/Oct/2021:07:17:02 +0000","upstream_cache_status":"MISS","method":"HEAD","uri":"/v2/go-containerregistry/crane/manifests/debug","request_type":"manifest-default-disabled","status":"200","bytes_sent":"0","upstream_response_time":"0.592","host":"gcr.io","proxy_host":"gcr.io","upstream":"74.125.133.82:443"}
{"access_time":"25/Oct/2021:07:17:02 +0000","upstream_cache_status":"HIT","method":"GET","uri":"/v2/go-containerregistry/crane/manifests/sha256:f3aa77677f5f9caacba1c06dc97675db4bcc2377d461c0af91253fbb8a9bc9cd","request_type":"manifest-by-digest","status":"200","bytes_sent":"1170","upstream_response_time":"","host":"gcr.io","proxy_host":"gcr.io","upstream":""}
{"access_time":"25/Oct/2021:07:17:02 +0000","upstream_cache_status":"HIT","method":"GET","uri":"/v2/go-containerregistry/crane/manifests/sha256:c29f254715ed816109c9af5136928844ecd97c8d0d2d4b1d06c92c61deb57e4a","request_type":"manifest-by-digest","status":"200","bytes_sent":"1077","upstream_response_time":"","host":"gcr.io","proxy_host":"gcr.io","upstream":""}
2021/10/25 07:17:02 [error] 77#77: *96 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE6Cay47uOgUfzUR4GT9y3MmCzNsc3SJ1Yb0cbR8dvR4JJ7-Npci9-iBtY9HYN0KW2nkQPlpV3-pSQ_7vyuEezXj_BJWknPuUL7-2CeCsCOZUgqtcDJ2ZoGyuo3K_E9pjjX6ygqZTHYST9yeFnhREShZfkxxEobMfAAIV1b14M3lTf4FLUe2hZ0cmPwi3KOjuCwDZwQttHdVIBNir8WXsMPnNV2qddCe0fuYPKMh-vg-ClkKkMnFETJ8fmmXRrq8K1tk1_BTO4xSf85V-BfykpVOIBkAYjPKlyAVJUhEPt-lIm5Lo5w7zqhLxlI-G4x32rJRkOHCFXVQzjwWPmYHEhQm8pgwukY__Uz9QMSfyvYIbhpf2Ce4_QejyfGmlpBWXl_5cNEu6FaOWZNOnphkQlwOjbwCRT1CK-6QPlQBQ6DtLuvSsX2WZv4F6sNGvoty-TZXHM8jHwyQHeaOCDSUk6unJ7Glf9O84XYoT3nVo1zqQOJNdiWZySgqp2GB9LRMQGbpvUHwAbYscKhMxSP-HhsadAodrSm8tBjowt1BxPK0hn561qX3amCXaDLfzDewV-M=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:ec52731e927332d44613a9b1d70e396792d20a50bccfa06332a371e1c68d7785 HTTP/1.1", host: "gcr.io"
{"access_time":"25/Oct/2021:07:17:02 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:ec52731e927332d44613a9b1d70e396792d20a50bccfa06332a371e1c68d7785","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.248","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}
{"access_time":"25/Oct/2021:07:17:02 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:601401253d0aac2bc95cccea668761a6e69216468809d1cee837b2e8b398e241","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.492","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}
2021/10/25 07:17:02 [error] 76#76: *95 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE6p31rh8B1vJAxngAWPK54PwDiTdSlYSbkMoEZkn_hTlwBHaNm6dp0oMBQsSSJGd2yLEBn2rlfT4GHEZc9voo6Eyy1BDSpKLhZvBlFaL3DvBp6DkpQQZhhpJxvCDJBegrjdxPkE95N3-TruZ8BBTlCP2xcw6vKaC30byNJoWnthyEWGnaNBL3wv4ibznyQEs2n1RoCAUZH8zVumr111yly6oTe5ZE8gJrKPFit_qKiv8YvY4UlCqE7X8SMpkUf3AlXGBDtDlwVqzWiCTM1P__MRB_0_Vc19K8jEBo1MPvVg9tVKH-QP2vdieQBx15006LUZsx6PnXGNQT1hm21uGeGf_sVm_ykdFJ6_j0x0Oti6tH32isoEW4eluulkRgdEQcz-I241XQzA6AwmxTHj6RpyEIgAATT_s7HvUonfQSWQ8ZasjcNXouHQGPAZogsso7lyBISNXEiG7qA0RVZVrhS8Lci-BRA4n-ySfuf8HQ27hyZq5X9YJMt1Tq11IEKLrYkU1mGbKwLfZVdYjEgHl_r8OjXKiF8IXVepYoOBm0Vnk8qT_OFL3z0TKx7aOKwm1Is=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:601401253d0aac2bc95cccea668761a6e69216468809d1cee837b2e8b398e241 HTTP/1.1", host: "gcr.io"
2021/10/25 07:17:02 [error] 77#77: *94 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE4iqD2PQajhcW93R8vg3jiO7Y4E-u2ty9-8pFcfcIbj1chGNvaTMsABFpdPeG-_fU2Y_qWZqmJgJhcrLvIxXmBQLzbDFQQgMTSZueQlhwTTvRNjvaM4GT33uozlCrhF_eTSsPRU1KEwPxU0gJRrYtHyr9BRpcUJdzdxE9mbNynWELflKQ73YRhtNdlynMIGewLtcE7SpE1Wv1FimJJHIWTqR7bw4JtezK8uAt6zLsV1YKd6IfBN9HEsS_Kru7Tl621FbOG9g_hArsKjBuh7FDN8VApAPMLQav7Occb_my-CxaRHdZi23tHifzEOxdRCXEd3bw52EMI4HUOJ64Hi4U1Ahcr0yqiCPSp_0d71KN0Mz1c-tvaHr7zURiD-p2qDjYRhFJmc3nw8iAFsTIGTnHfYtMndMJV3gmfhD_N31VEXtqTVCjqIdjgblTZUIZKzSSEypHcxEMiWAob_6Vf2QXdWM6Tj1CMSXO4dLzoq_ehdJEx98BKwjv9BfFQngPvCPZjzg6RnE7Cmu0XvqMccPWjUKYePjQOHtXYwwnb3wri2rO615fMuG6EHznbYkjZYgOU=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:8907fc4ab049835499c850563c3d7d474f580f63056f02ca32063c8777d79ac5 HTTP/1.1", host: "gcr.io"
{"access_time":"25/Oct/2021:07:17:02 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:8907fc4ab049835499c850563c3d7d474f580f63056f02ca32063c8777d79ac5","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.500","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}
{"access_time":"25/Oct/2021:07:17:02 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:165636f582ea4490cd4f9dcfa9038f6c0244e506124c1176528dfb6673ba44db","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.520","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}
2021/10/25 07:17:02 [error] 76#76: *93 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE48Uxn6JD9_lgRGRAoAuGvv8YxsK_A8gMQEQ95DjI2An0RXkXJDmy4DBrY1NESCjeoxaSBJ80qgvDlZiOdNCnWkxdiHjuxY9W4gXPmoF_JseQl8gvap7I1ZSr6iRqXvRadBctuhoKmG7rFQg1lYGhsPMueAvkMp2f2z0oUN1k0oRY6eDGimAo4rUsEyt9aXvbHhpsVKeblZrHwVwEck01JejPaU1eWajJXKrq7eT-ZfaFNkXQybWl7zc9xiKj9Kk9JkTEhh0buIeeowcspM7G_2PeDV1R7oJLu7BfK0LIeBgfI_kMlPWCeYsNX5qoC4JP65P2Q7BMPEiHVf3g5dnaLbqn0klLyourN3TDt1x_DEedK0Zwe74QhYWWwFhUAiTJO-sWXhCw6jkyIg3ujEyJUJ2d0jPJKWdbP2Me0fndmPBvJWEbDX3RNrDDE7q09Nrf8nKgYnk5g6JT5KA8EMU98DiE3JEqtQUWBeZ80yVX60uufd-3txYXQmFOjkXw0yWJy-k0FbHVBGR7or8TmUWx0UFz-ApeEZRL_YBeDz9o2FGe5vl0tVUsVOo1bJadxNI2g=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:165636f582ea4490cd4f9dcfa9038f6c0244e506124c1176528dfb6673ba44db HTTP/1.1", host: "gcr.io"
2021/10/25 07:17:02 [error] 75#75: *107 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE4nIaSwvZ3ziYOEcE3ChLP9tCSnH0lTR1evmCHS70OVZSCzLz8kEpcrP-7X_Ynqvbi3_o4o7smoFjMB3SEc2Hzif43RdqXULFra2nv_wQpb-siOG-90YYAMTA8jnsLifEWGFXDJbJfCVRX16ZOWZNdSMt-LFTS9r_74Wydi2Ny420PteCJAgpiujg82bU1LFVYTZFVhsdO4_TsnviSpoqe-6U3nmZndrHS6AZS-i5gyKMjEDDZaWa9Gsw8eoi1zSqcm8VvSXRfroK0HE60LkoQ_uUnVvvDzZAuTeSk9N5VlHQVddHwAkN4mjxHeoTT1A0CfRbsZilW7-0B25qyH0mGGuuKx2r3zMAEdM3txcKguXkBRIEKZGhSYHe24wUIZr825IQJgWUCLcJwMoZJp8SJ83p1YdLCR_QH10ciDr1AYEvqmNLkzTuFj4lr2ynpzPSNyM8luqQPyiUH2nfThNBDZQ8OuznkshNTdMsF8YNZ21uTWwrdukZBYW4h9_41m3JkMKBRGp4UggQ_LG7sQzS51o_5TOD0F6tCb8-Ey4Dkx1jZNDOaYxHm9b1A2j2FhWGs=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:601401253d0aac2bc95cccea668761a6e69216468809d1cee837b2e8b398e241 HTTP/1.1", host: "gcr.io"
2021/10/25 07:17:02 [error] 74#74: *118 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE46uwtTDyRuNIrosnxhEyjMHElNshRL3v5hM9-AUWkfVAZeecDPIx-ZhdcYPEdefSqkpscJQRv6byn889fC3qr-w1c3A5h643aVwZ1Pvcs0YeWBwziuAeHtjBLXKNQPi5wNvXz6_vWzqrFhN9Lf__CDuTnJsRzHd-DyAdmP4HPuK3-fYYVeDqDMAl192BQMttmokDllKYtlaHgFU3ivuKTrvnJ3QZuupNOLudMgSpAwgipSv_9jNXb3pNn-CGCcPPpxw2JCy_OrhL6W8JiamFzdza6t_0EjN2iuYWuVO4fICs1K9nPvkrT0ZyW4rl8fmiRKUgasT8H1RAxJfgaTrQs4v418UyQ-3HosurfR64g0rc6-9LQ0OxxzCfSIaA5ge_ZR3drIbOOiaU5EhwwHrHSlWeU4nrC6qBzYtZMPw3mUGnvzrl5F8yILmeiZhX9uukVI9Z9w4ClQc9U4xeCCxHyZhj_M_Xyydxpf6i_MNKiU0rQFjeevlkqbcoOHbNyMh9addikh88jMZqSLr0jLRHIxZ9lzsr4sK_Q7m4Dgx93U54Cw_3Kohoa8ySk68-A6zlU=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:72164b581b02b1eb297b403bcc8fc1bfa245cb52e103a3a525a0835a58ff58e2 HTTP/1.1", host: "gcr.io"
{"access_time":"25/Oct/2021:07:17:02 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:72164b581b02b1eb297b403bcc8fc1bfa245cb52e103a3a525a0835a58ff58e2","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.216","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}
{"access_time":"25/Oct/2021:07:17:02 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:601401253d0aac2bc95cccea668761a6e69216468809d1cee837b2e8b398e241","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.248","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}
2021/10/25 07:17:03 [error] 75#75: *103 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE7kpornunc46Kiczng0NGqNkUNUFWiNN4IQYgUUGErDJ0i5iz48SIOSkaTneWzTc4MbdtMJ4yZt0N9rWUY7W0Mlk2a8nwlmlVy5dqv4SkrpU83ypFJB2fWK_XFnGqcZftRBIGWn9ObiE9giHIymgENz_0Ng5NeZHC95OqXfR0A5h7mRT9zuV5n8ZtJmJBc4lh9mMvpcgMB8NoEesTy4JLqAW0LF7-DzqrBiN6Odq9om3SgQrPVGfedUIbUPLVPtxi8f1NyRikLpoMpp0c8VyM8w6eFyn4YbQ2W0g2NducYXDXFbORjXt1e8sxUEzuFWAI_M7WAIXN0aYnrd8I3UrpArlHMH0fcJYe1oHmgG50_ikmUy1YRBzJqU0aqo-p6H3DqvAanvBSKInkNryfZJ_sTVFzpE5pawxaueYqRqYMfhSMc5gZQc7MVS8m0hjsMQh_tCeUYeKNtDux8kdS-A2g47Wr5WZVCi7UtXcPI9mZeM7wkMGTxhd8vRLzodbGn_wP4r_FgoLpUzlebKPmrCxRCRkatXmv8RF0eFNB36ClWgQq_0gWwvcoQRtLx_czGTE4U=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:ec52731e927332d44613a9b1d70e396792d20a50bccfa06332a371e1c68d7785 HTTP/1.1", host: "gcr.io"
{"access_time":"25/Oct/2021:07:17:03 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:ec52731e927332d44613a9b1d70e396792d20a50bccfa06332a371e1c68d7785","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.512","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}
2021/10/25 07:17:03 [error] 75#75: *111 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE6E-pA7w09k0zQuVJrP_QhnzeT9v5I5ffQkgloDmymwpdVvzRZknZQ0UeDXJ3tqUsUOjjeQ0FP_BzEs1NKnNGIuLAn7jyiJchws2TfNO_r1ZIX-trK6OvMBlzJjPGslZXJp0onFRHBqkWQLyTiJ0y0hTtbprvYaiGwq-MXXy1rZWx2buvyvIa8LYSM74EcesppF0Qvti6cqpDg3Frj_th2gSyBEmEo1hYzl7VKOTcMuNxSx1LvzcJDtfPBxHwQjyN-Nu48qDwUAnfpWvFiMqRTi-N87E7uZ0MfjRP9q60pngOP_BIu78RlklKrG6O5_rIR30_K-X8LD69UIu55gCmCAwKSUZBLE8o7HmJlMvwbnzjU47-RypgMNAMJbfh5uw_MfXERGSsbKeJKW31nGvfoshA5syDADvKfmn_FTFmmUXJNXKAKLLJl5C4K5KqCIHotPJfrcwpFYDrdqW7gvtgIG56WiI6Wm1k7nMaGoHSG71e9OdSI6uY1FDixf_MxOOGEcQc51xjmRagoQ0eC_1xwr9sXtjmjZCh6eY2AhwgI435IhK-LiUd-YO-_H_z7T3lE=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:8907fc4ab049835499c850563c3d7d474f580f63056f02ca32063c8777d79ac5 HTTP/1.1", host: "gcr.io"
{"access_time":"25/Oct/2021:07:17:03 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:8907fc4ab049835499c850563c3d7d474f580f63056f02ca32063c8777d79ac5","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.520","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}
2021/10/25 07:17:03 [error] 74#74: *117 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE7ylbQixYR8hLm8QAdEMbnzttfpBQ4ZmGSTwbZ5xBFfd_yIiAxUG0w09iD1goOBMmjT-Vujcot-lW1fGqfZLpln55WwC1UL85FRgUNmCbt_812MvMiQdyC08h1LOwwe_tW6OC1OYsOQq1qJAMR8XECm1HtcQQfFUJvOjsz0yhoVGks54ibmw5i58njht08L-M230PbqzthxfGQ8d9PzMT3krxxpcoUZiVikzvrIWPMr7p4KtwMvArkK96GYhNY2sVKrZBdvqFcC7gNEtd9L3uhOhJ6lYlsChinpvxk1NQdbBYXNqjSPbaOJJdF8rhgEAsZM87MsGs3BM8Y5MyB8jMc2KF_PQsqiv6AdlToKsFMTvXCIpU-s2SIHLVmGITM24DOvkoBCcU91Q6hneGL3keIbNb48qE9f_rzX8NHPbig_9dB33xYmKOl6uJ2iOpenJHxd6V_T2JrKRykiICrqct9lEoAnAxdM_erbRGyLm2YsEaNrzUOLB5ndeswfG8pCQY5epx5vNco1qwY38O8aWtWTCcn4ZkrRhFEvQuQc_ezUsCz5_I6Lfwzlgg0kkj7Dw1Y=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:465b1109f79df7779c7af5a9cfa09412b956e827767b1b5e0c9577adab79db21 HTTP/1.1", host: "gcr.io"
{"access_time":"25/Oct/2021:07:17:03 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:465b1109f79df7779c7af5a9cfa09412b956e827767b1b5e0c9577adab79db21","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.516","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}
2021/10/25 07:17:03 [error] 75#75: *123 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE5M2_msSO_JBF3tkgtmetnzKCAlgKZmOqn0P4m-DUNJAhxVHqunRc03ooU78iBMVwgt-pMNgt3d67K4X6Xrd5IQoseD-LxK3I25jSMQkVtQQyfDiBohkb0E5bqWSMoMYhUGaTUv1Ulk-hWaLeSWZUq5_VNkxvtnOOaRDu8UQTgYy1729mYI0SNgZDDs-03ZP_NUloebSGmdQTQHp_BhhyJI2lFvurV6D6QxkfPgV783UeD0etIM7AwzRCQ1hlFkC5lcvRQWA8SU0reUKsbh4ag2I5JcI9rQJIoQdnCAXFniOA0Bo5yooz_s8UKTcDRSCh7UqCUPj-JIebQ2ccubP4uDyaqhV2MBLNHJREdr0K-TnkBYBbYfIT50PThsEYirtTwXMf6Vwq9EgBGBHeLMne5EfnlVsyn1BW1Y5nN2J-QTvsaZX17kF4YfEs4EXqDPLAyZ3oRc3IG4whVJ6aB_P5qUwVk18Z4N-iAYO-rLes3-D3sOTUKvitU5fB5rIyaMkF3E4LMpJgTv5GUT3gLkB4BkvCGNYoNbFrJ84RrT8w7kzr14xkMzhZGH6IGKMHH9JvE=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:72164b581b02b1eb297b403bcc8fc1bfa245cb52e103a3a525a0835a58ff58e2 HTTP/1.1", host: "gcr.io"
{"access_time":"25/Oct/2021:07:17:03 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:72164b581b02b1eb297b403bcc8fc1bfa245cb52e103a3a525a0835a58ff58e2","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.528","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}
2021/10/25 07:17:03 [error] 75#75: *127 invalid URL prefix in "/artifacts-downloads/namespaces/go-containerregistry/repositories/gcr.io/downloads/AHf3wE5dw8uD9XA6Kl0OSZOe_INuQE-Z9zyHts47uG14-LVOhkww6hsUgChGKENwDnGbMxUGYCdM2mM7kB0wcg0n_FNd3T6PxfM33PTWtZ0f4NOp3WQxt52SXcdJHyfTlytlI5b6yTsIrkDtBSFS5ebQ7erULeNKxQ_NLTrsYrrFWvSWTiezDvJR1POw1AFCatTacDYls8deOSp7Kna0Z1ngXns04Gh_IGKFEh0JT0PFDqXAU-VEVrTbHSeqmEPGyOhbjJC_-Pz-GFrpSPZkrfGvVVGca6EGrnUrn0HT1Aq1WyANhYi3u3vaICaTeC9EVTv8H-X4imWm4FNty6PzQuF6XdHAemQdUtIgocBzyuSBtSMh1MqarG-me1nmGcFbQyuC8xue6LmSY44cbVYBGyRLPMOmkTeATrtmQr1zGWJ0DRqkMMoQYLKmU9qxHPiqP3jG2oaZXuiBg619KNgDOSXBPJAcAAVWq6ALAN-wq2DDYmzH_OoBnrv7cpouGUFnOhJ4zJbU480DOxCsCmEfwaCvyZ7y-ibGYQUF5BYqkS440sfTjOmv-AkHiV8t-Vg5-kgV_ls=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/go-containerregistry/crane/blobs/sha256:465b1109f79df7779c7af5a9cfa09412b956e827767b1b5e0c9577adab79db21 HTTP/1.1", host: "gcr.io"
{"access_time":"25/Oct/2021:07:17:03 +0000","upstream_cache_status":"","method":"GET","uri":"/v2/go-containerregistry/crane/blobs/sha256:465b1109f79df7779c7af5a9cfa09412b956e827767b1b5e0c9577adab79db21","request_type":"blob-by-digest","status":"500","bytes_sent":"177","upstream_response_time":"0.520","host":"gcr.io","proxy_host":"","upstream":"74.125.133.82:443"}

It's working fine with gcr.io/kaniko-project/executor:debug

[rpardini]

Hey @baby-gnu thanks for the report. I've still to find time to work on gcr.io stuff.
You could try running -debug version (some docs on main README) and try to capture the traffic using mitmweb, that will probably shine some light here.

[rblaine95]

Just to toss in some 2c here.

Having this same issue, but only when running in Github Actions, and with istio pilot:

Error: 2024/09/13 06:36:37 [error] 90#90: *17 invalid URL prefix in "/artifacts-downloads/namespaces/istio-release/repositories/gcr.io/downloads/AC5eRcKSGK1GA3YECUlcwDNMfoxmMIENRd2RuY3s-PxKPn3ZwmJCKDflHa1dwGS8d9802rUQPLQtTrke-cD2gmJcn_K0hGKyUeRzgBL8acE-CmL8tzaBCnruPwLxF_U9dx4MRTH__uGBUkcMupmwP3Y5Us3rLKddsGUHNPudZr3bNIgOfZh4BzBRS09exXL8gXKUEzuKrZLzemwh0ReuuVr5sEbX1D9FHa-pxPWFBmY0NYF8pKmbqqCuFl3rl2Io0gKe30XtQKtT0P-KcwT7AvNSpXBPlIMOY13nAgR6IwMo_tlzI-GWT8RIwcsEN6fj4e--of-269QPl0sn3QyyRyWrWu0UsVJ2fovOo8lR6SBYpdAehGSddo_vD2jHy0Q_Ka7c35vYm3GqlxfC9Jykj_i9DbmrqpFOgOM4vGMji33aatUAgyBSB_gIo1mtG_p2I2McSN1ZwnCLz55EBOL78-RnCrsY2v_4SsvVBa1OKGEQ5HiUKJM1ZQ1022nDa5aleGnb5g1PM2qYl4bc_fCwhMkkDlm0drWmn-4YdVAeci1YN2Gafxs_L8x_Qe022OiVgKvgUB09k0NoJCcezK5GFOf7auYjnPjgJuKkaXMnwBSODdI2ZUaOwWPlZATzCJKNNh-o9ZoWeQEVe3kPnQ__cZOWA9icq-IsH6izk3ipzIpPvB8yYYIfwS-dklycHg_DsfR2pa_cJv36Q92g7N163MGJl32utUi8u9QxKePPN4VM8RHe9fbsntg0UIgNuUYRe3wfBuzn6931cxUJxg0a17Y=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/istio-release/pilot/blobs/sha256:57a80614796a1a123387536370f16c8e1b4d50a04cf2fce70af51df078ec1447 HTTP/1.1", host: "gcr.io"
Error: 2024/09/13 06:36:53 [error] 90#90: *89 invalid URL prefix in "/artifacts-downloads/namespaces/istio-release/repositories/gcr.io/downloads/AC5eRcJ1pBPc5esyjJmfaP0773LHnIwEaQbyp9qF1L2iqZGVu-zvSNbhPpxnS49AIokI6zEQSeUhpnAlx02_lxSy2RADEUMId5O6N6g4GTMLy9u1mZSW2A6seXo1p2a9PSGja8Ips-KqWPiVcmbUv_7wt-qPPmuSBiGoE31olmNibqcclgDf28SqrZO8nLDBUFi09U02Qwk8Loa-2paGFCPV11KPWqwwyuHJHfQ4vTQ23-g89KFtafICtMyzMhqVor_jacqPMJ6D1jgVFu5jPlRRMM0RyxFs-EPJSHmT95IwWNq8rQerMWM5qShgp_YtotY1nJv1mFbZvtVDQwyx0v9V2oXQfzArBoGJF5maa7jv8mGy5AP5msvBycJVESYrMyOzecmXXu01CM-FM8_vQHaXzbzjyjQ3s7mw-lgBb9CuBehDnlkrvObRSf4y3AxDx63eAM3fy57W4xNQEKD8l6yd1shgragbWe2rkpClCimg0tbbLcDnhlKhyRy1dh7ry5F5fkCO9vx9VT5JBKgSh9ueq9CoOorTtUPkpDJI4-MhIFNo83zcAylGOBt3MbsRMXPTVt2VMqrb-z7YMtPtMJDNr0Q7qhgD_p14aCb7-jgnuwaO2u6UEAUjBfth6uFHy8zzhhccokSR-nNPGAMEsKL2DKyZdJtE3HG5AivL74J_KIPf1N9PL7Vfe1fKvV7TW18nKMNTh5MMP53jq5aLZQRlunnCxJVEHsQ9iagaVUqD42PkKM8eA9An2DiwkCMYsbjW7oUlwu3RtxvEnJpuobs=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/istio-release/pilot/blobs/sha256:57a80614796a1a123387536370f16c8e1b4d50a04cf2fce70af51df078ec1447 HTTP/1.1", host: "gcr.io"
Error: 2024/09/13 06:37:18 [error] 88#88: *102 invalid URL prefix in "/artifacts-downloads/namespaces/istio-release/repositories/gcr.io/downloads/AC5eRcLJH9dtF2SONMVywGeo8WHjG0yTZxwz4KC0aCT2HXuN8hEAGILfGRMPy7VpMdmEqJ1day7uYDMemzDrfEE58mAMebaWZ0QKq8yjbeJQ2Tt1imYE[30](https://github.com/didx-xyz/aries-cloudapi-python/actions/runs/10844197999/job/30092813449#step:16:31)dOc3u8jw9kp-NtupcE-mUrnLgUTf4K4d6zaNFOON0Pzmqz1sL6nhvuBJ5ZxNnqkTBiIAAVNrVumCFQ1EbGE6Zq-4zUvGfy_gTnfs_jmj0Gaoarw7DMU3iFSK8UCwkszf7p8PXD1FqgJ_js6knpIDrDUN8PIeR6McJafGGNLbhYuafIbCItzb2bwz5DKbsjjMkHWYeHMIa3984y3HtWm2C9PJCEHLd9WMbjPLmkyozNogEABWyKnGDzJVwKZCnE3EAfDnlZHd1Qku_EYqjy-cYt_xntTGz9VvccrpiPejO4_jfUKxNWhyjGtJ0PkA5IQPQzs56oW5Jy2Xv5FxMpoq5fq_1gVYkOLMVFqipS6Fhjvo7WtsGHfHafi_7dG0kpch1nd_bR0__6tCVKTlsuVc-2CBMErQmoZyq88G1XjUMIR6KEU6oSou9YM7ttM3XNxFY7JLrWMkKqfaVBisv1xuX90trty37PapnCmhw_-t_jP1Gf4Dpc-taHusgyaMw8OwhemFjITD5En_D9qTxxzA6ZZ0zmSPvJT-td8G5r72CjM1nOX9zn4x7YWBzqlXY3xUuBltNYkWkZ6p_OI276IyNwvrwOtjEWlg7Gf2tK6ZeL5U10g-Wl96IiDAx3AkQy9jl6Lt_SfnwpiM3wuPbWEr-juXQtEh-bdDk=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/istio-release/pilot/blobs/sha256:57a80614796a1a123387536370f16c8e1b4d50a04cf2fce70af51df078ec1447 HTTP/1.1", host: "gcr.io"
Error: 2024/09/13 06:38:08 [error] 88#88: *109 invalid URL prefix in "/artifacts-downloads/namespaces/istio-release/repositories/gcr.io/downloads/AC5eRcJIFL0k6BuFmrJa6fry1TbaJk3XM4kXUaegiKGHI72krSmEKFBnFDwXuZbZRz6_Xs8HgySXm7rJGKlQZGy8G3eszmhTET8pSsuFdC3XkUguEkKcQHAVohBTaeYL-6PHFDTeAp0AT6kAC-Zva4Vs1Wi9s-lExQbr3g4RqEw8N7H6yShY-fKyDtHSGtiPACCo1GxWVD70NecHohaKXC2CE5rel91z_4bxVk3tEgYCFjtdT-HV00vQ8cURXkHDeKRmXiXVFIHSXX6WfMCc2gNrryVioaODaCEharg7bt_YKI5t_ZazcBIWF3Jqg2SxaUZ86SshBQ-QNiPAGgX84_wrXfiYTFSxXknMG8AvxS4f9zRhxtmTybwZEMBVZV8kKRaHR-gP3jl375tZ3dfZLlvqXuxWCQxoX8v3AtQER49PvEKDq-RpkyfYisCr2K1zda-xw-B3U5Kjfke3GDftM_UXs0Pu0XgBlx2dlITPqGt5x9uBjkFR6TLMoMCwO3CjeCOWwQGkJ15uscBBsKa5LBW8VAjXQspxx5HRV5vB7GgG47XqihiQrpK1KS8xUsc_XGlh63F9M7WqkUx249zEvSQxdglHSKHy2rRRdHGADGXYkJ_wS-r1xZywFIXtxMzirFOFdXZQGZcDCOrBNtoGiNRMksV5Y6j49EEpepwbS2Gp1XZWlG8fbjhWDR7mFO8LkYydSxaWLz1bmSZT6_fHtw8vTvXrhcGxONTS8HOklz9BO0HB6VxhYhYz-ANHzYWj8a1eTfxLVx2lG4AUYwW3APE=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/istio-release/pilot/blobs/sha256:57a80614796a1a123387536370f16c8e1b4d50a04cf2fce70af51df078ec1447 HTTP/1.1", host: "gcr.io"
Error: 2024/09/13 06:39:39 [error] 90#90: *116 invalid URL prefix in "/artifacts-downloads/namespaces/istio-release/repositories/gcr.io/downloads/AC5eRcJGJ1LiqHBh3TxejiGsW4XQywtkXWhHz-lcbxDEw1EayjMEUblaCUQRCIwVLOL2V1DjjLaeFoa_SfjJ0eWpUNxbDirSQ5AzItNLEfwDMnkTyuo-dveQxl0wAc7Jmy2RRTMd6aVl20ypKZzHYAYgLMCGioUKUI0M1ir3rNBflEn8LLJzQMXsZ06BYQCIyDGsgvvn2JkQBSTCiNFCN4GYa_AmmmXE_JLV_sPK2qdOzeDH3QghLH2UuUjysO9eefNkkpIJPZ_cs1ZNnTArCcV9kQHF3trram7dx4RLkwsX7YOzlvaNrKM-BvXzzl-OwJs4IdGAXBG_jvRsINc_WQ9UHWPutkBzY7CgpzZlvS8rRBxY1IZbVxMARkp393PJyA47NWGGasHjUFRuiMscz63eKlNUVK_H7AlAYkVMc3gWQbxOIxI591pt-mBIdFtUEMFVBHndJVOFE8mNcMqNhZriHoSMebsxwe1JwQ1e9fbuaPPCqZEGpLnGmWpI5mjMTvIhtRO2ki9YfN_NSU7_LNa_xX9bMNmDfCBcif5QEkvfuXvup9QK1Aqz7Y8k5wmvV3YRMDCfVLaDfH7-t63oqlxx0kxZt2vMn1RUxISFTXRpWVq8dNLK_Zp5MVNWqQif_JxoydoQefHlVYPetb5vcj-FJpwsyTYV6EjP_AN-Lpt0zhgb1H9TNeUv1ZAZH8MHDTdmnIKZBsSq_d4p5jrTLsXx-rtlK_veXwqH_sf5KJ6dZbH3po5WHIg16Epw4PdWxFKGJiOf6G3v8goP3wssZvk=" while sending to client, client: 127.0.0.1, server: proxy_caching_, request: "GET /v2/istio-release/pilot/blobs/sha256:57a80614796a1a123387536370f16c8e1b4d50a04cf2fce70af51df078ec1447 HTTP/1.1", host: "gcr.io"

Running on my local machine (M2 Macbook Pro), caching works as expected, but as of 11 September 2024 (when I first picked up on this) gcr.io is suddenly not working anymore.

Edit:
This is the docker run command I use:

mkdir -p $(pwd)/docker/cache $(pwd)/docker/certs

echo "resolver 1.1.1.1 ipv6=off;" > $(pwd)/docker/resolvers.conf

docker run -d --name cache-docker \
    -p 0.0.0.0:3128:3128 \
    -e ENABLE_MANIFEST_CACHE=true \
    -e REGISTRIES="quay.io ghcr.io registry.k8s.io gcr.io" \
    --hostname cache-docker \
    -v $(pwd)/docker/cache:/docker_mirror_cache \
    -v $(pwd)/docker/certs:/ca \
    -v $(pwd)/docker/resolvers.conf:/etc/nginx/resolvers.conf \
    ghcr.io/rpardini/docker-registry-proxy:0.6.4-debug