Remote caching issues beyond pants 2.19

[basvank]

Hi all,

First and foremost, thanks for all the hard work on maintaining pants! Me and my team are happy users of pants, and it has been making our lives a lot easier for well over a year now!

That being said, I do currently have a little issue when upgrading pants. I’m not sure this is an issue in pants itself, or I’m overlooking something maybe, so for now I’m starting this as a discussion.

The problem is with remote caching, we use a pretty straightforward setup for it:

• pants with remote_cache_read and remote_cache_write set to true
• A locally running process of bazel-remote (version 2.4.4).
• bazel-remote points to an S3 bucket on AWS

This setup has been running for well over a year without any major issues whatsoever.

However, now I’m trying to update pants, but then issues arise with the caching. It looks like the cache isn’t used properly, as when I run the same CICD job multiple times the runtime doesn’t go down. This is something that did use to happen before, and of course would be expected as pants should be able to skip tasks that were already run on previous runs.

The issue arises whenever I update pants beyond 2.19.1 (I’ve tried up until 2.22.0). I do see notes regarding the caching system in the release notes for 2.20. It doesn’t look like it says anything about the reapi provider, which AFAIK this setup is using.

So, my first question is if anything changed at this point that should be adjusted or configured in my setup to keep this working?

If that is not the case, I can provide pants stats output and other details to show the difference between 2.19 and other versions. If there’s other configs or logs that could help trace this issue, please let me know as well!