Consider automating dependency PRs for dependencies with fast-moving major versions (e.g. Puppeteer)

[danyalaytekin]

Options:

1. Something like Renovate
2. Artisanal cron job
3. Don't do this

[aarongoldenthal]

@danyalaytekin I'm a huge fan of Renovate and use it in most projects, so happy to lend insights or help if you're looking for it.

[danyalaytekin]

Thanks @aarongoldenthal, might take you up on this offer! Let's see what the rest of the team thinks. On the security side I noticed you're using Socket on your fork - I liked the comments it was leaving. Do you have one example of a repo where you think Renovate is being used well, and for a limited set of dependencies? Only if you have time though. I spotted this list but haven't browsed yet.

[aarongoldenthal]

@danyalaytekin Most of my work is on GitLab, and I've evolved to use Renovate for as much as possible with a set of common Renovate config templates (and in most projects). By deafult it will try to do as much as it can, although it's a single config setting to reverse that and only list what it's allowed to do. And can do as little as keep a running issue with a list of available updates and not make any changes without being explicitly told to (the "dependency dashboard").

Maybe this is a helpful example for some capabilities that I think help simplify aspects of dependency management (and each of these could be implemented independently):

• This rule groups minor/patch devDependencies together in one update to help minimize the change traffic (since those things tend not to not interact). Any major devDependencies, or any dependencies are updated individually.
• This project uses playwright for testing, so this rule and this rule together group playwright npm package updates with playwright container image updates (used in GitLab CI) since they're coupled. Without that they're treated separately and both updates fail with a package/browser version mismatch.
• This rule re-creates the package-lock.json to update transitive dependencies once a month. Maybe more significantly, it shows how schedule can be used to limit when updates are made. Some people find it distracting to constantly get dependency updates, so you could limit to a specific day of the week, for example.

I'm a fan of Socket too, although it can be a lot of data. I'd really like to see a way to identify package capabilities that could be a risk, but are expected given the nature of the package (for example, puppeteer is expected to access the network, eslint is expected to access the filesystem). Just to help focus attention of areas of concern.

[hollsk]

Hi both! Fancy seeing you two here 🤔 ( 😆 )

I have no particular opinions on Renovate/Socket/Dependabot/whatever, but I do think that automating dependency updates is a Good Thing™. I actually had a very strange conversation when I came back to work after the new year with a colleague who was suggesting the same thing, and I was like, "... but we already have that?" VERY CONFUSING. But it turned out that we have Dependabot set up on Pa11y Dashboard, just not anywhere else, and I am not actually sure why that is. I don't think there's a good reason for us not to be automating this stuff, and we're obviously comfortable with it on Dashboard. I think we probably just forgot to enable it elsewhere.

So yeah, I'm personally happy with the idea, and I don't mind what solution we use. If you two both have strong opinions on Renovate or Socket (or whatever!) then I am as chill as an ocean breeze on what you prefer. My only caveat is that if it's not Dependabot then we'll need to retire Dependabot from Dashboard 😅

[danyalaytekin]

Hey both, thank you for thinking so much about this.

---

@aarongoldenthal

GitLab, I remember now, finding your Pa11y-related plugins there when I was first exploring this project's ecosystem. Those configs will come in really handy as a reference if we proceed, thanks. I'll review the repos themselves too for nudges towards other good practices.

identify package capabilities that could be a risk, but are expected given the nature of the package

I see what you mean. Nice to have all this info though.

---

@hollsk

Great, let's think some more about it. As for Dependabot I'll check the configs to make sure it's behaving itself, and I should have communicated this better, but I think we do now have Dependabot enabled on each of our pa11y* repos. The safe settings feature we were discussing might also help us make sure everything is configured correctly, and isn't modified by accident later. I suppose its current quietness is a good sign... @aarongoldenthal's upgrades have quenched many of the alerts/PRs. It does a good job of notifying us of advisories but I don't think it beeps for less urgent issues such as when a dependency's publisher simply marks a version as deprecated, e.g.

$ danyal, ~ > npm install -g pa11y@6
npm WARN deprecated [email protected]: < 21.8.0 is no longer supported

---

A few other (very) undeveloped thoughts relevant specifically to Puppeteer:

1. We could recommend a mechanism by which a user could override the installed version (ideally with a very simple syntax), with the understanding that such an override would not receive support until Pa11y catches up more formally.
   • perhaps using something like requireg
   • or the mechanism we were recommending to restore ubuntu > 20.04's ability to install/run the project, although that might have been more for Chromium than Puppeteer
   • this would be in addition to the choice of a Renovate-like tool; it would just allow users a way to unblock themselves during the hopefully short period of time it would take us to ship a new major version.
2. A return to pa11y@4's spirit, if not the method, of installing this kind of dependency alongside Pa11y, breaking the formal connection in package.json.
3. For projects like pa11y-ci, some inventiveness in package.json using Semver ranges, if we can guarantee that both pa11y's and pa11y-ci's dependencies will be hoisted into node_modules/*. But this wouldn't solve the problem for pa11y itself.