Incrementally upgrade edx-platform NPM dependencies

[jmbowman]

Many of the NPM packages used in edx-platform are very stale, and attempts to upgrade many of them at once have generally failed. Pick a tool(s) and process of incrementally upgrading subsets of the dependencies a little at a time, to see if we can gradually make the situation better. The general process should look something like:

1. Upgrade a bunch of packages to new versions that should be mostly harmless upgrades.
2. Test and deploy.
3. Do one major upgrade after reading the changelog and making any minor upgrades and code changes strictly necessary to make it work.
4. Test (maybe more carefully) and deploy.
5. Go back to step 1.

One tool that attempts to make such a project easier is npm-check-updates:

• https://www.npmjs.com/package/npm-check-updates
• https://chrispennington.blog/blog/how-to-update-npm-packages-safely-with-npm-check-updates/
• https://www.youtube.com/watch?v=0XQXGx3lLaU

npm-check is another option, there may be others as well. It's probably worth spending a few hours researching and comparing the options here.

Once a tool and process are finalized, start using it in edx-platform to update the package*.json files. Don't try to change too many things in one deployment, we don't want to make it too hard to track down what may have caused a bug that slips past tests to affect stage or prod.

If all goes reasonably well, document the process so we can repeat it in other repositories which are falling a little behind in their NPM package updates.

Related tasks

• Resolve npm dependency issues in edx-platform #26

[jmbowman]

This is listed as Blocked on the FED-BOM board; @ishahroz , what is it currently waiting on?

[abdullahwaheed]

we are trying to upgrade webpack from v2 to v4. Facing issues in underscore configs with this upgrade.
we are also trying to upgrade edx-ui-toolkit, which would result in a lot of security and linting issues resolution but faced some issues with this upgrade. Fixing those as well.
Also tried to upgrade karma, but its dependency is on webpack.
Some other package upgrades were also planned but majorly webpack becomes a blocker

[feanil]

Many major package updates were done as a part of the Node 18 upgrade and the package.json now compiles properly without issues thanks to some cleanup in the proctoring dependencies which were previously using npm shrinkwrap further updates will be coordinated via openedx/edx-platform#34900

See openedx/edx-platform#34873 and https://github.com/openedx/edx-platform/pulls?q=is%3Apr+author%3Aapp%2Frenovate+is%3Aclosed for some recent evidence of this.