Upgrade pantsbuild from 2.18 to 2.23.0.dev2

[cognifloyd]

The commits step through updating from 2.18 to 2.19 to 2.20.

Interesting new features in 2.19

• https://www.pantsbuild.org/blog/2024/01/24/pants-2-19
• https://github.com/pantsbuild/pants/blob/2.19.x/src/python/pants/notes/2.19.x.md

Pants can auto retry tests to deal with flaky tests in CI. I enabled that in pants.ci.toml so it retries a test up to 3x.

Interesting new features in 2.20

• https://www.pantsbuild.org/blog/2024/03/27/pants-2-20
• https://github.com/pantsbuild/pants/blob/2.20.x/src/python/pants/notes/2.20.x.md

Pants can now experimentally use GHA as a remote cache.

Pants has a pex_binary target for generating targets that run a particular entry point or script. Pants 2.20 gained a pex_binary(executable=...) field which would allow us to wrap our st2-* python scripts in a pex at some point. This feature means that pex will embed the script contents instead of attempting to import from a file that is not a valid package name (The "-" character is not valid). For now, I just left a comment in a BUILD file about this.
Note: I asked for this feature some time ago. It was first added to pex 2.1.93, and then to pants. Now its available.

Interesting new features in 2.21

• https://www.pantsbuild.org/blog/2024/05/29/pants-2-21
• https://github.com/pantsbuild/pants/blob/2.21.x/src/python/pants/notes/2.21.x.md

A new [export].py_hermetic_scripts feature allows us to use nosetest in the exported venv. This was added in:

• Add export option: py_hermetic_scripts pantsbuild/pants#20763
• https://www.pantsbuild.org/2.21/reference/goals/export#py_hermetic_scripts

This release also allows us to simplify our __defaults__ BUILD file config. Instead of specifying (python_test, python_tests), we can just use python_test because now applies to generated targets (python_tests generates python_test targets), not just the targets in BUILD files.

Interesting new features in 2.22

• https://github.com/pantsbuild/pants/blob/2.22.x/docs/notes/2.22.x.md

A new [export].py_generated_sources option. I don't think we really need this now, but it makes sense to make any generated sources available in the venv (for use by an IDE like VSCode).

• https://github.com/pantsbuild/pants/pull/20975/files
• https://www.pantsbuild.org/2.22/reference/goals/export#py_generated_sources

The options system is moving to the rust-based pants engine instead of in python to speed up cli args parsing.

Interesting new features in 2.23

• https://github.com/pantsbuild/pants/blob/main/docs/notes/2.23.x.md

I want to update to 2.23 ASAP, even if that means using a dev prerelease, to take advantage of the python_test(entry_point_dependencies={...}) field, added to resolve issues enabling our runner tests in #6202:

• Generate entry_points.txt for python_tests that require entry points from a python_distribution pantsbuild/pants#21062

pants-plugins lockfile diffs

From 2.18 to 2.19

Lockfile diff: lockfiles/pants-plugins.lock [pants-plugins]

==                    Upgraded dependencies                     ==

  pantsbuild-pants               2.18.3       -->   2.19.3rc1
  pantsbuild-pants-testutil      2.18.3       -->   2.19.3rc1
  pex                            2.1.137      -->   2.2.1
  pluggy                         1.4.0        -->   1.5.0
  psutil                         5.9.0        -->   5.9.8
  ujson                          5.9.0        -->   5.10.0

From 2.19 to 2.20

Lockfile diff: lockfiles/pants-plugins.lock [pants-plugins]

==                    Upgraded dependencies                     ==

  pantsbuild-pants               2.19.3rc1    -->   2.20.1
  pantsbuild-pants-testutil      2.19.3rc1    -->   2.20.1

From 2.20 to 2.21

Lockfile diff: lockfiles/pants-plugins.lock [pants-plugins]

==                    Upgraded dependencies                     ==

  ijson                          3.1.4        -->   3.2.3
  pantsbuild-pants               2.20.1       -->   2.21.0
  pantsbuild-pants-testutil      2.20.1       -->   2.21.0
  pex                            2.2.1        -->   2.3.1

From 2.21 to 2.22

Lockfile diff: lockfiles/pants-plugins.lock [pants-plugins]
                                                                  
==                    Upgraded dependencies                     ==

  pantsbuild-pants               2.21.0       -->   2.22.0rc0
  pantsbuild-pants-testutil      2.21.0       -->   2.22.0rc0

From 2.22 to 2.23

                                                                  
Lockfile diff: lockfiles/pants-plugins.lock [pants-plugins]
                                                                  
==                    Upgraded dependencies                     ==

  pantsbuild-pants               2.22.0rc0    -->   2.23.0.dev2
  pantsbuild-pants-testutil      2.22.0rc0    -->   2.23.0.dev2
  pex                            2.3.1        -->   2.3.3
  typing-extensions              4.3.0        -->   4.12.2