test_env_system_packages_are_relative_to_lib fails on some systems

[dimbleby]

Description

eg I am on ubuntu 24.04, and have a virtual environment derived from /usr/bin/python3.12, which in turns comes from the python3.12-minimal package:

$ dpkg -S /usr/bin/python3.12
python3.12-minimal: /usr/bin/python3.12

in that parent python:

>>> sysconfig.get_paths()
{'stdlib': '/usr/lib/python3.12', 'platstdlib': '/usr/lib/python3.12', 'purelib': '/usr/local/lib/python3.12/dist-packages', 'platlib': '/usr/local/lib/python3.12/dist-packages', 'include': '/usr/include/python3.12', 'platinclude': '/usr/include/python3.12', 'scripts': '/usr/local/bin', 'data': '/usr/local'}

and sure enough, /usr/local/lib/python3.12/dist-packages is an empty directory (I run a tight ship)

poetry unit tests assert that this is impossible:

$ pytest -k test_env_system_packages_are_relative_to_lib
...
=================================================== short test summary info ====================================================
FAILED tests/utils/env/test_env.py::test_env_system_packages_are_relative_to_lib[False] - Failed: No distributions found in the base environment of the virtualenv
FAILED tests/utils/env/test_env.py::test_env_system_packages_are_relative_to_lib[True] - Failed: No distributions found in the base environment of the virtualenv

Workarounds

I guess I could install something on my system just to keep the poetry unit tests happy

Poetry Installation Method

other

Operating System

Ubuntu 24.04

Poetry Version

1.8.4

Poetry Configuration

n/a

Python Sysconfig

No response

Example pyproject.toml

n/a

Poetry Runtime Logs

n/a

[abn]

Was discussing this internally on discord. The issue is the same on Fedora as well.

python -c 'import json; import sysconfig; print(json.dumps(sysconfig.get_paths()))' | jq
{
  "stdlib": "/usr/lib64/python3.13",
  "platstdlib": "/usr/local/lib64/python3.13",
  "purelib": "/usr/local/lib/python3.13/site-packages",
  "platlib": "/usr/local/lib64/python3.13/site-packages",
  "include": "/usr/include/python3.13",
  "platinclude": "/usr/include/python3.13",
  "scripts": "/usr/local/bin",
}

$ python -m site
sys.path = [
    '/tmp',
    '/usr/lib64/python313.zip',
    '/usr/lib64/python3.13',
    '/usr/lib64/python3.13/lib-dynload',
    '/home/abn/.local/lib/python3.13/site-packages',
    '/usr/lib64/python3.13/site-packages',
    '/usr/lib/python3.13/site-packages',
]
USER_BASE: '/home/abn/.local' (exists)
USER_SITE: '/home/abn/.local/lib/python3.13/site-packages' (exists)
ENABLE_USER_SITE: True

It seems that the get_paths() returns locations that are "writable" by non distribution package managers. Site paths correctly infers the correct paths. I have not checked if installing something to the /user/local prefix will magically make the path show up in python -m site.

[dimbleby]

it is not going to matter but for interest I have

$ python3 -m site
sys.path = [
    '/tmp',
    '/usr/lib/python312.zip',
    '/usr/lib/python3.12',
    '/usr/lib/python3.12/lib-dynload',
    '/home/dch/.local/lib/python3.12/site-packages',
    '/usr/local/lib/python3.12/dist-packages',
    '/usr/lib/python3/dist-packages',
]
USER_BASE: '/home/dch/.local' (exists)
USER_SITE: '/home/dch/.local/lib/python3.12/site-packages' (exists)
ENABLE_USER_SITE: True

which I think seems more consistent: first the user base directory, then /usr/local/lib, then /usr/lib. I expect, or anyway hope, that Fedora know what they are doing though.

Either way: I expect the assertion should simply be removed from the tests.

[abn]

The test failure is a symptom of a slightly larger issue I think. Specifically affecting cases Poetry tries to find what packages are already installed when system site packages are enabled in the virtual environment. Since Poetry does not find the package in what it considers the only paths to look for packages, it will try to install a package again which might lead to failure at build time.