Pip does not work in poetry installation

[emilioag]

Description

I am having issues when I try to run pre-commit hook but I noticed that the issue is in pip, when I create a new poetry project, the pip installation is broken (note that the pip's system installation works fine):

<▸> ~/Devel poetry new test                       
Created package test in test
<▸> ~/Devel cd test          
<▸> ~/D/test poetry shell       
Creating virtualenv test-jDPINLwQ-py3.11 in /Users/e/Library/Caches/pypoetry/virtualenvs
Spawning shell within /Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11
<▸> ~/D/test emulate bash -c '. /Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/bin/activate'
(test-py3.11) <▸> ~/D/test pip         
Traceback (most recent call last):
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/bin/pip", line 5, in <module>
    from pip._internal.cli.main import main
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/cli/main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/cli/main_parser.py", line 9, in <module>
    from pip._internal.build_env import get_runnable_pip
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/build_env.py", line 19, in <module>
    from pip._internal.cli.spinners import open_spinner
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/cli/spinners.py", line 9, in <module>
    from pip._internal.utils.logging import get_indentation
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/utils/logging.py", line 13, in <module>
    from pip._vendor.rich.console import (
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_vendor/rich/console.py", line 47, in <module>
    from . import errors, themes
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_vendor/rich/themes.py", line 1, in <module>
    from .default_styles import DEFAULT_STYLES
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_vendor/rich/default_styles.py", line 3, in <module>
    from .style import Style
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_vendor/rich/style.py", line 8, in <module>
    from .color import Color, ColorParseError, ColorSystem, blend_rgb
  File "/Users/e/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_vendor/rich/color.py", line 11, in <module>
    from .terminal_theme import DEFAULT_TERMINAL_THEME
ModuleNotFoundError: No module named 'pip._vendor.rich.terminal_theme'

Workarounds

I have a workaround to fix the pip installation (upgrading it inside the venv) but pre-commit hook still fails because I don't have any way to update pip the poetry pe-commit hook.

Poetry Installation Method

install.python-poetry.org

Operating System

MacOS Sonoma 14.5

Poetry Version

Poetry (version 1.8.3)

Poetry Configuration

cache-dir = "/Users/e/Library/Caches/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /Users/e/Library/Caches/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true

Python Sysconfig

No response

Example pyproject.toml

No response

Poetry Runtime Logs

poetry -vvv run pip
Loading configuration file /Users/emilio/Library/Application Support/pypoetry/config.toml
Loading configuration file /Users/emilio/Library/Application Support/pypoetry/auth.toml
Using virtualenv: /Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/__main__.py", line 22, in <module>
    from pip._internal.cli.main import main as _main
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/cli/main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/cli/main_parser.py", line 9, in <module>
    from pip._internal.build_env import get_runnable_pip
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/build_env.py", line 19, in <module>
    from pip._internal.cli.spinners import open_spinner
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/cli/spinners.py", line 9, in <module>
    from pip._internal.utils.logging import get_indentation
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_internal/utils/logging.py", line 13, in <module>
    from pip._vendor.rich.console import (
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_vendor/rich/console.py", line 47, in <module>
    from . import errors, themes
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_vendor/rich/themes.py", line 1, in <module>
    from .default_styles import DEFAULT_STYLES
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_vendor/rich/default_styles.py", line 3, in <module>
    from .style import Style
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_vendor/rich/style.py", line 8, in <module>
    from .color import Color, ColorParseError, ColorSystem, blend_rgb
  File "/Users/emilio/Library/Caches/pypoetry/virtualenvs/test-jDPINLwQ-py3.11/lib/python3.11/site-packages/pip/_vendor/rich/color.py", line 11, in <module>
    from .terminal_theme import DEFAULT_TERMINAL_THEME
ModuleNotFoundError: No module named 'pip._vendor.rich.terminal_theme'

[dimbleby]

highly unlikely that poetry is at fault here, you'll likely have to figure out for yourself what's going wrong

to get you started, poetry creates environments just by calling into virtualenv. I doubt that virtualenv is at fault either, but that's where the action happens.

[emilioag]

Thank you for your answer @dimbleby. When I do it with virtualenv, it works fine:

<▸> /U/em/Devel mkdir test                      
<▸> /U/em/Devel cd test 
<▸> ~/De/test python3 -m venv venv-test     
<▸> ~/De/test source venv-test/bin/activate                                                                      
(venv-test) <▸> ~/De/test pip                      

Usage:   
  pip <command> [options]

[dimbleby]

venv is not virtualenv.

but either way: you're going to have to diagnose for yourself what is happening in a lot more detail than that if you want to make the case that poetry should be doing something different.

This works for more or less everyone, so much the most likely explanation is that there's something unusual on your system, and you are the only one who can figure out what that is

[larroy]

I have run into the same issue with nvidia-pyindex:

[[package]]
name = "nvidia-pyindex"
version = "1.0.9"
description = "A tool that adds the NVIDIA PIP Index to the environment."
optional = false
python-versions = "*"
files = [
    {file = "nvidia-pyindex-1.0.9.tar.gz", hash = "sha256:22300a8d5aff35b047830f790a737437402d96e0794037102e796edbceb2fec6"},
]

I think this should be the basis of a repro.

[dimbleby]

I think this should be the basis of a repro.

It is not. Perhaps you left this comment in the wrong issue, it seems to have no connection with the original report?

A dockerfile would be a good way to demonstrate a repro, if you have one.

[larroy]

@dimbleby installing this dependency caused a failure in pip similar to the one in this issue, once I have time I can try to make full reproduction.