poetry init Unable to Find Package During Interactive Dependency Addition

[hakuna-max]

Description

When using the poetry init command to create a new project, the interactive dependency addition step fails to find packages (e.g., numpy). However, the poetry add command works as expected for the same package. The issue appears to be related to how poetry init handles package search and HTTP requests.

Workarounds

After completing the poetry init process, dependencies can be added using the poetry add command. For example:poetry add numpy. This bypasses the issue with the interactive dependency search step.

Poetry Installation Method

install.python-poetry.org

Operating System

Windows 11 24H2

Poetry Version

1.8.4

Poetry Configuration

cache-dir = "C:\\Users\\rolfz\\AppData\\Local\\pypoetry\\Cache"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}\\virtualenvs"  # C:\Users\rolfz\AppData\Local\pypoetry\Cache\virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true

Python Sysconfig

Platform: "win-amd64"
Python version: "3.12"
Current installation scheme: "nt"

Paths:
        data = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0"
        include = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0\Include"
        platinclude = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0\Include"
        platlib = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0\Lib\site-packages"
        platstdlib = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0\Lib"
        purelib = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0\Lib\site-packages"
        scripts = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0\Scripts"
        stdlib = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0\Lib"

Variables:
        BINDIR = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0"
        BINLIBDEST = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0\Lib"
        EXE = ".exe"
        EXT_SUFFIX = ".cp312-win_amd64.pyd"
        INCLUDEPY = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0\Include"
        LIBDEST = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0\Lib"
        TZPATH = ""
        VERSION = "312"
        VPATH = "..\.."
        abiflags = ""
        base = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0"
        exec_prefix = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0"
        installed_base = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0"
        installed_platbase = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0"
        platbase = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0"
        platlibdir = "DLLs"
        prefix = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0"
        projectbase = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0"
        py_version = "3.12.0"
        py_version_nodot = "312"
        py_version_nodot_plat = "312"
        py_version_short = "3.12"
        srcdir = "C:\Users\rolfz\.pyenv\pyenv-win\versions\3.12.0"
        userbase = "C:\Users\rolfz\AppData\Roaming\Python"

Example pyproject.toml

The issue occurs during the poetry init process, so no pyproject.toml file is generated yet.


# Example pyproject.toml that would have been generated if the issue did not occur
[tool.poetry]
name = "test"
version = "0.1.0"
description = ""
authors = ["peachy <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.12"
numpy = "^1.26.0"  # This dependency cannot be added during the `poetry init` process


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Poetry Runtime Logs

PS D:\hub\test> poetry init -vvv

This command will guide you through creating your pyproject.toml config.

Package name [test]:
Version [0.1.0]:
Description []:
Author [peachy <[email protected]>, n to skip]:
License []:
Loading configuration file C:\Users\rolfz\AppData\Roaming\pypoetry\config.toml
Compatible Python versions [^3.13]:  ^3.12

Would you like to define your main dependencies interactively? (yes/no) [yes]
You can specify a package in the following forms:
  - A single name (requests): this will search for matches on PyPI
  - A name and a constraint (requests@^2.23.0)
  - A git url (git+https://github.com/python-poetry/poetry.git)
  - A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
  - A file path (../my-package/my-package.whl)
  - A directory (../my-package/)
  - A url (https://example.com/packages/my-package-0.1.0.tar.gz)

Package to add or search for (leave blank to skip): numpy
[urllib3:urllib3.connectionpool] Starting new HTTPS connection (1): pypi.org:443
[urllib3:urllib3.connectionpool] https://pypi.org:443 "GET /search?q=numpy HTTP/11" 301 213
[urllib3:urllib3.connectionpool] https://pypi.org:443 "GET /search/?q=numpy HTTP/11" 200 23878
Unable to find package

Add a package (leave blank to skip): matplotlib
[urllib3:urllib3.connectionpool] Starting new HTTPS connection (1): pypi.org:443
[urllib3:urllib3.connectionpool] https://pypi.org:443 "GET /search?q=matplotlib HTTP/11" 301 218
[urllib3:urllib3.connectionpool] https://pypi.org:443 "GET /search/?q=matplotlib HTTP/11" 200 24053
Unable to find package

Add a package (leave blank to skip): scipy
[urllib3:urllib3.connectionpool] Starting new HTTPS connection (1): pypi.org:443
[urllib3:urllib3.connectionpool] https://pypi.org:443 "GET /search?q=scipy HTTP/11" 301 213
[urllib3:urllib3.connectionpool] https://pypi.org:443 "GET /search/?q=scipy HTTP/11" 200 23932
Unable to find package

Add a package (leave blank to skip):

Would you like to define your development dependencies interactively? (yes/no) [yes]
Package to add or search for (leave blank to skip): pytest
[urllib3:urllib3.connectionpool] Starting new HTTPS connection (1): pypi.org:443
[urllib3:urllib3.connectionpool] https://pypi.org:443 "GET /search?q=pytest HTTP/11" 301 214
[urllib3:urllib3.connectionpool] https://pypi.org:443 "GET /search/?q=pytest HTTP/11" 200 23843
Unable to find package

Add a package (leave blank to skip):

Generated file

[tool.poetry]
name = "test"
version = "0.1.0"
description = ""
authors = ["peachy <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.12"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"


Do you confirm generation? (yes/no) [yes]

[hakuna-max]

I have verified the connectivity to https://pypi.org/simple/ using the following command:
curl -I https://pypi.org/simple/

The response confirms that the endpoint is accessible, as shown below:

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 30348005
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Content-Type, If-Match, If-Modified-Since, If-None-Match, If-Unmodified-Since
Access-Control-Allow-Methods: GET
Access-Control-Max-Age: 86400
Access-Control-Expose-Headers: X-PyPI-Last-Serial
X-PyPI-Last-Serial: 26316589
Cache-Control: max-age=600, public
ETag: "4xfxvUQA5tXse+HrhPZ9MQ"
Content-Security-Policy: default-src 'none'; sandbox allow-top-navigation
Referrer-Policy: origin-when-cross-origin
Accept-Ranges: bytes
Date: Tue, 03 Dec 2024 11:47:08 GMT
X-Served-By: cache-iad-kjyo7100076-IAD, cache-hkg17923-HKG
X-Cache: HIT, HIT
X-Cache-Hits: 914156, 1009
X-Timer: S1733226428.271778,VS0,VE0
Content-Type: text/html
Vary: Accept-Encoding, Accept
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Permitted-Cross-Domain-Policies: none
Permissions-Policy: publickey-credentials-create=(self),publickey-credentials-get=(self),accelerometer=(),ambient-light-sensor=(),autoplay=(),battery=(),camera=(),display-capture=(),document-domain=(),encrypted-media=(),execution-while-not-rendered=(),execution-while-out-of-viewport=(),fullscreen=(),gamepad=(),geolocation=(),gyroscope=(),hid=(),identity-credentials-get=(),idle-detection=(),local-fonts=(),magnetometer=(),microphone=(),midi=(),otp-credentials=(),payment=(),picture-in-picture=(),screen-wake-lock=(),serial=(),speaker-selection=(),storage-access=(),usb=(),web-share=(),xr-spatial-tracking=()

This may indicate that the issue is not related to network connectivity or access to the PyPI repository. Instead, it seems specific to how poetry init handles package search queries.

[dimbleby]

indeed the search returns no results

$ curl https://pypi.org/simple/search/?q=numpy
<!DOCTYPE html>
<html>
  <head>
    <meta name="pypi:repository-version" content="1.3">
    <title>Links for search</title>
  </head>
  <body>
    <h1>Links for search</h1>
</body>
</html>
<!--SERIAL 515503-->d

perhaps something has changed or broken at https://github.com/pypi/warehouse, I expect you'll want to ask over there

[Secrus]

PyPI removed info about the latest version from their search results, breaking our search logic.

[stefansjs]

I'm surprised this was ever supported. Pip has been unable to search for years at this point

pip search numpy
ERROR: XMLRPC request failed [code: -32500]
RuntimeError: PyPI no longer supports 'pip search' (or XML-RPC search). Please use https://pypi.org/search (via a browser) instead. See https://warehouse.pypa.io/api-reference/xml-rpc.html#deprecated-methods for more information.

Probably the best solution is to just use the index directly e.g. https://pypi.org/simple/pytest/. I doesn't let the user do any kind of fuzzy matching, but it at least lets us interactively define dependencies when we're sure we know the package name exactly.

[newbery]

I'm guessing the culprit is this?
pypi/warehouse#17204

[SANJITH-KUMAR-20]

I am new to using poetry and I was very confused why it wasn't working... now I know why