-
-
Notifications
You must be signed in to change notification settings - Fork 468
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature: allow to generate requirements.lock for each modules in a workspace #615
Comments
I quite like the idea but that could be quite slow. Might be a good idea to brainstorm how this would be configured and how it would behave. |
I might be missing something, but I don't think it will be much less performant. It would just be writing a file per each of the workspace projects. I would like to propose my idea and maybe even work on it (my rust knowledge is really basic, but I would love to give it a shot): Configuration:Under Behavior:The per member locks would be purely symbolic, they wont be used to install any requirements, that would be all handled with the top level lock Found this issue, as it happens to be the exact use case I would like to have in rye and found missing (usecase is a monorepo), and would love to aid and implement it if I can! |
Rye does appear to know which sub-projects need which modules, since it documents the references in the lockfile itself:
Although it would require the sub-projects to correct declare their depencies, rather than relying upon the universe of installed packages being sufficient, this does seem like the sort of thing that could be determined by pruning the dependency tree. (Honestly, you might not need to get THAT sophisticated if you parsed the lockfile comments and build the tree from that...) |
Update: First, I see that there is a PR open to fix this, and thanks @davfsa for taking a stab at it! I the meantime, I have managed to hack this together as follows: Imagine I have a workspace with project_a, project_b, and project_c. Now imagine that project_b depends on project_a, and project_c depends on project_b In each sub-project's pyproject.toml, I list dependencies for that project of course, as well as dependencies on other locally-installed package in my workspace. I just reference them by name though! I don't reference them by any sort of relative path. (This works fine because every time we need to resolve that package at package install time, it's already been installed for us by rye or another mechanism anyway.) So this is where it starts to get a bit hacky. As a sibling of each pyproject.py, I've got a requirements.in file, that looks like this.
Now I can go into project_b and run: WORKSPACE_DIR=$(git rev-parse --show-toplevel)
FOLDER=src/project_b
RELATIVE_PATH_FROM_FOLDER_TO_WORKSPACE=$(realpath --relative-to="$FOLDER" "$WORKSPACE_DIR")
rye run uv pip compile \
--constraint "$RELATIVE_PATH_FROM_FOLDER_TO_WORKSPACE/requirements.lock" \
--strip-extras \
./pyproject.toml \
./requirements_local.in \
-o requirements.lock (You can do a standard This tells pip compile: It will also include the references to the editable installs of the local packages, which you might not want:
If you don't want that sort of thing, you can add another option to the That way, the reference to project-a itself isn't in the lockfile, but its deps still are. This is equivalent to pip's |
If you want to (ab)use this lockfile a bit more and make sure that your project-specific lockfile is sufficient, you can create a sub-venv with that lockfile you made:
Now you can run in your sub-project:
|
When using a workspace with multiples modules, all packages are installed in a top-level
.venv
with a requirements.lock that combine all packages. It's good for local developpementBut when you want to build a Docker image for each module, you want to minimize the size of your build, so you want to only install the package necessary for each module of the workspace. Currently you can use RYE and docker very easily with:
The issue is that, in workspace we only have a top-level
requirements.lock
.Feature suggestion: when using a workspace, we should add the option to generate a per-module
requirements.lock
Like:
The text was updated successfully, but these errors were encountered: