First Class Python Support (4000USD Bounty)

[lihaoyi]

---

From the maintainer Li Haoyi: I'm putting a 4000USD bounty on this issue, payable by bank transfer on merged PRs implementing this. Standard bounty terms apply

---

Python is common in industry, and lacks good build tools. Mill could a be a good tool to help manage large multi-module Python projects (see What Makes Mill Unique?) providing automatic caching and parallelism for the various workflows (dependency resolution, typechecking, testing, packaging, publishing) that are core to the local development experience to help keep them fast and responsive.

The goal of this ticket is to generate a set of Python build examples that match the Java/Scala/Kotlin equivalents. We already have an example PythonModule for demo purposes, but we would need to flesh it out using the equivalent Python tools and libraries.

For the purposes of this ticket, each examples should match as closely as possible the Java/Scala/Kotlin equivalents, to provide a useful minimal-but-still-educational code example, along with associated english documentation and explanations. You should read through the relevant sections of the Building Java with Mill documentation before proceeding, even if you don't know Java, just to get a feel for what the documentation and examples for each section should cover.

1. example/pythonlib/basic/ (500USD)

   • 1-simple/: A minimal Python module demonstrating typechecking/running/testing/pex
     • We should also demonstrate how to use the Python REPL hooked up to your module
   • 2-custom-build-logic/: A Python module with custom build logic
   • 3-multi-modules/: Multiple inter-related Python modules

2. example/pythonlib/dependencies/ (500USD)

   • 1-pip-deps/
   • 2-unmanaged-wheels/
   • 3-downloading-unmanaged-wheels/
   • 4-repository-config/: examples of how to use alternate PyPI mirrors or repositories

3. example/pythonlib/linting/ (1000USD)

   • 1-black/: equivalent to 1-scalafmt but using Black
   • 2-code-coverage/: not sure what the popular Python code coverage lib is, but they should have something
   • 3-ruff/: using Ruff to lint the Python code

4. example/pythonlib/module/ (500USD)

   • 1-common-config/
   • 2-custom-tasks/
   • 3-override-tasks/
   • 4-compilation-execution-flags/
   • 5-resources/
   • 6-pex-config/

5. example/pythonlib/testing/ (500USD)

   • 1-test-suite/: we should have examples of Python being used with at least two different testing frameworks: pytest and unittest
   • 2-test-deps/
   • 3-integration-suite/

6. example/pythonlib/publishing/1-publish-module/, example/pythonlib/basic/4-realistic/ (500USD)

   • publishing/1-publish-module should demonstrate how to publish a Python package to PyPI
   • basic/4-realistic is only doable after all the previous bullets are done, so I'm grouping its bounty together with publishing/

7. example/pythonlib/web/ (500USD)

   • 1-hello-flask/: hello world website using Flask framework
   • 2-todo-flask/: TodoMVC webapp using Flask framework

8. example/pythonlib/web/ (500USD)

   • 3-hello-django/:hello world website using Django framework
   • 4-todo-django/: TodoMVC webapp using Django framework

[himanshumahajan138]

@lihaoyi Ready for this but need your Guidance (in between)

I think we should do this in parts...

[lihaoyi]

@himanshumahajan138 yes, will need to think about the best way to approach this. It would be a pretty large scale effort and will definitely need to be broken down into parts

[himanshumahajan138]

Aaah! Great approach for subparts

Working on this as it will require huge code base so will start from small to big

Let's do this ✨

[himanshumahajan138]

@lihaoyi I want(have) to create a Proper PythonModule.scala using the Last PR Code(#3882) and Need some time so fully create it for each of the above-mentioned task

So, I request to avoid Clashes if anyone trying to do this please once comment...
(and i know bounties are available for everyone, Still if possible, assign me as one hand code contains flow 🤞)

[mpkocher]

@lihaoyi Perhaps It would be useful to clarify how pyproject.toml fits (or doesn't fit) into your vision for adding Python support to mill.

https://peps.python.org/pep-0621/

[lihaoyi]

@mpkocher I don't actually know yet, we'll probably need to figure that out as we flesh out the implementation

[lihaoyi]

For anyone reading, I merged #3970 which sets up the basic scaffolding:

• Code in pythonlib/src/
• Unit test code in pythonlib/test/src/, run via ./mill pythonlib.test
• Unit test code in example/pythonlib/, run via ./mill example.pythonlib.__.local.test

From there you should be able to iterate on the examples necessary for this ticket's bullet points to flesh out the necessary functionality

[himanshumahajan138]

@lihaoyi that's great you just gave the foundation brick for the building ✨