Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add duration information to stats + pants logs #21587

Open
drubinstein opened this issue Oct 29, 2024 · 3 comments
Open

Add duration information to stats + pants logs #21587

drubinstein opened this issue Oct 29, 2024 · 3 comments

Comments

@drubinstein
Copy link

Is your feature request related to a problem? Please describe.
We are trying to understand what steps in our transitive builds are slow. From what I can tell in the documentation, Pants does not have a way to display how long a build task takes.

Describe the solution you'd like
A couple of solutions.

  1. Add the duration it takes for a task to take. I believe that could be done by adding timing information to Results and modifying the check log.
  2. Add durations stats as an option to the stats block in pants.toml. If set, then a table of tasks and their durations would print at the end of execution.

Describe alternatives you've considered
I've thought about making a new "timer rule" for every task, but that seems excessive.

Additional context
An example of what I'm asking for:

Before:

13:11:38.34 [INFO] Completed: Run Pytest - dir1:test1 - succeeded.
13:11:39.30 [INFO] Completed: Run Pytest - dir2:tests2 - succeeded.

After:

13:11:38.34 [INFO] Completed: Run Pytest - dir1:test1 - succeeded in 53s.
13:11:39.30 [INFO] Completed: Run Pytest - dir2:tests2 - succeeded in 2m45s.
@benjyw
Copy link
Contributor

benjyw commented Oct 29, 2024

Pants does gather all this nested trace information, via the "workunits" concept (basically a tree of trace information).

You can get it as JSON by running with --workunit-logger-enabled

Does this provide the information you're after?

@huonw
Copy link
Contributor

huonw commented Oct 29, 2024

Adding a bit of colour to Benjy's comment, here's what we do in our CI:

# pants.toml
[GLOBAL]
...

backend_packages = [
...
  "pants.backend.experimental.tools.workunit_logger",
]

...
# pants.ci.toml
...

[workunit-logger]
enabled = true
logdir = ".pants.d/workunit-logger"

And then our CI is configured to read pants.ci.toml (https://www.pantsbuild.org/stable/docs/using-pants/using-pants-in-ci#configuring-pants-for-ci-pantscitoml-optional) and also upload the .pants.d/workunit-logger as an artifact for later analysis.

@drubinstein
Copy link
Author

That is extremely helpful. It'd still be nice if the durations could be printed to the console itself for manual debugging, but this is enough to get us started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants