Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Book generation: the process is too slow #551

Open
spirosChv opened this issue Jan 6, 2022 · 3 comments
Open

Book generation: the process is too slow #551

spirosChv opened this issue Jan 6, 2022 · 3 comments
Labels
Infrastructure Things related to the workflows (e.g., book generation, process notebooks) jupyterbook Issues related to jupyterbook (render, workflow, etc) performance Changes that are for performance improvements only.

Comments

@spirosChv
Copy link
Collaborator

The book generation workflow runs on CPU via github actions. The complete process needs approx. 3.5 hours. This is too much especially during the course. The process must be faster.

Possible workarounds:

  1. Run on GPUs (either locally, or on cloud service gpus)
  2. Use the cache options, so that the book will run only for the updated notebooks, if possible. See here
@spirosChv spirosChv added Infrastructure Things related to the workflows (e.g., book generation, process notebooks) performance Changes that are for performance improvements only. labels Jan 6, 2022
@spirosChv
Copy link
Collaborator Author

spirosChv commented Jan 9, 2022 •
edited
Loading

  1. Run on GPUs (either locally, or on cloud service gpus)

Added a local machine to execute all notebooks --> Several CUDA errors. Needs debugging

@spirosChv spirosChv added Code-improvement Something is not working as expected, but does not prevent the correct execution Code-update Version issue and/or update code to current versions Urgent Things that break tutorials and need immediate fixes labels Jan 9, 2022
@spirosChv spirosChv removed Code-improvement Something is not working as expected, but does not prevent the correct execution Code-update Version issue and/or update code to current versions labels Jul 22, 2022
@spirosChv spirosChv added help wanted Extra attention is needed jupyterbook Issues related to jupyterbook (render, workflow, etc) and removed Urgent Things that break tutorials and need immediate fixes labels Jul 6, 2023
@spirosChv
Copy link
Collaborator Author

The book generation leads also to several error:

  1. Unhandled exception. System.IO.IOException: No space left on device : '/home/runner/runners/2.304.0/_diag/Worker_20230606-082117-utc.log'
  2. System.IO.IOException: No space left on device : '/home/runner/runners/2.304.0/_diag/Worker_20230525-115206-utc.log'

The tweak for now is to run the book generation on a local machine with GPU.

Potential fix: run automatically the process on a local machine.

@spirosChv spirosChv removed the help wanted Extra attention is needed label Aug 2, 2023
@iamzoltan
Copy link
Contributor

Yea, I think trying to solve the cache is the best bet, although it ends up only being stored for about a week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Infrastructure Things related to the workflows (e.g., book generation, process notebooks) jupyterbook Issues related to jupyterbook (render, workflow, etc) performance Changes that are for performance improvements only.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@spirosChv @iamzoltan