Graceful Training Shutdown

[jquaglia-uss]

It would be a SUPER helpful feature if Ctrl-C gracefully shut down a training and just ended the train at the current epoch rather than aborting abruptly. Maybe not Ctrl-C but some other way to be like "hey i said train for 200 epochs but i'm good at 40, let's shut it down gracefully."

[glenn-jocher]

@jquaglia-uss hello! Thank you for the suggestion regarding a graceful shutdown feature during training. We understand the importance of having control over the training process and the ability to end it at a desired point without losing progress.

Currently, YOLOv8 does implement a form of graceful shutdown. When you initiate a training session and decide to stop it early, you can press Ctrl-C once to trigger a graceful shutdown that will complete the current epoch, run a final validation, save the latest weights, and then exit. This allows you to retain the progress up to the last completed epoch.

If you press Ctrl-C more than once, it will force an immediate shutdown, which is less graceful and may not save the latest progress.

We always aim to improve the user experience, so your feedback is valuable to us. If you have any more suggestions or questions, feel free to share! Happy training! 😊🚀