Exp vs queue

[dberenbaum]

See discussion in iterative/dvc.org#3715 and comments from @shcheklein and @jorgeorpinel.

Need to clarify when to use dvc exp vs. dvc queue and reduce the need for users to learn two separate sets of commands or switch back and forth between them.

This will only get more complicated because we are looking to introduce hyperparameter sweeps using syntax from Hydra. Does this make sense as an option to dvc exp run or should it be part of some queue-specific command (@daavoo)?

[dberenbaum]

@dberenbaum makes total sense. We can still probably expose queue to the experiments level in a different way? e.g. dvc exp run-all ? etc

Originally posted by @shcheklein in iterative/dvc.org#3715 (comment)

IMO we can make an alias to dvc queue start in dvc exp run-all or dvc exp start. Any better naming suggestions? cc @jorgeorpinel.

To replicate existing functionality of dvc exp run --run-all, we should add something like a --foreground option to dvc queue start.

Thoughts?

[jorgeorpinel]

expose queue to the experiments level in a different way? e.g. dvc exp run-all ?

make an alias to dvc queue start in dvc exp run-all or dvc exp start. Any better naming suggestions?

Would have to be run-all or start-queue. Just start sounds like run.

Also what about queue stop?

add something like a --foreground option to dvc queue start.

Good idea. Maybe just --fg ?

[jorgeorpinel]

when to use dvc exp vs. dvc queue

If we move the exp run --queue function over to queue run, the overlap would be reduced.

reduce the need for users to learn two separate sets of commands or switch back and forth

Can you elaborate? If this is a want, why did we create all the new queue commands? Vs. e.g. exp queue with a bunch of options (--start/stop/status/etc.)

[dberenbaum]

reduce the need for users to learn two separate sets of commands or switch back and forth

Can you elaborate? If this is a want, why did we create all the new queue commands? Vs. e.g. exp queue with a bunch of options (--start/stop/status/etc.)

I don't think it would be possible to fit everything into exp queue unfortunately. We considered putting everything under exp but it seemed too busy and better as a separate command, especially as we saw that there are important differences between a queue task and a finished experiment. Still trying to figure out the right balance here to create simple UX.

[dberenbaum]

If we move the exp run --queue function over to queue run, the overlap would be reduced.

🤔 If we want to consolidate as much as possible into dvc exp, what about exp add instead of exp run --queue or queue run? Seems like a nice parallel to stage add/run, where you can either do setup and execution in one step or two.

Edit: queue add also makes sense if we want to emphasize dvc queue. It was originally considered but had so much overlap with exp run that I didn't think it was worthwhile.

[jorgeorpinel]

No strong opinions really, but I don't think we're addressing "reduce the need for users to learn two separate sets of commands or switch back and forth", which to me still sounds contradictory to the current design.

[dberenbaum]

See also #7866 (comment)

[jorgeorpinel]

> expose queue to the experiments level in a different way? e.g. dvc exp

run-all ?

make an alias to dvc queue start in dvc exp run-all or dvc exp start. Any

better naming suggestions? Would have to be `run-all` or `start-queue`. Just `start` sounds like `run`. Also what about `queue stop`?

add something like a --foreground option to dvc queue start.

Good idea. Maybe just `--fg` ?

…

On Mon, Aug 29, 2022 at 4:26 PM Dave Berenbaum ***@***.***> wrote: @dberenbaum <https://github.com/dberenbaum> makes total sense. We can still probably expose queue to the experiments level in a different way? e.g. dvc exp run-all ? etc *Originally posted by @shcheklein <https://github.com/shcheklein> in iterative/dvc.org#3715 (comment) <iterative/dvc.org#3715 (comment)>* IMO we can make an alias to dvc queue start in dvc exp run-all or dvc exp start. Any better naming suggestions? cc @jorgeorpinel <https://github.com/jorgeorpinel>. To replicate existing functionality of dvc exp run --run-all, we should add something like a --foreground option to dvc queue start. Thoughts? — Reply to this email directly, view it on GitHub <#8123 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALIXH2VKAHQ7JOKGOIZLR3V3UMFRANCNFSM56EXXC3A> . You are receiving this because you were mentioned.Message ID: ***@***.***>