[FEATURE REQUEST] "Queue prompt" button instead of or next to Generate (or maybe as a cli option when launching web-ui)

[Nyxeka]

I'm always a little peeved at having to wait for a prompt to finish generating before I can hit generate on my next several ideas - it would be cool if we could tune all the sliders we want, then just mash the "queue this cli command" - it pushes your prompt+algorithm+batch count+sampling step count etc onto the queue, and automatically generates it as soon as the current one is finished (up to an infinite amount of times).

Ideally we would have a few buttons here:

• add to queue
• [add to front of queue]
• [add to back of queue]
• cancel queue
• resume queue

Optionally this could include a list of upcoming prompts and batch sizes it is going to queue up in a table or something, with a remove button on each one. Remove should come up with "are you sure" and "don't ask me again".

Each queue option should probably write itself out to a log file in case someone deletes a queue by accident, they can go back and find it again.

mvp: "add to queue" button

[Nyxeka]

So, I eventually found out this is built into gradio, and it looks like it's more or less automatic if you just open a new tab and hit generate... It would be nice, though, if it could be consolidated. Alternatives are just refreshing the page and hitting generate again. Maybe a UX improvement to make this more obvious? I tried queue=true on generate button but it didn't seem to work.

[search620]

I would make it more simple as a + - button to add or remove from the queue (you can select multi lines with the shift key and delete them)
the outlook of it will be just like an empty list in which every "+" button will push the current setting into the list and the "-" key will remove it. and if you want to change the order of the queue you just drag and drop the line you want in the order you want.

With also option to freeze the queue for other seed and so on... maybe presets

[codedealer]

This is prompt history but in another direction lol. The demand for prompt queue/stack is definitely high.

Queueing is indeed built in in gradio as a network feature to balance the server (your pc in this case) load. My guess is if you're running it locally it can double as a prompt queue.

[Nyxeka]

I would make it more simple as a + - button to add or remove from the queue (you can select multi lines with the shift key and delete them) the outlook of it will be just like an empty list in which every "+" button will push the current setting into the list and the "-" key will remove it. and if you want to change the order of the queue you just drag and drop the line you want in the order you want.

With also option to freeze the queue for other seed and so on... maybe presets

I think the issue with the queue table is that it's limited (for now) by the built-in GUI elements that gradio library supports... I'm sure anyone can put together custom UI elements but it's definitely a more advanced feature that would take a day or two at least to get functioning for someone who's actually experienced with making custom UI on top of gradio.

[search620]

Yes but we all need it. this feature is a must-one because after I test hours and days on this I understand now that if I want to take out the juice of a certain generate I must do it with a queue of steps and classifier and also prompts... but you know what I mean. in my opinion, this feature is must to have.

[outlookhazy]

this might make it possible for multiple users to work. Currently if a second connection tries to generate something, the entire thing becomes unresponsive and has to be restarted

[Nyxeka]

sounds like the job queueing broke 🤔

[Nyxeka]

this might make it possible for multiple users to work. Currently if a second connection tries to generate something, the entire thing becomes unresponsive and has to be restarted

go to frontend/job_manager.py:160 and change the line to:

self._avail_job_tokens.append(token if token is not None else 0)

This should allow concurrent browser tabs to queue properly. The problem before was that after you queue more than one thing and they all finish, avail_job_tokens gets a None added to it instead of a 0 and seems to break things.

I would put up a PR but I'm guessing this is already fixed somewhere...

[outlookhazy]

I would put up a PR but I'm guessing this is already fixed somewhere...

It wasn't as of this morning's merge (related #858), I tested in case the partial queue cancelling included a fix.
I'd test now but don't have remote access at the moment. I will tonight if nobody else chimes in

[Nyxeka]

edit: seems that after >2 queues (tabs open) it also breaks even with the fix