Question about async route handlers

[jeff-h]

My app will want to be able to call the helper several times, and have it run various multiple Process() handlers concurrently (or even the same handler multiple times concurrently). I'm a little uncertain what my route handlers will need to do for that to be possible.

I notice in the sample you've used waitUntilExit(). If I do likewise, will the helper only start a subsequent call to this (or any other) handler once the first one finishes?

I'm targeting 10.15+ so I can mark my Process-related route handlers with async — will that ensure they can all be called and run concurrently?

[jakaplan]

This is a good question. This isn't something I've tested out comprehensively in my own implementations. I suspect this ought to be an area worthy of future improvement as well as documentation.

tl;dr It actually does appear that using the async variants will currently do what you want, although this is entirely unintentional.

In more detail

By default all incoming requests are run on the default priority global concurrent queue. You can specify a different one by setting XPCServer's targetQueue property. You might reasonably assume that either using the default concurrent queue or setting your own concurrent queue would mean that multiple handlers can be running simultaneously, but the underlying XPC documentation makes clear that's not necessarily the case. Two requests from the same XPCClient will always be "processed" serially even if the targetQueue is a concurrent one.

However, the SecureXPC framework itself is the one which directly does the "processing" and then routes to the corresponding handler. When the handler is async what's happening internally is a new Task is created and then the dispatch queue is no longer being used and so a new incoming request can be processed. This is in contrast to synchronous handlers which are run on the current thread which is being managed by that dispatch queue and so no new incoming requests will be processed until it completes. The fact these differ is undesirable and unintentional.

I'll look into making these behaviors consistent; this is likely a simple change. But for now I believe the async variants will do what you want.

[jeff-h]

I've tested it both with and without async and it appears to work just fine either way; it's happy to run numerous slow route handler calls concurrently in both cases. Which seems odd to me but I'm aware of my knowledge deficit in this area.

It did raise a separate but related question — in the previous incarnation of my XPC Helper code I was able to pass data from the helper back to the app's specified callback as many times as I wanted. So, for example, I believe I could run some binary using Process() in the helper, and have it send the output and error pipes back to the app as their data came in. Is anything like that possible with SecureXPC? I don't think my app actually needs it, just curious (I'm still converting old code to new SecureXPC routes).

[jakaplan]

I was able to pass data from the helper back to the app's specified callback as many times as I wanted. So, for example, I believe I could run some binary using Process() in the helper, and have it send the output and error pipes back to the app as their data came in. Is anything like that possible with SecureXPC? I don't think my app actually needs it, just curious (I'm still converting old code to new SecureXPC routes).

I believe what you're describing is the ability for an XPC service to initiate a send to an XPC client. SecureXPC doesn't support this due to the conceptual complexity it introduces. In SecureXPC an XPCServer can only send data to an XPCClient in response to a request made by the client.

This could change in the future, but isn't currently being considered because it will significantly increase complexity. If enough use cases accumulate that show this functionality is compelling this will be reconsidered.

[jeff-h]

Makes sense. If a client expects a flow of data back, polling the server every second or so would probably work in any use case I personally can envisage wanting this.

[jakaplan]

It did raise a separate but related question — in the previous incarnation of my XPC Helper code I was able to pass data from the helper back to the app's specified callback as many times as I wanted. So, for example, I believe I could run some binary using Process() in the helper, and have it send the output and error pipes back to the app as their data came in. Is anything like that possible with SecureXPC? I don't think my app actually needs it, just curious (I'm still converting old code to new SecureXPC routes).

I believe your scenario is now supported as of 0.5.0. A new capability "sequential replies" has been introduced which allows for a server to reply arbitrarily many times to the same request. If you try it out, I'd appreciate any thoughts/feedback you have as a new discussion!

[jeff-h]

Ha, this sounds awesome! :) I've rebuilt all my helper calls now and realised none of them needed this, but I have a few other ideas which could require it. Unfortunately I need to concentrate on more mundane matters in the meantime.