core: Document preferred threading model for scalable endpoints

Recommend that applications and providers use FI_THREAD_COMPLETION
as the preferred threading model for lockless operation when using
scalable endpoints.  This helps align application design with the
provider implementation.

Signed-off-by: Sean Hefty <[email protected]>

man/fi_domain.3.md

@@ -247,18 +247,16 @@ which can guarantee serialization in their access of provider allocated
 resources and interfaces enable a provider to eliminate lower-level locks.
 
 *FI_THREAD_COMPLETION*
-: The completion threading model is intended for providers that make use
-  of manual progress.  Applications must serialize access to all objects
-  that are associated through the use of having a shared completion
-  structure.  This includes endpoint, transmit context, receive context,
-  completion queue, counter, wait set, and poll set objects.
-
-  For example, threads must serialize access to an endpoint and its
-  bound completion queue(s) and/or counters.  Access to endpoints that
-  share the same completion queue must also be serialized.
-
-  The use of FI_THREAD_COMPLETION can increase parallelism over
-  FI_THREAD_SAFE, but requires the use of isolated resources.
+: The completion threading model is best suited for multi-threaded applications
+  using scalable endpoints which desire lockless operation.  Applications must
+  serialize access to all objects that are associated by a common completion
+  mechanism (for example, endpoints bound to the same CQ or counter).  It is
+  recommended that providers which support scalable endpoints also support this
+  threading model.
+
+  Applications wanting to leverage FI_THREAD_COMPLETION should allocate
+  transmit contexts, receive contexts, and completion queues and counters to
+  individual threads.
 
 *FI_THREAD_DOMAIN*
 : A domain serialization model requires applications to serialize