Improving docs

docs/dev.rst

@@ -3,9 +3,7 @@
 Developer guide
 ===============
 
-For getting started as a developer, just follow the :ref:`installation` guide.
-
-
+For getting started as a developer, first follow the :ref:`installation` guide.
 
 Technical notes
 ---------------
@@ -121,10 +119,11 @@ Extensible by default.
 
 Do not implement a compiler
    We have a number of ongoing projects for implementing a world class differential equations compiler
-   for LUCIDAC/REDAC. At the same time there is an urgent need for programming LUCIDAC in a less
-   cryptic way then ``route -- 8 0 -1.25 8``. Therefore, the lucipy :py:class:`.Circuit` class and friends
-   tries to provide as few code as possible to make this more comfortable, without implementing too
-   many logic.
+   for LUCIDAC/REDAC. The lucipy :py:class:`.Circuit` class provides a very shim layer ontop of
+   the "raw" numeric configuration of the interconnection matrix. It barely hides the fact that there
+   is a lot of indices going from A to B and allows to interact with these numbers. The approach is
+   a greedy "place early" approach. Instead of providing a compiler, lucipy tries to be a toolbox
+   for conversion formats.
 
 Implement the UNIX principle
    *Do one thing and do it good* is the major design goal of lucipy. Any sophisticated task should
@@ -167,9 +166,12 @@ No async co-routines
    My personal preference is that async gives a terrible programmer's experience
    (I wrote about it: `python co-routines considered bad <https://denktmit.de/blog/2024-07-11-Reductionism-in-Coding/>`_).
    It is also *premature optimization* for some future-pointing high performance message broker
-   which does single-threaded work while asynchronously communicating with the REDAC. So
-   let's get back to the start and work synchronously, which *dramatically* reduces the
-   mental load.
+   which does single-threaded work while asynchronously communicating with the REDAC.
+
+   The main problem with async's is that it somewhat breaks the brief code style python can have.
+   Python can serve as an excellent domain specific language (DSL) with a pretty terse syntax. Adding
+   ``async`` in front of literally every word makes this much harder to read and write. Furthermore,
+   asyncs require an ``async main`` and thus in general disturb the REPL kind of use.
 
 No typing
    There is little advantage of having a loosely typed server (firmware without typed JSON mapping)

docs/simulation.rst

@@ -28,8 +28,8 @@ This Simulator adopts the convention with the column order
 
 ::
 
-    M1 Mul
-    M0 Int
+    M0 = Integrator Block (8 Integrators)
+    M1 = Multiplier Block (4 Multipliers, 4 identity elements)
 
 The basic idea is to relate the standard first order differential equation
 :math:`\dot{\vec x} = \vec f(\vec x)` with the LUCIDAC system matrix
@@ -58,10 +58,10 @@ elements, in particular the multipliers. By splitting the matrix
 
 .. math::
 
-    \begin{pmatrix} M^{in} \\ I^{in} \end{pmatrix}
+    \begin{pmatrix} I^{in} \\ M^{in} \end{pmatrix}
     = 
     \begin{bmatrix} A & B \\ C & D \end{bmatrix}
-    \begin{pmatrix} M^{out} \\ I^{out} \end{pmatrix}
+    \begin{pmatrix} I^{out} \\ M^{out} \end{pmatrix}
 
 Note how this notation maps implicit summing of the UCI matrix on the summing
 property of a matrix multiplication.
@@ -76,21 +76,21 @@ First, let us write out the vectors
 
     \begin{aligned}
     M^{in} &= \left( M^{in}_{0a}, M^{in}_{0b}, M^{in}_{1a}, M^{in}_{1b}, \dots, M^{in}_{3a} \right) \\
-    M^{out} &= \left( M^{out}_0, M^{out}_1, M^{out}_2, M^{out}_3, c_0, c_1, c_2, c_3 \right) \\
+    M^{out} &= \left( M^{out}_0, M^{out}_1, M^{out}_2, M^{out}_3, M^{in}_{0a}, M^{in}_{0b}, M^{in}_{1a}, M^{in}_{1b} \right) \\
     I^{in} &= \left( I^{in}_0, \dots, I^{in}_7 \right) \\
     I^{out} &= \left( I^{out}_0, \dots, I^{out}_7 \right)
     \end{aligned}
 
 This is a definition for REV0 where the superfluous Math block outputs are used
-for constant givers :math:`c_i = 1`.
+for identity elements.
 
 The algorithm is as following: The set of equations is written out as
 
 .. math::
 
     \begin{aligned}
-    M^{in}_i &= A_{ij} ~ M^{out}_j + B_{ij} ~ I^{out}_{ij} \quad&&\text{(eq I)} \\
-    I^{in}_i &= C_{ij} ~ M^{out}_j + D_{ij} ~ I^{out}_{ij} \quad&&\text{(eq II)}
+    I^{in}_i &= A_{ij} ~ I^{out}_j + B_{ij} ~ M^{out}_{ij} \quad&&\text{(eq I)} \\
+    M^{in}_i &= C_{ij} ~ I^{out}_j + D_{ij} ~ M^{out}_{ij} \quad&&\text{(eq II)}
     \end{aligned}
 
 and then compute (eq I) :math:`M^{in} = g(I^{out})` with an initial guess :math:`M^{out}=0` and

docs/synchc.rst

@@ -3,18 +3,5 @@
 Lucipy Synchronous Client
 =========================
 
-.. automodule:: lucipy.synchc
-   :no-members:
-
-Usage
------
-
-.. autoclass:: lucipy.synchc.LUCIDAC
-   :members:
-
-API refererence
----------------
-
 .. automodule:: lucipy.synchc
    :members:
-   :exclude-members: LUCIDAC

docs/usage.rst

@@ -1,6 +1,52 @@
 .. _usage:
 
-Basic patterns and examples
-===========================
+Basic patterns and usage
+========================
 
-something comes here
+This page gives an introductory overview about how the code can be used to solve common
+problems. This shall not replace a general introduction about the LUCIDAC computer and its
+abilities. This can be read in the *lucidoc* end user documentation (to be linked here
+as PDF once it is ready).
+
+Circuit configuration
+---------------------
+
+The typical first time user experience of lucipy is to run a few of the :ref:`example-circuits`.
+They all follow the same schema:
+
+#. Construct :class:`~lucipy.circuits.Circuit`
+#. Connect to :class:`~lucipy.synchc.LUCIDAC` and configure/upload the circuit
+#. Start run, do data aquisition
+#. Visualize data
+
+Parameter study
+---------------
+
+In order to do a parameter study, one wants to loop the previously described process.
+Many "unit test cases" provided in the repository do this in order to aquire a lot of data.
+
+It is possible to alter only parts of the configuration and send them, for instance by using
+the :meth:`~lucipy.synchc.LUCIDAC.set_by_path` method of :class:`~lucipy.synchc.LUCIDAC`.
+This, however, requires some understanding of the
+`entity concept <https://anabrid.dev/docs/hybrid-controller/de/dfd/entities.html>`_ of LUCIDAC.
+This can probably speed up the configuration process a bit.
+
+Data aquisition and Manual Steering
+-----------------------------------
+
+For interactive sessions with an oscilloscope, it can be very interesting to decouple the
+integrated analog-to-digital data aquisition from the analog computer operating mode, i.e.
+the simple state machine `IC -> OP -> HALT`. The method
+:meth:`~lucipy.synchc.LUCIDAC.manual_mode` is suitable for steering this mode for instance
+from the python prompt or within a script. Note that this is far from allowing real time
+control.
+
+At the other hand, data aquisition can be triggered manually with the
+:meth:`~lucipy.synchc.LUCIDAC.one_shot_daq` call, which will return a single sample of the
+eight ADC samples.
+
+Master/Minion multi-device use
+------------------------------
+
+The :class:`~lucipy.synchc.LUCIGroup` allows for steering multiple LUCIDACs in one go. For
+having precisely locked state machines, this requires a digital front panel connection.

docs/zeroconf.rst

@@ -8,13 +8,53 @@ LUCIDAC device. The concept is described in the
 `Firmware docs <https://anabrid.dev/docs/hybrid-controller/>`_ and is similar to
 the `NI VISA resource syntax <https://www.ni.com/docs/en-US/bundle/ni-visa/page/visa-resource-syntax-and-examples.html>`_
 
+Available endpoints in lucipy
+-----------------------------
+
 Which endpoint notation and protocol support is available depends on the client
 implementation. Lucipy understands the following endpoints:
 
-* ``serial:/`` - USB Serial terminals speaking JSONL
-* ``tcp:/`` - "Raw" TCP/IP speaking JSONL
-* ``sim:/`` - A shorthand to the integrated simulator
+USB Serial (virtual terminal) speaking JSONL: ``serial:/``
+   A device name is expected afterwards which is immediately passed to the
+   `pySerial constructor <https://pyserial.readthedocs.io/en/latest/pyserial_api.html>`_.
+   Examples are ``serial://dev/ttyACM0`` on GNU Linux/Mac OS X or ``serial:/COM0`` on
+   MS Windows. USB Serial connection is typically error-prone, in particular at startup
+   when the buffers still can be filled with old contents. When using the serial
+   connection, calls to :meth:`hc.slurp` can help to clear the buffers.
+
+"Raw" TCP/IP speaking JSONL: ``tcp:/``
+   This is the native protocol of the LUCIDAC and the most realiable way
+   to connect to the LUCIDAC. Typical examples are a Host name
+   ``tcp://my-lucidac.local.`` or an IPv4 address ``tcp://192.168.150.229``.
+   An optional port can be given, ``tcp://192.168.150.229:5732``.
 
+Integrated Simulator/Emulator: ``emu:/``
+   This will *emulate a socket* to the lucipy-integrated LUCIDAC Emulator (see :ref:`emu`). 
+   Note that this is also possible by starting the Emulator at another place
+   and connecting via something like ``tcp://localhost:1234``. However, this way
+   it acts as a shorthand for starting up the emulator at the same time as the
+   client, one does not have to transfer the TCP port information. Furthermore, the
+   connection does *not* use TCP/IP but is just a python-internal function call
+   (this is what was refered to an *emulated socket* in the beginning).
+
+   To use this endpoint, just instanciate with ``LUCIDAC("emu:/")``. There are no
+   further paths in this URL. However, the optional argument ``?debug`` can be attached
+   to start the python debugger if the Emulator crashes.
+
+Device autodetection: ``zeroconf:/``
+   Use this endpoint string to explicitely use autodetection even in the presence
+   of an environment variable ``LUCIDAC_ENDPOINT``. In the same way as ``emu:/``,
+   this endpoint URL supports no further arguments.
+
+Other endpoints not listed here are explicitely not supported, in particular
+websocket and HTTP endpoints. If you need to make use of them, consider using a
+proxy such as `lucigo <https://github.com/anabrid/lucigo>`_. In particular, a
+typical need is to proxy an USB virtual serial terminal over TCP/IP and there
+are many solutions for this listed at the previous link.
+
+LUCIDAC autodetection
+---------------------
+
 For convenience, the :ref:`client code <lucipy-client>` allows for autodetection
 of the endpoint using MDNS/Zeroconf. This works by making an instance without
 providing the endpoint:
@@ -24,6 +64,9 @@ providing the endpoint:
    >>> from lucipy import LUCIDAC
    >>> hc = LUCIDAC() # this will trigger the autodetection
 
+Typically, the autodetection connects to the *first* LUCIDAC found in the network
+and warns if multiple have been found.   
+
 Direct access to the underlying API should not be neccessary, but is possible
 with :code:`import lucipy.detect`. The folling reference shows the exposed
 functions.
@@ -36,10 +79,25 @@ functions.
    `pySerial <https://pyserial.readthedocs.io/>`_ libraries installed.
 
    If these dependencies are not installed, the code will print warnings suggesting
-   you to install them in order to make autodetection work. That means that a line
-   such as :code:`hc = LUCIDAC()` will not work without an endpoint argument to the
-   :code:`LUCIDAC()` call.
-
+   you to install them in order to make autodetection work.
+
+Environment variable
+--------------------
+
+Conveniently, you can use the operating system environment variable
+``LUCIDAC_ENDPOINT``. This can help to keep your scripts clean of volatile,
+frequently changing and probably meaningless IP addresses and ports.
+
+Usage is, for instance, in your operating systems shell
+
+::
+
+    export LUCIDAC_ENDPOINT="tcp://10.10.77.123"``
+    python some-script-using-luci.py
+
+Note that if you have set the environment variable ``LUCIDAC_ENDPOINT``, this will
+effectively overwrite (and thus disable) the autodetection, except you call
+``LUCIDAC("zeroconf:/")`` explicitely in your code.
 
 Code refererence
 ----------------

lucipy/simulator.py

@@ -38,14 +38,34 @@ class Simulation:
     A simulator for the LUCIDAC. Please :ref:`refer to the documentation <sim>`
     for a theoretical and practical introduction.
     
+    The Simulator has an API which is suitable for getting a LUCIDAC circuit solved in
+    Python as easy as possible. The usage can boil down to something like
+    ``integrator_data = Simulation(circuit).solve_ivp(t_final)``. In particular, the
+    Simulator will allow to inspect the LUCIDAC and its internal states as much as you
+    want.
+    
+    If you want to use an API which is simliar to the LUCIDAC Hybrid Controller, use the
+    :class:`Emulation` instead. It uses the :class:`Simulation` under the head but does
+    not expose it but instead just returns the requested ADC sampling values when using
+    it in a way like
+    ``e = Emulation(); e.set_circuit(circuit.generate()); data = e.start_run(...)``.
+    This way, you can steer for instance the sampling times.
+    In contrast, if not requested explicitely with suitable ADC settings, it won't
+    return the full integrator state and also won't provide any support for ACL_IN/OUT
+    or further device inspection, given the limited JSON_API of the LUCIDAC. See there
+    for getting a list of limitations.
+    
     Important properties and limitations:
     
     * Currently only understands mblocks ``M0 = Int`` and ``M1 = Mul`` in REV1-Hardware fashion
     * Unrolls Mul blocks at evaluation time, which is slow
-    * Note that the system state is purely hold in I.
-    * Note that k0 is implemented in a way that 1 time unit = 10_000 and
-      k0 slow results are thus divided by 100 in time.
-      Should probably be done in another way so the simulation time is seconds.
+    * Note that the system state is purely hold in the integrators.
+      It always evolves all 8 integrators and returns all 8 integrators.
+      It is up to the user to use the provided mapping functions to derive the
+      neccessary output.
+    * Note that by default ``k0`` is implemented in a way that ``1 time unit = 10_000``
+      and ``k0`` slow results are thus divided by ``100`` in time. Use ``realtime``
+      to measure simulation time in seconds instead.
     
     :arg circuit: An :class:`circuits.Circuit` object. We basically only need it
       in order to make use of the :meth:`~circuits.Circuit.to_dense_matrix` call.
@@ -55,9 +75,9 @@ class Simulation:
       applications. You can set the time factor later by overwriting the ``int_factor``
       property.
     
-    .. note::
+   
+    Note, here is a tip to display the big matrices in one line:
     
-       Here is a tip to display the big matrices in one line:
        >>> import numpy as np
        >>> np.set_printoptions(edgeitems=30, linewidth=1000, suppress=True)
 
@@ -181,7 +201,7 @@ def Mul_out(self, Iout, t=0):
     def nonzero(self):
         """
         Returns the number of nonzero entries in each 2x2 block matrix. This makes it easy to
-        count the different type of connections in a circuit (like INT->INT, MUL->INT, INT->MUL, MUL->MUL).
+        count the different type of connections in a circuit (like ``INT->INT, MUL->INT, INT->MUL, MUL->MUL``).
         """
         import numpy as np
         sys = np.array([[self.A,self.B],[self.C,self.D]])
@@ -278,6 +298,10 @@ def set_acl_in(self, callback=None):
         
         If you want to remove the ACL_IN callback function, call the method with
         argument ``None``.
+        
+        .. warning::
+        
+           This is merely a stub, the ACL Input is NOT YET fully implemented.
         """
         self.use_acl_in = True
         self.acl_in_callback = callback