Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Routers and Selectors. Part 1 if #802

Open
emil14 opened this issue Dec 13, 2024 · 3 comments
Open

Routers and Selectors. Part 1 if #802

emil14 opened this issue Dec 13, 2024 · 3 comments
Assignees

Comments

@emil14
Copy link
Collaborator

emil14 commented Dec 13, 2024

This is a first part in the series of drafts that explores the syntax-level features needed to handle dataflow use cases, beyond the already supported capabilities like fan-in/out, binary/ternary operations, ranges, and struct selectors.

We focus on two main categories of features:

  1. Routers - Components that choose a direction for message flow
  2. Selectors - Components that choose a specific value

Routers:

  • If - Basic conditional routing
  • Switch - Multi-branch routing
  • Race - Order-based routing

Selectors:

  • Match - Value-based selection
  • Select - Order-based selection

These components can be further categorized by how they make decisions:

  1. Value-based - Uses the content of messages to make choices (If, Switch, Match)
  2. Order-based - Uses message arrival order/source to make choices (Race, Select)

An important design principle is that all these features must be race-condition free and intuitive to use. For example, instead of extended switch form, deferred connections could theoretically be used, but they introduce intermediate lock nodes with latency that can cause race conditions. By providing first-class routing constructs instead, we ensure safe and predictable behavior without needing complex locking mechanisms.

If

Basic

If is the simplest router that receives a boolean message and routes it into two different directions based on its value.

sender -> if {
    then -> then_receiver
    else -> else_receiver
}

Example

true -> if {
    then -> println
    else -> panic
}

One Branch, Many Receivers

If supports multiple receivers per branch, which are handled like a fan-out. Keep in mind that the If component will not receive new messages until the previous ones are handled. Therefore, the sender of the If will be blocked by the slowest receiver in a branch.

sender -> if {
    then -> [then_receiver1, then_receiver2]
    ...
}

Example

job:failed -> if {
    then -> log:fatal
    else -> [log:info, db:write]
}

With Final Receiver

The If component has an extended form where its body is connected to a receiver. In this case, If sends a message to the selected branch receiver first, and then, after that receiver has received the message, it sends to the final receiver.

It is important to note that the basic If, after receiving a boolean message, is blocked only by the selected branch receiver(s). However, If with a final receiver is also blocked by that final receiver. Similarly, final receiver is blocked by the selected branch receiver.

This setup allows answering the question "when has the selected branch received the message?" but not "when has the selected node finished processing?" For example, sending a message to println will start printing, but in if {...} -> final_receiver, the final receiver has no guarantee that printing is finished.

sender -> if {
    then -> then_receiver
    else -> else_receiver
} -> final_receiver

One might think that this form of if could be emulated by putting final_receiver into branches

// this is incorrect
sender -> if {
    then -> [then_receiver, final]
    else -> [else_receiver, final]
}

This is not true. It has different semantics because the selected and final receivers will receive messages concurrently, not sequentially. Additionally, it is forbidden to refer to the same receiver twice.

Example

(:user.age > 18) -> if {
    then -> greet
    else -> show_banner
} -> println

However, an If component with a final receiver can also have multiple receivers per branch, and even multiple final receivers. They are handled like a fan-out and affect the If component in the same way in terms of blocking.

sender -> if {
    then -> [then_receiver1, then_receiver2]
    else -> else_receiver
} -> [final_receiver1, final_receiver1]

Example

:user.paid -> if {
    then -> [greet, println]
    else -> log:fatal
} -> [db, cache]

All If features, including those that are described below, can be combined in various ways. For brevity, we won't demonstrate all combinations (only some of them), but all forms of If follow this semantics: The If component always waits for its inports first, then waits for the selected receivers, and finally waits for the final receivers. It does not receive the next messages until the previous iteration is complete.

With Two Inports

Previous forms of If had only one inport, which is for the condition. It's possible to use if with 2 inports: one for the condition and one for the data. The purpose is to route one message based on the value of another message.

The condition sender has been moved from the left part of the expression to the right. This might seem confusing at first, but it is done to avoid even greater confusion, given how we are accustomed to seeing if-statements in other programming languages.

data_sender -> if cond_sender {
    then -> then_receiver
    else -> else_receiver
}

Example

42 -> if true {
    then -> println // prints 42, not true
    else -> panic // would panic with 42, not false
}

The If component can have both a final receiver and 2 inports at the same time:

data_sender -> if cond_sender {
    then -> then_receiver
    else -> else_receiver
} -> final_receiver

Example

:user -> if :theme {      // if doesn't care if what order user and theme came, it waits for both
    dark -> respect:inc   // either foo or bar receives 42, not true or false
    light -> respect:decr // after foo or bar recieved, IfResult<int> is sent to println
} -> println              // after println receiver, if is able to receive next condition and data

The message sent to the final receiver in this case will have the type IfResult. This type allows access to both the condition and the data:

pub type IfResult<T> struct {
    cond bool
    data T
}

Example

{ cond: true, data: 42 }

Selector

Like every router, If has a form where, in addition to routing, it also performs selecting. It allows selecting both the message to send and the direction to send it.

Although it is a natural extension for previous form of If "with two inports" the syntax resembles a basic If, moving the condition sender back to the left.

cond_sender -> if {
    then: data_then_sender -> then_receiver
    else: data_else_sender -> else_receiver
}

There are no chained or deferred connections involved in this example. Everything happens "atomically" - If receives messages from all senders, selects the message and branch, and sends it to the corresponding receiver. This is race-free.

This form of if also supports a final receiver. The message it will receive will also have the type IfResult<T>. In this case, data will be the message from the selected data sender.

cond_sender -> if {
    then: data_then_sender -> then_receiver
    else: data_else_sender -> else_receiver
} -> receiver

You can figure out which branch was selected by looking at if { ... } -> .cond -> ... boolean value

This form of If also supports multiple branch receivers. Each branch receiver will receive a message from selected branch-sender, final receiver will receive IfResult message with selected message.

cond_sender -> if {
    then: data_then_sender -> [then_receiver1, then_receiver2]
    ...
} -> receiver

As a pure router, If has only one sender that blocks it - the condition sender. After receiving a condition, it is blocked only by the receivers (first the selected receivers, then the final receivers). As a hybrid that also selects, it is also blocked by data senders. Like any other selector, it will wait for all data senders in each iteration, as this is the only way to synchronize computation into iterations.

@emil14 emil14 added the Idea label Dec 13, 2024
@emil14 emil14 self-assigned this Dec 13, 2024
@emil14 emil14 changed the title Routers and selectors comprehensive syntax Routers and Selectors Dec 13, 2024
@emil14
Copy link
Collaborator Author

emil14 commented Dec 14, 2024

bumble — Today, at 09:42

(:user.age > 18) -> if {
    then -> greet
    else -> show_banner
} -> println

if the flow started from 'if' rather than condition ':user.age > 18', it would feel more intuitive for me as a reader. This may simply be because the syntax is unfamiliar to me
otherwise, the flows seem clear

@emil14
Copy link
Collaborator Author

emil14 commented Dec 14, 2024

if the flow started from 'if' rather than condition ':user.age > 18', it would feel more intuitive

So it would look like this

if (:user.age > 18) {
    then -> greet
    else -> show_banner
} -> println

And extended form with two inports would look like this

:data -> if (:user.age > 18) {
    then -> greet
    else -> show_banner
} -> println

I like that condition is on the right in both forms, unlike existing draft


My concern is that you can't chain if this way. For example (existing draft)

user -> is_admin -> if { // is_admin is chained
    ...
}

VS if we move condition to the right

user -> is_admin // send to is_admin node
if is_admin { // receive from is_admin node
    ...
}

But that is also true for an extended form with 2 inports (in both my and your design) - since condition is on the right, you can't chain it

user -> [
    is_admin, // send `user` to `is_admin` node
    if is_admin {...} // send `user` to `if` and receive from `is_admin` inside `if`
]

in this example is_admin condition is not chained because of that


TLDR: it makes if easy to understand for newcomers (because condition is on the right like we used to) and also makes it more consistent (because condition is always on the right and not only in extended form), but it damages its flexibility (because it's not possible to chain it)


UPD: This statement is true for all routers, because all of them support extended form with 2 inports

@emil14 emil14 changed the title Routers and Selectors Routers and Selectors. Part 1 if Dec 14, 2024
@emil14
Copy link
Collaborator Author

emil14 commented Dec 14, 2024

One addition that can be made to "selector-mode" is that some branches can remain "simple", without dedicated data sender. It can be useful if we want to send condition message in one case, but data message in another.

cond_sender -> if {
    then: data_then_sender -> then_receiver // receives data message
    else -> else_receiver // receives condition message
}

This is especially useful due to the fact that we can't refer to the same sender twice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant