Streams

[benjaminbrumbaugh]

👋 I'm looking into a couple of possible services for managing the workflows of a KaftkaStreams-centric (or possibly Pulsar) K8s serverless architecture. I'm really liking Flyte so far, and am hoping to be able to use it. But I'm shooting to do end-to-end stream processing, where the flow of information pauses as little as possible. I can't seem to confirm or deny if Flyte allows for stream processing. The other possible engines I'm currently looking into are Argo, Nifi, Airflow, and Kubeflow. Of them, I've only seen documentation in Nifi that indicates that it supports streams. Does anyone have insight into Flyte stream processing?

Thank you!
Ben