-
Notifications
You must be signed in to change notification settings - Fork 602
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add interop.flow.pipeToProcessor
#3449
Open
BalmungSan
wants to merge
10
commits into
typelevel:main
Choose a base branch
from
BalmungSan:flow-processor
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
e4b45a7
Add interop.flow.StreamProcessor and pipeToProcessor
BalmungSan dee368d
Add missing header for StreamProcessor.scala
BalmungSan 9fb2f6e
Fix StreamProcessor.subscribe signature
BalmungSan f5c50c8
Add interop.flow.pipeToProcessor Scaladoc
BalmungSan 685dcd9
Add interop.flow.unsafePipeToProcessor
BalmungSan 42115b5
Fix interop.flow.StreamProcessor.unsafeFromPipe in JS
BalmungSan 2e7fc2f
Add interop.flow.StreamProcessorSpec
BalmungSan 9ae7a62
Add missing header for StreamProcessorSpec.scala
BalmungSan 55f5e49
Add toProcessor syntax to Pipe
BalmungSan 24a844d
Merge branch 'main' into flow-processor
BalmungSan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
44 changes: 44 additions & 0 deletions
44
core/shared/src/main/scala/fs2/interop/flow/StreamProcessor.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
package fs2 | ||
package interop | ||
package flow | ||
|
||
import java.util.concurrent.Flow | ||
import cats.effect.{Async, Resource} | ||
|
||
private[flow] final class StreamProcessor[F[_], I, O]( | ||
streamSubscriber: StreamSubscriber[F, I], | ||
streamPublisher: StreamPublisher[F, O] | ||
) extends Flow.Processor[I, O] { | ||
override def onSubscribe(subscription: Flow.Subscription): Unit = | ||
streamSubscriber.onSubscribe(subscription) | ||
|
||
override def onNext(i: I): Unit = | ||
streamSubscriber.onNext(i) | ||
|
||
override def onError(ex: Throwable): Unit = | ||
streamSubscriber.onError(ex) | ||
|
||
override def onComplete(): Unit = | ||
streamSubscriber.onComplete() | ||
|
||
override def subscribe(subscriber: Flow.Subscriber[? >: O <: Object]): Unit = | ||
streamPublisher.subscribe(subscriber) | ||
} | ||
|
||
private[flow] object StreamProcessor { | ||
def fromPipe[F[_], I, O]( | ||
pipe: Pipe[F, I, O], | ||
chunkSize: Int | ||
)(implicit | ||
F: Async[F] | ||
): Resource[F, StreamProcessor[F, I, O]] = | ||
for { | ||
streamSubscriber <- Resource.eval(StreamSubscriber[F, I](chunkSize)) | ||
inputStream = streamSubscriber.stream(subscribe = F.unit) | ||
outputStream = pipe(inputStream) | ||
streamPublisher <- StreamPublisher(outputStream) | ||
} yield new StreamProcessor( | ||
streamSubscriber, | ||
streamPublisher | ||
) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't have access to the upstream
Publisher
yet, so we can't do thesubscribe
here. So we do nothing and wait for it.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not entirely sure I understand the waiting part. Can/should we use a
Deferred
here?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No sorry, my point is that
subscribe
should be something likeIO(publisher.subscribe(streamSubscriber))
as you can see in the implementation ofsyntax.fromPublisher
.Here, however, we don't have a
Publisher
yet, so we don't control when we are published.Basically, this is very similar to the
fromPublisher
overload that accepts aSubscriber => F[Unit]
, just that rather than receiving the lambda, we are just returning a rawProcessor / Subscriber
.