-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC - Add the DataAdapter pattern to the SQS Stream Provider #8723
base: main
Are you sure you want to change the base?
RFC - Add the DataAdapter pattern to the SQS Stream Provider #8723
Conversation
Could a core member check if the SQS tests are actually getting run in CI? If you pull master and run |
f4306cc
to
b8aface
Compare
Just rebased this ontop of main for Orleans 8.0.0 Would appreciate someone looking to see if we can merge this PR. |
@benjaminpetit Any possibility of getting this change into Orleans 8.1.0? |
5087395
to
f4bb41f
Compare
SqsOptions are now passed down to SQSStorage to be accessible
0b40d64
to
519eb4d
Compare
Hey Orleans team, Is there something that is designed wrong in this PR that is preventing us from pulling it into Orleans? I would really like to get away from having to rebase on each Orleans release and maintaining a fork for our apps. I'm sure being able to support Native -> Orleans streams via SQS would be useful for others. Let me know if I need to alter this PR in any way to get it included. |
The EventHub StreamProvider has a concept of a DataAdapter, where the user can inject their own implemention to parse messages for the queue, specifically for the purpose of picking up messages from sources that are not Orleans, and have not already passed through the StreamProvider.
I am attempting to implement this in the SQS provider, as I need similar functionality.
Request for comments / code review on this PR, I need to pull some more data out to configuration still.
The first commit was required to get the SQS tests passing, before I moved on to the commit with the Data Adapter change.
Contract changes:
internal
topublic
(as the DataAdapter needs to construct one, this is crux of the change).internal
topublic
as it is the only method of passing a SequenceToken to the object.Alternative approach, would be to have the consumer implement an
IBatchContainer
but it would be nice to have a default one.Microsoft Reviewers: Open in CodeFlow