Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Jpleger/sqs #30

Open
wants to merge 47 commits into
base: release/0.1.0
Choose a base branch
from
Open

Jpleger/sqs #30

wants to merge 47 commits into from

Conversation

jeanpatrickleger
Copy link
Contributor

Fixes # .

Description

This is a pull request for AWS SQS ( Simple Queue Service ) support. This proposes an alternative option to using RMQ. The documentation about the activation of the plugin and its configuration can be found here : https://github.com/Project-MONAI/monai-deploy-messaging/blob/jpleger/SQS/src/Messaging/SQS/README.MD

Status

Ready/Work in progress/Hold

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • All tests passed locally by running ./src/run-tests-in-docker.sh.
  • Documentation comments included/updated.
  • User guide updated.
  • I have updated the changelog

mocsharp and others added 14 commits April 5, 2022 16:45
@mocsharp
Update release job
2bb7035
@mocsharp
Skip publishing duplicate nuget packages
367882e
@mocsharp
Requires manual trigger to deploy to NuGet.org
05376c0
@mocsharp
Update ci.yml
a852ec7 
Publish & close release only on main branch.
@mocsharp
Update release job
da1c02b
@mocsharp
Skip publishing duplicate nuget packages
bfafff0
@mocsharp
Requires manual trigger to deploy to NuGet.org
529d9c5
@mocsharp
Change target framework to .net standard 2.1
5fff089
@mocsharp
Merge branch 'release/0.1.0' into vchang/netstandard
d1450c8
@mocsharp
Merge pull request #2 from Project-MONAI/vchang/netstandard
776b217 
Target .NET Standard 2.1
@mocsharp
Update GitVersion.yml
f90bf2e
@mocsharp
Update GitVersion.yml
649c656
@mocsharp
Update ci.yml
fd4e2f6
@mocsharp
[Breaking] Unit tests for RabbitMQ plugin (#4)
ffcf29a 
* Implement unit tests for RabbitMQ plugin
* Fix sonar code smells
* Update PR template

Signed-off-by: Victor Chang <[email protected]>
@codecov
Copy link

codecov bot commented Jun 14, 2022 •
edited
Loading

Codecov Report

Merging #30 (ed2f2b5) into release/0.1.0 (ffcf29a) will decrease coverage by 27.11111%.
The diff coverage is 53.31412%.

Impacted file tree graph

@@                   Coverage Diff                    @@
##           release/0.1.0         #30          +/-   ##
========================================================
- Coverage       93.77778%   66.66667%   -27.11112%     
========================================================
  Files                 19          36          +17     
  Lines                450        1065         +615     
  Branches              15          70          +55     
========================================================
+ Hits                 422         710         +288     
- Misses                24         343         +319     
- Partials               4          12           +8
Flag Coverage Δ
unittests 66.66667% <53.31412%> (-27.11112%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/Messaging/Common/Log.cs 100.00000% <ø> (ø)
.../Messaging/Configuration/ConfigurationException.cs 50.00000% <ø> (+25.00000%) ⬆️
src/Messaging/Messages/MessageBase.cs 100.00000% <ø> (ø)
...rc/Messaging/RabbitMq/RabbitMqConnectionFactory.cs 0.00000% <0.00000%> (ø)
src/Messaging/SQS/ConfigurationKeys.cs 0.00000% <0.00000%> (ø)
src/Messaging/SQS/Log.cs 0.00000% <0.00000%> (ø)
src/Messaging/SQS/QueueFormatter.cs 0.00000% <0.00000%> (ø)
src/Messaging/SQS/SQSMessagePublisherService.cs 0.00000% <0.00000%> (ø)
src/Messaging/SQS/SQSMessageSubscriberService.cs 0.00000% <0.00000%> (ø)
src/Messaging/Common/MessageConversionException.cs 25.00000% <25.00000%> (ø)
... and 30 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffcf29a...ed2f2b5. Read the comment docs.

mocsharp and others added 15 commits June 14, 2022 09:41
@mocsharp @jeanpatrickleger
Event validation & add Task related events (#7)
367ce0f 
* Rename Message to Event used in the message body.
* Add validation to events and validate event messages on all properties recursively.

Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@jackschofield23 @jeanpatrickleger
Add subscription no requeue reject (#9)
5a314d3 
Signed-off-by: JP LEGER <[email protected]>
@mocsharp @jeanpatrickleger
New RunnerCompleteEvent (#10)
e3c4c91 
- Add RunnerCompleteEvent
- Rename TaskCompleteEvent to TaskUpdateEvent with new fields and changes.
- Allow multiple inputs/outputs for TaskDispatchEvent
- Add Accepted TaskStatus

Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@mocsharp @jeanpatrickleger
Add additional failure reasons (#11)
7fe7ce0 
Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@mocsharp @jeanpatrickleger
Update FailureReason.cs
2402d20 
Signed-off-by: JP LEGER <[email protected]>
@mocsharp @jeanpatrickleger
Ability to convert Message to JsonMessage<T> (#13)
bfc08d9 
* Ability to convert Message to JsonMessage<T>

Signed-off-by: Victor Chang <[email protected]>

* Add SubscribeAsync with a Func<> callback

Signed-off-by: Victor Chang <[email protected]>

* Update unit test for SubscribeAsync

Signed-off-by: Victor Chang <[email protected]>

* Use existing RabbitMQ connection if available, otherwise, create a new one.

Signed-off-by: Victor Chang <[email protected]>

* Fix code smells

Signed-off-by: Victor Chang <[email protected]>

* Breaking: IRabbitMqConnectionFactory to CreateChannel and grantee a single connection is used

Signed-off-by: Victor Chang <[email protected]>

* APIs to bind multiple topics to a single queue

Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@mocsharp @jeanpatrickleger
Handle corrupted messages (#12)
7909d03 
* Rename RunnerCompleteEvent to TaskCallbackEvent with additional Metadata property.
* Handle corrupted messages
* Check message properties
* Use RabbitMQ Timestamp
* Declare exchanges are durable & !auto delete
* Fix JsonProperty mapping
* Include metadata in the TaskUpdateEvent

Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@mocsharp @jeanpatrickleger
Update TaskUpdateEvent.cs
19b141d 
Signed-off-by: JP LEGER <[email protected]>
@jackschofield23 @jeanpatrickleger
update task dispatch event
6436c66 
Signed-off-by: jackschofield23 <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@mocsharp @jeanpatrickleger
Make credentials optional in TaskDispatchEvent.Storage (#17)
50c66fc 
Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@jackschofield23 @jeanpatrickleger
change task status name
bdb5be7 
Signed-off-by: jackschofield23 <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@jackschofield23 @jeanpatrickleger
Rename workflow instance id (#20)
f637151 
Signed-off-by: jackschofield23 <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@mocsharp @jeanpatrickleger
Intermediate Storage for TaskDispatchEvent (#21)
5f360a7 
* Add intermediate storage for TaskDispatchEvent
* Implement ICloneable for Storage

Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@mocsharp @jeanpatrickleger
Throw MessageConversionException on error converting JSON event messa…
64437ab 
…ge (#22)

Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@jackschofield23 @jeanpatrickleger
update taskupdate to include output artifacts
310d005 
Signed-off-by: Jack Schofield <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
jackschofield23 and others added 10 commits June 14, 2022 09:41
@jackschofield23 @jeanpatrickleger
update task callback event (#24)
1761aad 
Signed-off-by: Jack Schofield <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@jackschofield23 @jeanpatrickleger
fix task callback and task update event (#25)
8f63ccd 
Signed-off-by: Jack Schofield <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@jackschofield23 @jeanpatrickleger
add payload id to task dispatch (#26)
b076491 
Signed-off-by: Jack Schofield <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@jackschofield23 @jeanpatrickleger
add export status (#28)
c167be8 
Signed-off-by: Jack Schofield <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@jeanpatrickleger @awsjpleger
Options to enable secure RabbitMQ (#27)
16a53b7 
* RMQ connection support for SSL/TLS (no certificate check).
* Refactored RabbitMqConnectionFactory to support TLS connection and Port number config.
* Update on IRabbitMqConnectionFactory Summary.

Co-authored-by: JP Leger <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@jeanpatrickleger
Added SQS plug-in as RMQ alternative.
d0ba49e 
Signed-off-by: JP LEGER <[email protected]>
@jeanpatrickleger
SQS Support adde to messaging lib
bc71ebd 
Signed-off-by: JP LEGER <[email protected]>
@jeanpatrickleger
Code analysis fixes.
4edabf0 
Signed-off-by: JP LEGER <[email protected]>
@awsjpleger @jeanpatrickleger
remove unecessary folder.
8cdb394 
Signed-off-by: JP Leger <[email protected]>
Signed-off-by: JP LEGER <[email protected]>
@jeanpatrickleger
problematic README.MD removed
75973f0 
Signed-off-by: JP LEGER <[email protected]>
@mocsharp mocsharp force-pushed the release/0.1.0 branch 2 times, most recently from e08d9a2 to 6b0d721 Compare June 27, 2022 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mocsharp mocsharp Awaiting requested review from mocsharp

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jeanpatrickleger @mocsharp @jackschofield23 @awsjpleger