MicroProfile Reactive Messaging 3.0 Support

[Emily-Jiang]

Description

The current MicroProfile Reactive Messaging 1.0 does not work with MicroProfile 4.x. Customers that use RM 1.0 are stuck. Quite a few customers asked about when RM 2.0 will be supported in Open Liberty many times (I was asked at JChampion conference 2022 on Friday 21st Jan 2022).

We should directly support the recent release of RM 3.0 (Jakarta EE 9.1 alignment) to make it work with Jakarta EE 10 and MP 6.0.

---

Documents

When available, add links to required feature documents. Use "N/A" to mark particular documents which are not required by the feature.

• Aha: Externally raised RFE (Aha)
  • Link the RFE with this issue
• UFO: https://ibm.box.com/s/bffbods91oeddvxgbb5gpodq6m8x4ymy
• FTS: (MP Reactive Messaging 3.0 Feature Test Summary #25853)
• Beta Blog: (BETA BLOG - MicroProfile Reactive Messaging and Reactive Stream Operators 3.0 #26642)
• GA Blog: (GA BLOG - Microprofile Reactive Messaging and Reactive Stream Operators 3.0 #27213)

---

Process Overview

• Prioritization
• Design
• Implementation
• Legal and Translation
• Beta
• GA
  • Focal Point Approvals
• Other Deliverables

General Instructions

The process steps occur roughly in the order as presented. Process steps occasionally overlap.

Each process step has a number of tasks which must be completed or must be marked as not applicable ("N/A").

Unless otherwise indicated, the tasks are the responsibility of the Feature Owner or a Delegate of the Feature Owner.

If you need assistance, reach out to the OpenLiberty/release-architect.

Important: Labels are used to trigger particular steps and must be added as indicated.

---

Prioritization (Complete Before Development Starts)

The (OpenLiberty/chief-architect) and area leads are responsible for prioritizing the features and determining which features are being actively worked on.

Prioritization

• Feature added to the "New" column of the Open Liberty project board
  • Epics can be added to the board in one of two ways:
    • From this issue, use the "Projects" section to select the appropriate project board.
    • From the appropriate project board click "Add card" and select your Feature Epic issue
• Priority assigned
  • Attend the Liberty Backlog Prioritization meeting

---

Design (Complete Before Development Starts)

Design preliminaries determine whether a formal design, which will be provided by an Upcoming Feature Overview (UFO) document, must be created and reviewed. A formal design is required if the feature requires any of the following: UI, Serviceability, SVT, Performance testing, or non-trivial documentation/ID.

Design Preliminaries

• UI requirements identified. (Owner and UI focal point)
• ID requirements identified. (Owner and ID focal point)
  • Refer to Documenting Open Liberty.
  • Feature Owner adds label ID Required, if non-trivial documentation needs to be created by the ID team.
  • ID adds label ID Required - Trivial, if no design will be performed and only trivial ID updates are needed.
• Serviceability Requirements Identified. (Owner and Serviceability focal point)
• SVT Requirements Identified. (Owner and SVT focal point)
• Performance testing requirements identified. (Owner and Performance focal point)

Design

• POC Design / UFO review requested.
  • Owner adds label Design Review Request
• POC Design / UFO review scheduled.
  • Follow the instructions in POC-Forum repo
• POC Design / UFO review completed.
• POC / UFO Review follow-ons completed.
• POC Design / UFO approval requested.
  • Owner adds label Design Approval Request
• Design / UFO approved. (OpenLiberty/chief-architect) or N/A
  • (OpenLiberty/chief-architect) adds label Design Approved
  • Add the public link to the UFO in Box to the Documents section.
  • The UFO must always accurately reflect the final implementation of the feature. Any changes must be first approved. Afterwards, update the UFO by creating a copy of the original approved slide(s) at the end of the deck and prepend "OLD" to the title(s). A single updated copy of the slide(s) should take the original's place, and have its title(s) prepended with "UPDATED".

No Design

• No Design requested.
  • Owner adds label No Design Approval Request
• No Design / No UFO approved. (OpenLiberty/chief-architect) or N/A
  • Approver adds label No Design Approved
• Feature / Capability stabilization or discontinuation or N/A
  • Owner adds label Product Management Approval Request and notifies OpenLiberty/product-management
  • Approver adds label Product Management Approved (OpenLiberty/product-management)
  • Note: For stabilized, superseded, and discontinued feature/capability, skip the Beta section of the template (you may delete it). Otherwise, proceed as normal.

FAT Documentation

• "Feature Test Summary" child task created
  • Use the Feature Test Summary Template
  • Add FTS issue link to the Documents section.

---

Implementation

A feature must be prioritized before any implementation work may begin to be delivered (inaccessible/no-ship). However, a design focused approach should still be applied to features, and developers should think about the feature design prior to writing and delivering any code.
Besides being prioritized, a feature must also be socialized (or No Design Approved) before any beta code may be delivered. All new Liberty content must be inaccessible in our GA releases until it is Feature Complete by either marking it kind=noship or beta fencing it.
Code may not GA until this feature has obtained the "Design Approved" or "No Design Approved" label, along with all other tasks outlined in the GA section.

Feature Development Begins

• Add the In Progress label
• Create mpReactiveStreams-3.0 feature #25385
• Create mpReactiveMessaging-3.0 feature #25386
• Run the MP Reactive 3.0 TCK #25388
• Add Repeats to MP Reactive Streams FATs #25389
• Add Repeats to MP Reactive Messaging FATs #25390
• Test MP Reactive Streams 3.0 with EE10 #25423
• Test MP Reactive Messaging 3.0 with EE10 #25424
• Check if the RSO integration with SmallRye Mutiny still needs a PrivilidgedAction #25450
• Run the MicroProfile RSO 3.0 TCK #25644
• Update Reactive Messaging Tests to use latest Kafka Client version #25645
• Resolve MP Reactive Messaging API Bundling #25852
• Add tests for MP RM 3.0 Nack with kafka connector #25969
• Add Tests for MP RM 3.0 Emitters with kafka connector #25970
• Check review comments on RSO 3.0 TCK PR #25975
• Set Smallrye.reactive.messaging to be in strict mode #26082
• MP RM 3.0 metrics #26204
• Support MP Reactive Messaging 3.0 with InstantOn #26220
• Add Tests for MP RM 3.0 with mpMetrics integration #26243
• Add Reactive Messaging 3.0 tests for Validation Errors. #26274
• Review asynchronous execution in Reactive Streams 3.0 #26352
• Create test cases that covers RM with other MP features such as mp Open Tracing and Open Telemetry for MP50, 60 and 61 #26353
• Improve messages for MP RM Kafka Connector #26428
• Override Smallrye exception logging to surface supressed messages #26664
• Enable Metrics tests for Emitter channels #26665
• Async improvements for Reactive Messaging 3.0 #26678
• Reactive Messaging handling of invalid context service configuration #27189
• Kafka message acknowledgement is slow if messages are fed to a subscriber method #26692
• Add tests for testing MTLS with RM Liberty-Kafka Connector #26852

Legal and Translation

In order to avoid last minute blockers and significant disruptions to the feature, the legal items need to be done as early in the feature process as possible, either in design or as early into the development as possible. Similarly, translation is to be done concurrently with development. Both MUST be completed before Beta or GA is requested.

Legal (Complete before Feature Complete Date)

• Changed or new open source libraries are cleared and approved, or N/A. (Legal Release Services/Cass Tucker/Release PM).
• Licenses and Certificates of Originality (COOs) are updated, or N/A

Translation (Complete 1 week before Feature Complete Date)

• PII updates are merged, or N/A. Note timing with translation shipments.

Innovation (Complete 1 week before Feature Complete Date)

• Consider whether any aspects of the feature may be patentable. If any identified, disclosures have been submitted.

---

Beta

In order to facilitate early feedback from users, all new features and functionality should first be released as part of a beta release.

Beta Code

• Beta fence the functionality
  • kind=beta, ibm:beta, ProductInfo.getBetaEdition()
• Beta development complete and feature ready for inclusion in a beta release
  • Add label target:beta and the appropriate target:YY00X-beta (where YY00X is the targeted beta version).
• Feature delivered into beta
  • (OpenLiberty/release-manager) adds label release:YY00X-beta (where YY00X is the first beta version that included the functionality).

Beta Blog (Complete 1.5 weeks before beta eGA)

• Beta blog issue created and populated using the Open Liberty BETA blog post template.
  • Add a link to the beta blog issue in the Documents section.
  • Note: This is for inclusion into the overall beta release blog post. If, in addition, you'd also like to create a dedicated blog post about your feature, then follow the "Standalone Feature Blog Post" instructions under the Other Deliverables section.

---

GA

A feature is ready to GA after it is Feature Complete and has obtained all necessary Focal Point Approvals.

Feature Complete

• Feature implementation and tests completed.
  • All PRs are merged.
  • All related/child issues are closed.
  • All stop ship issues are completed.
• Legal: all necessary approvals granted.
• Translation: Feature may only proceed to GA if it has either Translation - Complete or Translation - Missing label
  • If all translation has been delivered to release branch, feature owner adds label Translation - Complete.
  • If missing translation does not cause a break in functionality, nor a security or production outage risk, feature owner adds label Translation - Missing.
    • Once all missing translations are delivered, the Translation - Missing label is replaced with Translation - Complete.
  • If missing translation could cause a break in functionality or a security or production outage risk, feature owner adds the Translation - Blocked label.
    • Featues with Translation - Blocked may NOT proceed to GA until the label has been replaced with either Translation - Missing or Translation - Complete.
  • For further guidance, contact Globalization focal point or the Release Architect.
• GA development complete and feature ready for inclusion in a GA release
  • Add label target:ga and the appropriate target:YY00X (where YY00X is the targeted GA version).
  • Inclusion in a release requires the completion of all Focal Point Approvals.

Focal Point Approvals (Complete by Feature Complete Date)

These occur only after GA of this feature is requested (by adding a target:ga label). GA of this feature may not occur until all approvals are obtained.

All Features

• APIs/Externals Externals have been reviewed or N/A. (OpenLiberty/externals-approvers)
  • Approver adds label focalApproved:externals
• Demo Demo is scheduled for an upcoming EOI or N/A. (OpenLiberty/demo-approvers)
  • Add comment @OpenLiberty/demo-approvers Demo scheduled for EOI [Iteration Number] to this issue.
  • Approver adds label focalApproved:demo.
• FAT All Tests complete and running successfully in SOE or N/A. (OpenLiberty/fat-approvers)
  • Approver adds label focalApproved:fat.

Design Approved Features

• ID - Documentation is complete or N/A. (OpenLiberty/id-approvers)
  • Approver adds label focalApproved:id.

  • NOTE: If only trivial documentation changes are required, you may reach out to the ID Feature Focal to request a ID Required - Trivial label. Unlike features with regular ID requirement, those with ID Required - Trivial label do not have a hard requirement for a Design/UFO.

• InstantOn - InstantOn capable or N/A. (OpenLiberty/instantOn-approvers)
  • Approver adds label focalApproved:instantOn.
• Performance - Performance testing is complete or N/A. (OpenLiberty/performance-approvers)
  • Approver adds label focalApproved:performance.
• Serviceability - Serviceability has been addressed or N/A. (OpenLiberty/serviceability-approvers)
  • Approver adds label focalApproved:sve.
• STE - Skills Transfer Education chart deck is complete or N/A. (OpenLiberty/ste-approvers)
  • Approver adds label focalApproved:ste.
• SVT - System Verification Test is complete or N/A. (OpenLiberty/svt-approvers)
  • Approver adds label focalApproved:svt.

Remove Beta Fencing (Complete by Feature Complete Date)

• Beta guards are removed, or N/A
  • Only after all necessary Focal Point Approvals have been granted.

GA Blog (Complete by Feature Complete Date)

• GA Blog issue created and populated using the Open Liberty GA release blog post template.
  • Add a link to the GA Blog issue in the Documents section.
  • Note: This is for inclusion into the overall release blog post. If, in addition, you'd also like to create a dedicated blog post about your feature, then follow the "Standalone Feature Blog Post" instructions under the Other Deliverables section.

Post GA

• Replace target:YY00X label with the appropriate release:YY00X. (OpenLiberty/release-manager)

---

Other Deliverables

• Standalone Feature Blog Post A blog post specifically about your feature or N/A. (OpenLiberty/release-architect)
  • This should be strongly considered for larger or more prominent features.
  • Follow instructions in the blogs repo.
• OL Guides OL Guides assessment is complete or N/A. (OpenLiberty/guide-assessment)
• Dev Experience Developer Experience & Tools work is complete or N/A. (OpenLiberty/dev-experience-assessment)

---

[inad9300]

It's great to see this in the pipeline. Any estimation on when it will be available? What's the recommended solution in the meantime?

[Emily-Jiang]

We constantly revisit our features in the pipeline. For this feature, it is not high up in the pipeline as yet at the moment, due to resource constraints. Do you currently use MicroProfile reactive messaging 1.0 in the production? Unfortunately, the workaround is to stay with the older version of MicroProfile release if you want to use MicroProfile Reactive Message together with other MicroProfile features.

[inad9300]

Thanks for your prompt answer. We are not currently using this feature in production, but rather performing an evaluation of technologies for a new project. Unfortunately, I don't think it makes sense to begin a new project using MicroProfile 3.x, being two major versions behind now. Is there any other known workaround? For example, could we modify our pom.xml and depend directly on microprofile-reactive-messaging-api and smallrye-reactive-messaging-provider, or do you foresee any big issues with such an approach?

[jhanders34]

My notes from the UFO socialization that was done on September 8:

Java APIs / SPIs (page 17)

• Change metrics to mpMetrics in the second bullet

Emitter and @channel (page 19)

• Update the right example to be lower so you can see the word Message

Emitter and Channel Usage (page 21)

• Example should be updated. toUpperCase returns a value, but there is no @Outgoing so it can't return String or it needs @Outgoing added.

Backpressure Strategy (page 24)

• Make sure that the 128 message buffer default is highlighted better.

Negative Acknowledgment (Nack) (page 28)

• POST_PROCESSING implementation does the ack / nack. Make that more clear that implementation = Liberty. Maybe just added (Liberty) after the word implementation.

Admin / Config / Command Line (page 40)

• Provide details about what the sensible defaults are
• Confirm validity of liberty kafka connection bootstrap server line. bootstrap.servers vs bootstrap.server. Can provide multiple hosts/ports. I believe on the call we confirmed that it is servers and not server in the existing guides.

Automated Testing (page 46)

• Clarify how both user provided kafka connectors and the built-in liberty kafka connectors are covered. FATs I believe cover one and the TCK the other.
• FAT test for messages coming out for when validation fails for missing mp config properties for different configurations

Platform / cloud considerations (page 49)

• For Instant On, should have a statement that we will need to not start things that depend on the MP Config properties until restore. Open a GitHub issue for the Instant On folks to prioritize mpReactiveMessaging features enablement for Instant On so they can discuss on how to do this in the future outside of this UFO.

Serviceability (page 52)

• Update about messages for validation errors.
• Include a message when things get "stuck" when things are not acknowledged when MANUAL acknowledgement is configured and it is not done.

[NottyCode]

Slide 23 indicates a strategy of BUFFER has a limit, but the Javadoc from the screenshot indicates that all messages are buffered, which makes UNBOUNDED_BUFFER and BUFFER seem identical. This is perhaps a Javadoc bug from upstream?

How does this feature handle changes to the configuration?

In what way does the configuration processing of this feature block the possibility of InstantOn working?

[tngiang73]

@abutch3r : WASWIN L2 is good with STE slides. Thanks!

[abutch3r]

@OpenLiberty/demo-approvers Demo scheduled for EOI 24.03