Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Experimental gRPC server #16534

Draft
wants to merge 16 commits into
base: main
Choose a base branch
from
Draft

Experimental gRPC server #16534

wants to merge 16 commits into from
+1,933 −108

Conversation

finnegancarroll
Copy link
Contributor

@finnegancarroll finnegancarroll commented Oct 31, 2024 •
edited
Loading

Description

Auxiliary transports

Provides a framework for implementing pluggable auxiliary transports through NetworkPlugin.
Auxiliary transports run in parallel to existing http & native transport servers.
None or many Auxiliary transports may be provided and can be configured with the following settings:

aux.transport.types enables all listed transports which are installed.
aux.transport.<type>.ports configures the port range for the specified transport.

In the absence of aux.transport.<type>.ports the auxiliary transport falls back to a default port range (9400-9500).
If aux.transport.types is set and the corresponding transport plugin is not installed an exception is thrown.

transport-grpc plugin

A gRPC server plugin is implemented as an initial auxiliary transport.
This gRPC server includes only default list and Health/Check services.

To enable the gRPC plugin include the following in run.gradle.

setting 'aux.transport.types', '[grpc-transport]'
setting 'aux.transport.grpc-transport.ports', '9400-9500' //optional

Then run OpenSearch with the plugin installed.

./gradlew run -PinstalledPlugins="['transport-grpc']"

grpcurl commands for querying default services.

grpcurl -plaintext localhost:9400 list
grpcurl -plaintext localhost:9400 grpc.health.v1.Health/Check

Related Issues

RFCs detailing generation of protobuf files from api-specification - 1 2.

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for b5fddb1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 24a40ad: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@finnegancarroll finnegancarroll mentioned this pull request Nov 4, 2024
Open
1 task
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 7, 2024

❌ Gradle check result for 3bcd5bb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 7, 2024

❌ Gradle check result for b2a1078: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 8, 2024

❌ Gradle check result for ee85066: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for eda1c04: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 5038cf3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 1e5d354: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 00d4eb1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 6151622: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for b8d88d2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for b404cee: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 41e70eb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for e985043: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for bbabecd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 853b206: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for da16c66: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@finnegancarroll finnegancarroll changed the base branch from main to feature/grpc November 23, 2024 00:17
@finnegancarroll finnegancarroll marked this pull request as ready for review November 23, 2024 00:24
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 8a3f634: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 1a9170a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@reta
Copy link
Collaborator

reta commented Dec 16, 2024

Let me know if this is what you had in mind for this interface.

Looks great, thanks @finnegancarroll , one comment though - I think we should make the assumption there could be as many aux as necessary (in other words, not one but a list).

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for e1879ea: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@finnegancarroll
Refactor auxiliary transport port settings framework.
3ac47a6 
The motivation of this change is to allow for multiple configured
auxiliary transports. As such newAuxTransports() now returns a list
of enabled transports and each AuxTransport implements it's own
'aux.transport.type' and 'aux.transport.<type>.ports' setting. Since
Security.java initializes previous to Node.java during bootstrap
socket binding permissions are granted based on
'aux.transport.<type>.ports' for each enabled 'aux.transport.type',
falling back to a default if no ports are specified.

Signed-off-by: Finn Carroll <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 3ac47a6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@finnegancarroll
Copy link
Contributor Author

Let me know if this is what you had in mind for this interface.

Looks great, thanks @finnegancarroll , one comment though - I think we should make the assumption there could be as many aux as necessary (in other words, not one but a list).

Added a slight refactor to support this. Settings specific to each AuxTransport have been moved inside the plugin with core only exposing two auxiliary transport related settings:

'aux.transport.type'
'aux.transport.<type>.ports' //optional

Port permissions are granted based on 'aux.transport.<type>.ports' filtering for each 'aux.transport.type'.

@finnegancarroll
Fix typos:
e0753f9 
getAuxServerTransportSupplierList -> getAuxServerTransportList
aux.transport.type -> aux.transport.types

Signed-off-by: Finn Carroll <[email protected]>
@finnegancarroll
Replae redundant resolvePublishPort() with helper.
0056342 
Signed-off-by: Finn Carroll <[email protected]>
@finnegancarroll
Remove gRPC feature flag. Redundant for a plugin.
f0932e9 
Signed-off-by: Finn Carroll <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for f0932e9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 8c83da1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 759c434: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 9c24e1d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 9c24e1d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 9c24e1d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@opensearch-ci-bot opensearch-ci-bot mentioned this pull request Dec 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@karenyrx karenyrx karenyrx left review comments

@amberzsy amberzsy amberzsy left review comments

@andrross andrross andrross left review comments

@reta reta reta left review comments

@peternied peternied Awaiting requested review from peternied peternied is a code owner

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@ashking94 ashking94 Awaiting requested review from ashking94 ashking94 is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock dblock is a code owner

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@jainankitk jainankitk Awaiting requested review from jainankitk jainankitk is a code owner

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@linuxpi linuxpi Awaiting requested review from linuxpi linuxpi is a code owner

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

@nknize nknize Awaiting requested review from nknize nknize is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@VachaShah VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@finnegancarroll @karenyrx @andrross @reta @amberzsy