Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for CloudEvents #422

Closed
jamesward opened this issue Oct 31, 2019 · 20 comments
Closed

Add Support for CloudEvents #422

jamesward opened this issue Oct 31, 2019 · 20 comments
Assignees
Labels
Milestone

Comments

@jamesward
Copy link

CloudEvents a CNCF standard for event payloads just reached 1.0. It'd be great to have out-of-the-box support with Spring Cloud Functions. So we could do:

Function<CloudEvent, CloudEvent> foo()

There is an existing API for the payload, marshaling, unmarshalling, but it doesn't yet support version 1.0 of the spec

		<dependency>
			<groupId>io.cloudevents</groupId>
			<artifactId>cloudevents-api</artifactId>
			<version>0.3.1</version>
		</dependency>

Adding this will make integration with Knative Events easier.

@olegz olegz self-assigned this Oct 31, 2019
@olegz
Copy link
Contributor

olegz commented Oct 31, 2019

@jamesward I knew sooner or later we'll reconnect ;)

@matzew
Copy link

matzew commented Dec 13, 2019

@jamesward the lib now has 1.0 support, see: https://github.com/cloudevents/sdk-java/tree/v1.0.0

@olegz
Copy link
Contributor

olegz commented Jan 28, 2020

@jamesward @matzew It seems to me that all we need to do on out end is introduce a MessageConverter implementation for CloudEvent (which is effectively marshaller/unmarshaller).
I'll take a first crack at it following https://github.com/cloudevents/sdk-java/tree/master/api (unless you have something better, but it would be nice if you can delegate someone to review given my inexperience.

@artembilan
Copy link
Contributor

We have similar idea in Spring Integration: spring-projects/spring-integration#3103

@olegz
Copy link
Contributor

olegz commented Jan 28, 2020

That's cool. So i'll go with my plan and then it's really a question where those converters would live (SI or SCSt).

@trisberg
Copy link
Member

@olegz have you started this yet? Playing around with functions that accept a CloudEvent as input and would like to have something that implements a JsonMapper and gives me a CloudEvent back. I'm not using any of the SI or SCSt dependencies, maybe this should live either in SCFn along with the Jackson/Gson Mappers or somewhere else.

@olegz
Copy link
Contributor

olegz commented Feb 29, 2020

@trisberg No, I have not started yet, but as I said earlier it appears all that's needed is MessageConverter which is capable of converting from/to CloudEvents. @jamesward given that one of the drivers in our frameworks to pick the correct MessageConverter is content type (e.g., application/json), is there a known or widely used content type for CloudEvents? Something we can rely on?

@jamesward
Copy link
Author

It does look like the latest spec uses some specific Content-Type values:
https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md#3-http-message-mapping

Maybe @duglin can help clarify though.

@duglin
Copy link

duglin commented Mar 1, 2020

Hi - yes the CE spec does define a content-type that will help you determine if the incoming message is a CE or not - at least for structured CE mode (meaning the HTTP body is wrapped in a CE structure). However, knowing whether some random message is binary CE or not is harder. To try to address this I just created this: cloudevents/spec#577

@olegz
Copy link
Contributor

olegz commented Apr 29, 2020

@jamesward @duglin I am trying, but it appears based on some of the arguments from the gate-keepers of the Cloud Event SDK that we should not even be using CloudEvent type. That it is an anti pattern to have user code to depend on CloudEvent type.
If that's the case then perhaps be we should further clarify as to what such support for us would even mean.

@meltsufin
Copy link
Contributor

Should CloudEvent be instead converted to Message<T>, which is more idiomatic for Spring?

@olegz
Copy link
Contributor

olegz commented Apr 29, 2020

@meltsufin
Not at all. Spring Message is a central fixture of spring-messaging and will remain as such, plus Message can easily represent the cloud event (if you simply go by the spec) as it already provides a compatible structure.
So, it is really a question of CloudEvent as a type and whether it has any additional value.

In any event, I do envision simple transformation/conversion of Message to/from CloudEvent.

@jamesward
Copy link
Author

I guess this comes down to whether the CloudEvent SDK folks see it as a shareable construct. Should APIs ever build on CloudEvent? If not, then yeah we could just marshal CloudEvent payloads into Message (and out). But if that is the case, why is there even a CloudEvent SDK?

@duglin
Copy link

duglin commented Apr 30, 2020

@jamesward @duglin I am trying, but it appears based on some of the arguments from the gate-keepers of the Cloud Event SDK that we should not even be using CloudEvent type. That it is an anti pattern to have user code to depend on CloudEvent type.
If that's the case then perhaps be we should further clarify as to what such support for us would even mean.

I've asked if we can discuss this on the SDK call tomorrow (right after the normal CE call, which is from 12-1 ET unless it ends early). Anyone is free to join. See: https://docs.google.com/document/d/1OVF68rpuPK5shIHILK9JOqlZBbfe91RNzQ7u_P7YCDE/edit#

@olegz
Copy link
Contributor

olegz commented Apr 30, 2020

Thank you @duglin I will join. Perhaps @bsideup can do as well

@jamesward
Copy link
Author

So it looks like maybe some progress on the cloudevents API side of things?

@olegz
Copy link
Contributor

olegz commented May 22, 2020

Indeed @jamesward we now have cleanly defined CloudEvent - https://github.com/cloudevents/sdk-java/tree/master/api/src/main/java/io/cloudevents

@jamesward
Copy link
Author

Fantastic! Looks much better. Thanks for working through that!

olegz added a commit to olegz/spring-cloud-function that referenced this issue Nov 10, 2020
Added initial implementation of MessageConverter
At the moment there seem that MessageConverter(s) would be the only thing needed to integrate Cloud Events with various elements of Spring
olegz added a commit to olegz/spring-cloud-function that referenced this issue Nov 10, 2020
This commit adds initial set of samples and tests demonstrating various ways of how cloud event can be consumed (i.e., structured, bindary, pojo etc)
@olegz
Copy link
Contributor

olegz commented Nov 10, 2020

@jamesward things are finally moving. More to come, keeping it open for now

olegz added a commit to olegz/spring-cloud-function that referenced this issue Nov 11, 2020
Added initial README
Polished tests
olegz added a commit to olegz/spring-cloud-function that referenced this issue Nov 11, 2020
olegz added a commit to olegz/spring-cloud-function that referenced this issue Nov 11, 2020
olegz added a commit to olegz/spring-cloud-function that referenced this issue Nov 13, 2020
olegz added a commit to olegz/spring-cloud-function that referenced this issue Nov 16, 2020
olegz added a commit to olegz/spring-cloud-function that referenced this issue Nov 16, 2020
…ucture-mode CE message

Normalizing in this context means converting it to binary-mode so the rest of the processing logic is the same.
Added support for canonical attribute names. Now, internally any attribute can be set as 'ce_' regardless where it came from are where it goes to as the frameork will be able to recognize both
Removed CloudEventMessageConverter
Renamed CloudEventAttributes to CloudEventAttributesHelperas it is better suited to what it actually does
olegz added a commit that referenced this issue Nov 16, 2020
olegz added a commit that referenced this issue Nov 16, 2020
olegz added a commit that referenced this issue Nov 16, 2020
@olegz
Copy link
Contributor

olegz commented Nov 17, 2020

This has been addressed in spring-cloud-function-context module.
However, this work will be contributed to CNCF java SDK. Once accepted there will be minor dependency change, but given that outside of CloudEventAttributesProvider user generally would not interact with the API, this should not be an issue. In any event, by the time 3.1.0.RELEASE is out it should be settled.

This README contains initial usage documentation and sample instructions.
This will also be donated to CNCF java SDK

@olegz olegz closed this as completed in 16bce2d Nov 17, 2020
olegz added a commit that referenced this issue Apr 18, 2023
Added initial implementation of MessageConverter
At the moment there seem that MessageConverter(s) would be the only thing needed to integrate Cloud Events with various elements of Spring
olegz added a commit that referenced this issue Apr 18, 2023
This commit adds initial set of samples and tests demonstrating various ways of how cloud event can be consumed (i.e., structured, bindary, pojo etc)
olegz added a commit that referenced this issue Apr 18, 2023
Added initial README
Polished tests
olegz added a commit that referenced this issue Apr 18, 2023
…le binary-mode and structured-mode cloud events

Moved CloudEvent related artifacts to ‘cloud events’ package with hopes to eventually donating it to CNCF SDK
Created CloudEventUtils identifying necessary constants and utility methods
olegz added a commit that referenced this issue Apr 18, 2023
…e mapping

Added CloudEventAttributesProvider and default implementation
Added CloudEventMessageUtils
olegz added a commit that referenced this issue Apr 18, 2023
olegz added a commit that referenced this issue Apr 18, 2023
olegz added a commit that referenced this issue Apr 18, 2023
olegz added a commit that referenced this issue Apr 18, 2023
Normalizing in this context means converting it to binary-mode so the rest of the processing logic is the same.
Added support for canonical attribute names. Now, internally any attribute can be set as 'ce_' regardless where it came from are where it goes to as the frameork will be able to recognize both
Removed CloudEventMessageConverter
Renamed CloudEventAttributes to CloudEventAttributesHelperas it is better suited to what it actually does
olegz added a commit that referenced this issue Apr 18, 2023
olegz added a commit that referenced this issue Apr 18, 2023
olegz added a commit that referenced this issue Apr 18, 2023
olegz added a commit that referenced this issue Apr 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

7 participants