Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka To Elasticsearch Template #305

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

Cherepushko
Copy link
Contributor

@Cherepushko Cherepushko commented Oct 26, 2021

This is the initial PR for kafka-to-elasticsearch Dataflow template. It contains pom.xml and the empty directories that will be filled in the future pull requests. As a placeholder for the empty folders is used .gitkeep.

@google-cla google-cla bot added the cla: yes The PR submitter has a CLA label Oct 26, 2021
@Cherepushko
Copy link
Contributor Author

@prathapreddy123 @suyograo

@suyograo
Copy link

@prathapreddy123 is it better to place this inside of google cloud to elasticsearch folder?

@prathapreddy123
Copy link
Contributor

@prathapreddy123 is it better to place this inside of google cloud to elasticsearch folder?

Since kafka is third party product this template should remain outside google cloud to elasticsearch. Separate directory (i.e kafka-to-elasticsearch) should be fine.

@suyograo
Copy link

suyograo commented Nov 5, 2021

@prathapreddy123 can we get a review please?

@prathapreddy123
Copy link
Contributor

@prathapreddy123 can we get a review please?

@suyograo - will review by early next week.

Copy link
Contributor

@prathapreddy123 prathapreddy123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Directory Structure looks good. Can you please update PR with actual business logic to avoid merging with empty directories

@Cherepushko Cherepushko changed the title Created module with empty directories for kafka-to-elasticsearch Kafka To Elasticsearch Template Feb 11, 2022
@Cherepushko
Copy link
Contributor Author

As discussed earlier, this PR will be reworked to use original ElasticsearchIO from Apache Beam 2.34.

@Cherepushko Cherepushko marked this pull request as draft February 11, 2022 17:54
@Cherepushko Cherepushko marked this pull request as ready for review February 18, 2022 15:27
@stale
Copy link

stale bot commented Apr 16, 2022

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Apr 16, 2022
@Cherepushko
Copy link
Contributor Author

Comment just to remove "stale" label.

@bvolpato bvolpato added the Kafka label Feb 23, 2023
@bvolpato
Copy link
Contributor

Hi @Cherepushko -- thanks for your contribution here. Unfortunately we ran behind on reviews, and the structure of the project changed a bit. Is there still interest in pursuing this template? We can work together on a plan to resolve conflicts + review the code.

@github-actions github-actions bot removed the stale label Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes The PR submitter has a CLA Kafka size/XXL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants