From 80a7ddfec841b909a83fa86b7364ded895a94349 Mon Sep 17 00:00:00 2001 From: Dai MIKURUBE Date: Wed, 27 Nov 2024 14:49:23 +0900 Subject: [PATCH 1/3] Looking for long-term maintainers around the Embulk eco-system --- ...11-27-looking-for-long-term-maintainers.md | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 _posts/2024-11-27-looking-for-long-term-maintainers.md diff --git a/_posts/2024-11-27-looking-for-long-term-maintainers.md b/_posts/2024-11-27-looking-for-long-term-maintainers.md new file mode 100644 index 0000000..ddaa515 --- /dev/null +++ b/_posts/2024-11-27-looking-for-long-term-maintainers.md @@ -0,0 +1,68 @@ +--- +layout: posts +title: "Looking for long-term maintainers around the Embulk eco-system" +date: 2024-11-27 +description: "Despite the difficult situation of an open-source community, we must say that we need your help, especially in the long-term maintainer role, not in one-shot or short-term contributions." +author: "dmikurube" +--- + +The 10th anniversary of [Embulk's first release](https://github.com/embulk/embulk/releases/tag/v0.1.0) (by [@frsyuki](https://github.com/frsyuki)) is approaching. Embulk has been matured, stabilized, and used in the real world. We have also been working for modernizing Embulk, and we'd say that Embulk v1.0 is coming sooner. + +[EEP-8: Milestones to Embulk v1.0, and versioning strategies to follow](https://github.com/embulk/embulk/blob/master/docs/eeps/eep-0008.md) + +On the other hand, many things are left behind, indeed. For example, almost all "official" plugins under [https://github.com/embulk](https://github.com/embulk) still use such old libraries (while the Embulk core, built-in plugins and utility libraries recently started using up-to-date dependency libraries). The project web-sites are not maintained so often. The user and developer documents are inadequate yet. + +We have no bandwidth. Dai Mikurube ([@dmikurube](https://github.com/dmikurube)), the author of this article, is working on the open-source Embulk project in half of my working time at [Treasure Data](https://www.treasuredata.com/). What I can do are just some groundworking around the Embulk core, and appending some core architecture design documents like [EEPs](https://github.com/embulk/embulk/tree/master/docs/eeps). Hiroyuki Sato ([@hiroyuki-sato](https://github.com/hiroyuki-sato)) contributes a lot on his personal time by reviewing pull requests (from me and from outside), making suggestions, and raising issues. Hitoshi Tanaka ([@hito4t](https://github.com/hito4t)) also continues to review pull requests for JDBC-related plugins on his personal time. This is almost all of our power to maintain the Embulk eco-system today. + +I have to say that the Embulk project is not sustainable enough. + +The world is not easy, however +=============================== + +We need help. However, it is uneasy for us to say "We need **your** help" to everyone right away, unfortunately. You may remember the news about the XZ Utils backdoor that was revealed in April 2024. + +* [Social engineering aspect of the XZ incident | Securelist](https://securelist.com/xz-backdoor-story-part-2-social-engineering/112476/) +* [Timeline of the xz open source attack](https://research.swtch.com/xz-timeline) + +We expect that the same can happen to Embulk. Putting absolute trust in a completely new person has a risk in the world now. + +In fact, we have already received a couple of "malicious-looking" contacts from some "suspicious" developers. They seemed to be leveraging generative AIs to attack a number of open-source projects. The generative AIs give attackers much more power by automatically generating more attacks, while defenders (maintainers) can only get limited help. I'd say it's a hard time for open source maintainers. (I wish AIs could perform a perfect security review and guarantee a background check!) + +We need your help, still +========================= + +Despite the difficult situation of an open-source community, we must say that we need your help, especially in the long-term maintainer role, not in one-shot or short-term contributions. + +Contributions are great even if they are one-shot or short-term. However, if Embulk were to receive a one-shot contribution from outside the community, the maintainers would still need to review the contribution, check that the contribution does not break compatibility, confirm that the contribution does not disturb future enhancements, and communicate with the contributor if changes need to be made to the contribution or if the contribution cannot be accepted. We do not have the bandwidth to handle this. + +Here is a quote from an article about the Linux maintainers. + +> To sum it up, Torvalds said, "It's one of those things where a lot of people seem to think that open source is all about programming, but a lot of it is about communication, too. Maintainers are the ones who translate. I don't necessarily mean language. I mean, the context, the reason for the code. That makes for a tough job. But, if you want to be a maintainer, trust me, there's room at the top." +> +> Steven Vaughan-Nichols: ["Linus Torvalds on the state of Linux today and how AI figures in its future"](https://www.zdnet.com/article/linus-torvalds-on-state-of-linux-today-and-how-ai-figures-in-its-future/) (Dec. 5, 2023 at 2:13 p.m. PT) + +It is unrealistic for everyone to take on such a maintainer role only on their own time. Only powerful engineers can do this. If people who are supported by their organizations could take on maintainer roles, that would be ideal for sustainability. + +It is unreasonable for an organization to spend its resources just on volunteers. If an organization that already uses Embulk extensively and depends on Embulk for their activities and business could support their engineers in such maintainer roles, that would be perfect for the identity of interest between the organization and the Embulk community. + +It is uneasy for ourselves to invite someone who is completely new to the community, for the reason discussed above. If someone who has worked with the Embulk community can join, or introduce someone else to the maintainers, that would be straightforward for accountability. + +This is not to say that we won't accept someone who is completely new, of course. However, please forgive our rudeness that we may check your background. It would be easy if you're supported by your organization as mentioned above. + +We'd appreciate it if you and your organization would consider supporting the Embulk open-source community in this difficult time for open source. + +We have some starting projects, for you! +----------------------------------------- + +Anyway, it is impossible for anyone to take such a maintainer role for making decisions in the community. + +As starting points, we've prepared some "starting projects" for newcomers in GitHub Discussions: [Contributors wanted](https://github.com/orgs/embulk/discussions/categories/contributors-wanted) + +If you're interested in joining maintainers, they will be good candidates to start. Each project would need a certain amount of work. Let's work on them along with collaborating and discussing with us. We'd like you to expand your role in the community through the starting projects. + +Embulk and Treasure Data, jfyi +------------------------------- + +Just for your information, some of you might have thought that Embulk and the eco-system are owned and controlled by [Treasure Data](https://www.treasuredata.com/) historically. But, it's not now! + +Please take a look at another article if you're interested: [Embulk maintenance goes open](https://www.embulk.org/articles/2023/03/10/embulk-maintenance-gets-open.html). From 83cf22ea980b6f3908b705ac1bd8df94faed7149 Mon Sep 17 00:00:00 2001 From: Dai MIKURUBE Date: Thu, 28 Nov 2024 10:49:08 +0900 Subject: [PATCH 2/3] Rename 2024-11-27-looking-for-long-term-maintainers.md to 2024-11-28-looking-for-long-term-maintainers.md --- ...tainers.md => 2024-11-28-looking-for-long-term-maintainers.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename _posts/{2024-11-27-looking-for-long-term-maintainers.md => 2024-11-28-looking-for-long-term-maintainers.md} (100%) diff --git a/_posts/2024-11-27-looking-for-long-term-maintainers.md b/_posts/2024-11-28-looking-for-long-term-maintainers.md similarity index 100% rename from _posts/2024-11-27-looking-for-long-term-maintainers.md rename to _posts/2024-11-28-looking-for-long-term-maintainers.md From 7fd0695e04dbd08aefb3d095d42855f832c30cfe Mon Sep 17 00:00:00 2001 From: Dai MIKURUBE Date: Thu, 28 Nov 2024 13:47:29 +0900 Subject: [PATCH 3/3] Update "Looking for long-term maintainers around the Embulk eco-system" --- ...11-28-looking-for-long-term-maintainers.md | 32 ++++++++++--------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/_posts/2024-11-28-looking-for-long-term-maintainers.md b/_posts/2024-11-28-looking-for-long-term-maintainers.md index ddaa515..0fedf31 100644 --- a/_posts/2024-11-28-looking-for-long-term-maintainers.md +++ b/_posts/2024-11-28-looking-for-long-term-maintainers.md @@ -1,16 +1,16 @@ --- layout: posts title: "Looking for long-term maintainers around the Embulk eco-system" -date: 2024-11-27 -description: "Despite the difficult situation of an open-source community, we must say that we need your help, especially in the long-term maintainer role, not in one-shot or short-term contributions." +date: 2024-11-28 +description: "Despite the difficult situation for an open-source community, we must say that we need your help, especially in the role of long-term maintainer, not in one-shot or short-term contributions." author: "dmikurube" --- -The 10th anniversary of [Embulk's first release](https://github.com/embulk/embulk/releases/tag/v0.1.0) (by [@frsyuki](https://github.com/frsyuki)) is approaching. Embulk has been matured, stabilized, and used in the real world. We have also been working for modernizing Embulk, and we'd say that Embulk v1.0 is coming sooner. +The 10th anniversary of [Embulk's first release](https://github.com/embulk/embulk/releases/tag/v0.1.0) (by [@frsyuki](https://github.com/frsyuki)) is approaching. Embulk has matured, stabilized, and been used in the real world. We have also been working on modernizing Embulk, and we'd say that Embulk v1.0 is coming sooner. [EEP-8: Milestones to Embulk v1.0, and versioning strategies to follow](https://github.com/embulk/embulk/blob/master/docs/eeps/eep-0008.md) -On the other hand, many things are left behind, indeed. For example, almost all "official" plugins under [https://github.com/embulk](https://github.com/embulk) still use such old libraries (while the Embulk core, built-in plugins and utility libraries recently started using up-to-date dependency libraries). The project web-sites are not maintained so often. The user and developer documents are inadequate yet. +On the other hand, many things are indeed left behind. For example, almost all "official" plugins under [https://github.com/embulk](https://github.com/embulk) still use such old libraries (while the Embulk core, built-in plugins and utility libraries recently started using up-to-date dependency libraries). The project web-sites are not maintained so often. The user and developer documents are inadequate yet. We have no bandwidth. Dai Mikurube ([@dmikurube](https://github.com/dmikurube)), the author of this article, is working on the open-source Embulk project in half of my working time at [Treasure Data](https://www.treasuredata.com/). What I can do are just some groundworking around the Embulk core, and appending some core architecture design documents like [EEPs](https://github.com/embulk/embulk/tree/master/docs/eeps). Hiroyuki Sato ([@hiroyuki-sato](https://github.com/hiroyuki-sato)) contributes a lot on his personal time by reviewing pull requests (from me and from outside), making suggestions, and raising issues. Hitoshi Tanaka ([@hito4t](https://github.com/hito4t)) also continues to review pull requests for JDBC-related plugins on his personal time. This is almost all of our power to maintain the Embulk eco-system today. @@ -19,19 +19,21 @@ I have to say that the Embulk project is not sustainable enough. The world is not easy, however =============================== -We need help. However, it is uneasy for us to say "We need **your** help" to everyone right away, unfortunately. You may remember the news about the XZ Utils backdoor that was revealed in April 2024. +We need help. However, it is uneasy for us to say "We need **your** help" to everyone right away, unfortunately in fact. + +You may remember the news about the XZ Utils backdoor that was revealed in April 2024. * [Social engineering aspect of the XZ incident | Securelist](https://securelist.com/xz-backdoor-story-part-2-social-engineering/112476/) * [Timeline of the xz open source attack](https://research.swtch.com/xz-timeline) We expect that the same can happen to Embulk. Putting absolute trust in a completely new person has a risk in the world now. -In fact, we have already received a couple of "malicious-looking" contacts from some "suspicious" developers. They seemed to be leveraging generative AIs to attack a number of open-source projects. The generative AIs give attackers much more power by automatically generating more attacks, while defenders (maintainers) can only get limited help. I'd say it's a hard time for open source maintainers. (I wish AIs could perform a perfect security review and guarantee a background check!) +In fact, we have already received a couple of "malicious-looking" contacts from some "suspicious" developers. They seemed to be leveraging generative AIs to attack a number of open-source projects. The generative AIs give attackers much more power by automatically generating more attacks, while defenders (maintainers) can only get limited help from AIs. I'd say it's a hard time for open source maintainers. (I wish AIs could perform a perfect security review and guarantee a background check!) We need your help, still ========================= -Despite the difficult situation of an open-source community, we must say that we need your help, especially in the long-term maintainer role, not in one-shot or short-term contributions. +Despite the difficult situation for an open-source community, we must say that we need your help, especially in the role of long-term maintainer, not in one-shot or short-term contributions. Contributions are great even if they are one-shot or short-term. However, if Embulk were to receive a one-shot contribution from outside the community, the maintainers would still need to review the contribution, check that the contribution does not break compatibility, confirm that the contribution does not disturb future enhancements, and communicate with the contributor if changes need to be made to the contribution or if the contribution cannot be accepted. We do not have the bandwidth to handle this. @@ -47,22 +49,22 @@ It is unreasonable for an organization to spend its resources just on volunteers It is uneasy for ourselves to invite someone who is completely new to the community, for the reason discussed above. If someone who has worked with the Embulk community can join, or introduce someone else to the maintainers, that would be straightforward for accountability. -This is not to say that we won't accept someone who is completely new, of course. However, please forgive our rudeness that we may check your background. It would be easy if you're supported by your organization as mentioned above. +This is not to say that we won't accept someone who is completely new, of course. However, please forgive our rudeness that we may check your background. It would be easy if you're officially supported by your organization as mentioned above. We'd appreciate it if you and your organization would consider supporting the Embulk open-source community in this difficult time for open source. -We have some starting projects, for you! ------------------------------------------ +We have some "starter projects", for you! +------------------------------------------- -Anyway, it is impossible for anyone to take such a maintainer role for making decisions in the community. +Anyway, it is impossible for anyone to immediately take on such a maintainer role from ground zero for making decisions in the community. -As starting points, we've prepared some "starting projects" for newcomers in GitHub Discussions: [Contributors wanted](https://github.com/orgs/embulk/discussions/categories/contributors-wanted) +As a starting point, we've prepared some "starter projects" for newcomers in GitHub Discussions. See: [Contributors wanted](https://github.com/orgs/embulk/discussions/categories/contributors-wanted) -If you're interested in joining maintainers, they will be good candidates to start. Each project would need a certain amount of work. Let's work on them along with collaborating and discussing with us. We'd like you to expand your role in the community through the starting projects. +If you're interested in joining as a maintainer, these are good candidates to start with. Each project would require a certain amount of work, with several decisions to be made. Let's work on them together, collaborating and discussing with us. We'd like you to expand your role in the community through the projects. Embulk and Treasure Data, jfyi ------------------------------- -Just for your information, some of you might have thought that Embulk and the eco-system are owned and controlled by [Treasure Data](https://www.treasuredata.com/) historically. But, it's not now! +Just for your information... Some of you may have thought in the past that Embulk and the eco-system were owned and controlled by [Treasure Data](https://www.treasuredata.com/). But, that's not the case anymore! -Please take a look at another article if you're interested: [Embulk maintenance goes open](https://www.embulk.org/articles/2023/03/10/embulk-maintenance-gets-open.html). +Please take a look at this article if you're interested: [Embulk maintenance goes open](https://www.embulk.org/articles/2023/03/10/embulk-maintenance-gets-open.html).