Update or remove outdated / abandoned documentation translations

[jhildenbiddle]

Bug Report

The documentation translations available on docsify.js.org are outdated and in some cases completely abandoned. Inaccurate documentation in a preferred language is (IMHO) far worse than accurate documentation in a non-preferred language.

Last updated:

• docsifyjs/docs-es: 11/7/2023
• docsifyjs/docs-de: 4/23/2020
• docsifyjs/docs-ru: 4/23/2020
• docsifyjs/docs-zh: 8/28/2023

Without native language speakers to keep these translations up to date, it seems wise to consider the following options (ordered from most to least preferred):

1. Retire these translations altogether and rely on modern browser language translation features (see below).
2. Explore automated translation services to keep these repos up to date. GitLocalize is one free option worth considering.
3. Continue manually supporting only the translation(s) we can verify site visitors use and that maintainer(s) are willing to actively maintain.

IMPORTANT: At the time Docsify's documentation translations were created, browsers did not offer built-in language translation functionality. Today, all major browsers offer built-in language translation functionality. This means that non-native English speakers can visit our English docs and have them auto-translated to their native language by the browser automatically if they prefer.

Below are screenshots from taken while viewing docsify.js.org translations as an English-preferred visitor:

Chrome - 简体中文

Safari - Deutsch

Firefox - Spanish

[paulhibbitts]

Thanks for bringing this issue up @jhildenbiddle. As you've identified, up-to-date translation itself is a major on-going effort to create and then maintain - for example, another open source project I have been involved with is https://getgrav.org has a dedicated team of volunteer contributors using Crowdin https://crowdin.com/profile/getgrav.

The out-of-date docs is a major user experience issue in itself. Based on what I see, perhaps option 1 is a beneficial path to take for the initial release of v5 and then re-assess viability of either option 2 or 3? Interested to hear other perspectives.

[trusktr]

Browser translation is pretty good I think, especially after many years of improvements.

Does having pre-translated docs help with SEO? For example, do search engines show users in a nother country results for english sites in their language? Or do websites need to have content in the target country's language to become visible there?

If the answer is that pre-translation would be needed for this, I'd say maybe that's the single most reason to do it. But then the question would be how? And another question is, will keeping the current translations even if outdated help with this SEO?

[Koooooo-7]

I think we can keep the remained documentation translations, and add the last update date banner with docsify-top-banner-plugin. i.e

This label could be automate update, when new commits into.

Current site last update date is [xxxx-xx-xx] if you don't find the things you want, try find in latest [EN] site.

Browser translation is always available, every users can use it in our default update-to-date EN docs, if users can not find the target part in their prefer langs, they can go to the EN docs.

@trusktr mentioned the SEO thing is reasonable to me, and I think the Browser translation still exist the Gap for the natives.

[jhildenbiddle]

Our goal should be to provide accurate, useful documentation to Docsify users. If our translations don't do that, then they aren't serving their intended purpose and we should remove them for the benefit of our users and our project maintainers. It should be that simple.

I appreciate the desire to show translations on the site, but the reality is that offering localized content that serves users requires a significant upfront and continued investment that our team simply does not have the capacity to make. Pretending this is not the case by publishing inaccurate and/or outdated documentation that hasn't been updated since 2020 is a disservice to the site visitors that read those translations and, IMHO, reflects poorly on both the project and its maintainers.

As for SEO, I don't think there is any reason to believe our existing translations impact SEO primarily because docsify.js.org serves pages (including all translations) in router: hash mode which we know generates URLs that are not indexed reliably (if at all) by search engines.

Fortunately, we don't have to speculate about SEO. Our site is configured with Google Analytics so we can see exactly where our traffic is coming from and how many visitors are actually reading out current translations. Assuming that has been capturing data properly, we just need to log in and review the data.

---

All of this is to say my preference is to remove non-EN translations until we are able to create a plan for if/how we want to support localized documentation and which locals we will support.

[Koooooo-7]

Yea, I agree that the out of date docs maybe a disservice when we do have lots of breaking changes. The reality is, those sites' most part of the basic configuration/usage works . So I don't think we need remove it directly.
On the other side, I don't think the exist contents in search engine from other ref-links sites which doesn't links to non-EN site path should be invalid directly either.
Alternatively, we move to only keep the EN (and CN? @sy-records ) site, we needs some redirect solutions for those non-EN/CN paths directly to the EN contents (they have the same paths and structures).

[andrii-bodnar]

Hi, I just landed here by accident and would like to add some of my thoughts.

I think the main problem with the current setup is that the sources and localization files are in separate repositories. So it is a lot of work for a potential contributor to suggest new translations, especially if the sources are not synchronized with the translations (they have to go to the main repo, take the current source files, compare them to the translations from the separate repo, and then suggest new translations).

The community translators can work together in Crowdin, for example, translations would be delivered as PR, and volunteers can raise issues for problematic strings via Crowdin. In addition, everything could be automated via CI/CD (pushing sources to Crowdin, downloading translations).

You can also check out how Docusaurus manages i18n for their website.

I believe that a localized website can attract more users who are not fluent in English. Providing content in their native language makes the site more accessible and user-friendly, and encourages greater engagement from a diverse audience.

By the way, Crowdin is free for OSS. If you decide to improve the 18n flow and update the translation status, I can help with automated integration with Crowdin.

[sy-records]

I'll update CN's as they are released.

None of the @docsifyjs/translators team members seem to be active anymore.

I think we can try Crowdin.

[Koooooo-7]

Hi @andrii-bodnar , thx for your suggestion.
I have a quick shoot on Crowdin. It seems use something like the fine-tuning AI translator or 3rd translation providers to do the job. (correct me if I do misunderstanding something).
And I don't much clear understand about the sample you mentioned in Docusaurus.

So, before we consider the efforts about moving to Crowdin.

Could you plz provide more info about What we need prepare before we use the Crowdin :

• Do we need have one AI providers key? or order the 3rd translation provider?
• Based on our project current i18n things, what the structures of the i18n changes we may need to do?

[andrii-bodnar]

Hi @Koooooo-7! Currently, Crowdin supports the following AI providers:

• OpenAI
• Google Gemini
• Microsoft Azure OpenAI
• Mistral AI
• Anthropic

So it's possible to use them to pre-translate existing or new content. In addition, community contributors can also suggest their own translations.

And I don't much clear understand about the sample you mentioned in Docusaurus.

This was done to provide an example of how it can be implemented (it is an interesting approach, but more complex to set up).

Do we need have one AI providers key? or order the 3rd translation provider?

You can use your own key. Visit the Crowdin AI article for more details.

Based on our project current i18n things, what the structures of the i18n changes we may need to do?

I'd suggest keeping the source files and translations within the current repository (use some subdirectories for the translated documents).

For example:

.
├── docs/locales
│   ├── en
│   │   └── ...
│   └── es
│   │   └── ...
│   └── de
│   │   └── ...
└── ...

[sy-records]

https://store.crowdin.com/md

Crowdin supports markdown, so it shouldn't change much for us.

[Koooooo-7]

Hi @andrii-bodnar

So it's possible to use them to pre-translate existing or new content. In addition, community contributors can also suggest their own translations.

IIUC, it means that we could choose use own API key, or using the Crowdin's solution directly for free (if we don't do the pre-train stuff)?

I'd suggest keeping the source files and translations within the current repository (use some subdirectories for the translated documents).

For example:

.
├── docs/locales
│   ├── en
│   │   └── ...
│   └── es
│   │   └── ...
│   └── de
│   │   └── ...
└── ...

Could you show me an vivid PR case of the translation content via the github integration?
I wanna make sure how to category the contents.

[Koooooo-7]

https://store.crowdin.com/md

Crowdin supports markdown, so it shouldn't change much for us.

Yes, I believe it can handle multi file types with AI power.
I just wanna clarify more about the integration details scope/cost in our side, then, we could set up a planning.

[andrii-bodnar]

@Koooooo-7 the mentioned AI solutions are not included directly in the platform for free. There is only free "Crowdin Translate MT Engine" - Crowdin Neural Machine Translator uses Crowdin Global TM to produce way better suggestions.

To use the OpenAI, Google Gemini, Microsoft Azure OpenAI, Mistral AI, Anthropic providers you'll need your own key.

Could you show me an vivid PR case of the translation content via the github integration?

Sure. For example:

• https://github.com/nodejs/nodejs.org
• https://github.com/grafana/grafana
• https://github.com/MetaMask/metamask-mobile
• https://github.com/mastodon/joinmastodon

[Koooooo-7]

@Koooooo-7 the mentioned AI solutions are not included directly in the platform for free. There is only free "Crowdin Translate MT Engine" - Crowdin Neural Machine Translator uses Crowdin Global TM to produce way better suggestions.

I see. so it depends on whether we need the AI stuff and the tokens cost.
thx.

Sure. For example:

• https://github.com/nodejs/nodejs.org
• https://github.com/grafana/grafana
• https://github.com/MetaMask/metamask-mobile
• https://github.com/mastodon/joinmastodon

I checked the files in those samples, it seems the translation contents is only contains the translation json content instead of return a pure translated pages (markdownEN <-> markdwonCN ). such as
https://github.com/nodejs/nodejs.org/blob/main/i18n/locales/en.json
https://github.com/mastodon/joinmastodon/blob/main/locales/en.json

Does it mean that we need implement the i18n function or relying on some i18n components to make it works?
Does Crowdin have the ability to do the markdownEN.md <-> markdownCN.md thing?

[andrii-bodnar]

It's just an example of the integration. Basically, the process is the same, regardless of the file format, Crowdin is able to do the markdownEN.md <-> markdownCN.md synchronization as well.

Here is the example project with the synchronization of md files - https://github.com/lirantal/nodejs-cli-apps-best-practices

[andrii-bodnar]

The only thing here is that it might be tricky to upload the existing translations, especially in such a case of out-of-sync translations.

For files that do not have a defined structure, translation upload is handled by an experimental machine learning technology.

This includes the following file formats: HTML, Front Matter HTML, Markdown, Front Matter Markdown, TXT, Generic XML, Web XML, DOCX, HAML, IDML, DITA, Wiki, FLSNP, MIF, and ADOC.

https://support.crowdin.com/uploading-translations/#text-and-html-based-formats

There is a helper app for that - https://store.crowdin.com/aligner

[Koooooo-7]

It's just an example of the integration. Basically, the process is the same, regardless of the file format, Crowdin is able to do the markdownEN.md <-> markdownCN.md synchronization as well.

Here is the example project with the synchronization of md files - https://github.com/lirantal/nodejs-cli-apps-best-practices

Oh I see! It seems put all files together is easier to maintain and do the integration.
Thx for your patience!

[Koooooo-7]

The only thing here is that it might be tricky to upload the existing translations, especially in such a case of out-of-sync translations.

For files that do not have a defined structure, translation upload is handled by an experimental machine learning technology.

I think if we decide to do the translation via Crowdin, we could only rely on the EN docs to do the translation and drop current out of date translations to avoid the structure miss match issue.

[paulhibbitts]

Regardless of any additional tools involved, the challenge remains the management and quality control of all supported languages. At a minimum we would need someone to manage and monitor translation efforts/updates and then contributors (or reviewers re: generated translations) interested and be available for on-going updates etc.

Until someone comes forward to take on this significant task, I am still leaning towards removing incorrect/outdated docs and relying on Browser translation (where people then expect non-perfect results) and re-assess from there. If we were to keep these outdated docs available in some manner, then adding some sort of date banner as suggested by @Koooooo-7 might be something to consider so we can proceed with the planned release of v5.

Hmm.. one more alternative might be to (temporarily?) remove those languages most out-of-date (two - de/ru) but keep the two most recently updated (zh/es) with some sort of date banner?

[jhildenbiddle]

I support exploring localization options, however we have short-term needs that I believe need to take priority over long-term aspirations.

Equally important, this was not intended to be a debate about the value of high-quality localized content. That value is understood. If we could provide Docsify documentation in multiple languages quickly, easily, reliably, and affordably (preferably free) then of course we would do so.

With regards to continuing to serve the existing localizations, "People like reading in their preferred language", "it's [maybe] good for SEO", "some of it is still accurate", and "Project X does it so Docsify should, too" are not valid reasons for continuing to serve inaccurate and outdated documentation to our users. These also aren't good reasons to burden our small team of maintainers with decisions made long ago by other people which may no longer be in our best interests.

About those short-term needs...

We are trying to get Docsify v5 out the door. This is the first major version bump for this project since 2017. We do not want to launch v5 with inaccurate, missing, and outdated documentation in any language. If there's one thing we can learn from v4 and the upcoming v5 transition, it is that problematic documentation can cause signficant problems beyond frustrated users and may require signficant effort to address in the future. We don't want to repeat that mistake.

Given our history it seems highly unlikely that all of our localized documentation will be updated in time for v5's launch. Even if we somehow managed to update all localized documentation in time for the v5 launch, we have no system in place for mainting it so we would quickly find ourselves back in this same position. Let's not do that.

So, I propose the following to address our short-term needs and long-term localization aspirations:

Short Term (v5 Launch)

1. Provide complete documentation in English only for v5.
2. Provide the same "Translations" links we do today (English, 简体中文, Deutsch, Español and Русски) but direct each one to a landing page that contains the following:
   1. An introduction to the project (similar to EN docs landing page)
   2. A statement about our desire to provide localized documentation
   3. Instructions on how to view our complete documentation (EN docs)
   4. Instructions on how to optionally translate our complete documentation to their preferred language using their browser's built-in browser translation functionality.
   5. A link to a GitHub issue or discussion where visitors can vote and/or voice their support for a specific local or localized content in general. We can also request site owners share their site URL
   6. A sidebar with localized links (generated from the EN sidebar markdown) where each link redirects to an EN equivalent page.
3. In our Docsify config, redirect all previous translation URLs to the localized landing page described above. For example, https://docsify.js.org/#/zh-cn/quickstart will redirect to https://docsify.js.org/#/zh-cn/. If preferred, we can instead redirect to the EN equivalent content.

By doing this, we accomplish the following:

• Ensure that all public documentation is available, accurate, and up-to-date for v5
• Provide a way for users to view our documentation in their preferred language
• Significantly reduce the amount of localized content we have to manage
• Redirect previous localized URLs to appropriate replacement content
• Continue to showcase a "Translations" menu on docsify.js.org to highlight functionality
• Set expectations and establish trust with users by being transparent about the status of our localized content
• Create the ability to evaluate interest and engage with users about localized content via Google Analytics and "Localization" GitHub Issue/Discussion activity.

Long Term

1. Identify a localization leader (@sy-records?) to drive localization efforts.
2. Explore localization options like [Crowdin](https://crowdin.com), [GitLocalize](https://gitlocalize.com), and others.
3. Evaluate interest in localized content via Google Analytics and "Localization" GitHub Issue/Discussion activity.
4. Determine if / how / when to test selected options and potentially
5. Determine if / how / when to make localized documentaion publicly available.

By doing this, we accomplish the following:

• Allow localization exploration to begin immediately without impacting a v5 release.
• Give ourselves time to explore and test all available options.
• Make an informed decision on if/how to prioritize localization efforts based on interest evaluation (Google Analytics & GitHub Issue/Discussion activity).

---

If folks are okay with the short-term solution above, I will agree to work on it for v5. If someone else wants to take the lead, I'm happy to assist with the effort if needed.

[jhildenbiddle]

Also...

@sy-records, I truly appreciate your offer to keep the CN docs up to date as changes are made. To be clear, my goal is not to remove all non-English documentation from the site forever. I just don't think our small team should be burdened with tracking and manually updating markdown across PRs because we know this is problematic despite best intentions. I support providing localized documentation in the future if we determine there is enough interest and we can do so reliably and cost effectively.

[paulhibbitts]

Thanks for sharing your thoughts @jhildenbiddle on this issue, and I am in agreement with what you propose overall. I hope we can keep scope small to help drive forward a v5 release and then take next steps as needed etc.

I would be happy to help re: taking a stab at the text for some of the above you describe, here is just an example which tries to address several aspects you highlight so everyone can get a better sense of what might be involved etc.👇🏼

---

Preparing for the release of Docsify v5 it was identified that this available language translation was no longer up-to-date and correct. As the Maintainer Team did not want to launch v5 with inaccurate, missing, and outdated documentation in any language, the decision was made to temporarily remove this localized version and rely on Browser language translation of the English version of the Docsify.js.org documentation in order to provide some time to explore options and offer interested contributors to update specific languages etc.

For additional information about using popular Browsers for language translation, please refer to the following links:

• Google Chrome
• Mozilla Firefox
• Microsoft Edge
• Apple Safari

Interested in updating this previously available localized language Docsify documentation? Please join our Discord, https://discord.gg/docsify, and let us know in the new #documentation channel, thank you!

[Koooooo-7]

Hi @jhildenbiddle thx for ur clarify.
I do agree that we should and we'd better move forward to V5 as a new neat and accurate project. That's y I suggest only tag those docs as Out-of-date instead of try to translate it and sync up-to-date.

For the removal, it's hard to say when and where the multi langs support could come back...
So I engaged to do adaption instead of abandon.

I think your short term is a good solution but it seems add more efforts on it？ （glad to know anything can help）
Maybe we could split those tasks, @sy-records and I could work together on the i18n part of the short term changes, ( nice to have: also we do the sync of zh asap). @sy-records is that okay for u?

If those changes would cost much on the i18n changes, maybe we just add the banner to mention other non-EN docs to make the V5 release go ahead ( not pending on those part too much)？（As a worse backup PlanB）

About the Long term i18n solution. It could start the PoC after we finish the V5 settled to avoid any conflict getting in. (I suppose it may have a project structure changes or other integration stuff needs)

[jhildenbiddle]

I do agree that we should and we'd better move forward to V5 as a new neat and accurate project. That's y I suggest only tag those docs as Out-of-date instead of try to translate it and sync up-to-date.

Understood.

The issue with just adding banners to our localized documentation is that while they will inform visitors 1) that the documentation is outdated and therefore should not be relied upon and 2) where to find up-to-date documentation, they won't provide any information regarding if, how, and when we plan to address the outdated documentation. These are the things that visitors who want localized documentation will want to know. The landing page described above provides this information. They also provide an opportunity to contribute to our localization discussions and efforts via Discord and GitHub, which we can then consider when evaluating our localization aspirations and commitments in the future.

I think your short term is a good solution but it seems add more efforts on it?

It is more work than simply adding banners, but I believe the benefits justify the effort.

I assume the short-term effort would look something like this:

• Create a landing page template (EN, since that is the common language among maintainers).
• Generate localized versions (currently ES, DE, RU, and ZH) of landing page template using a translation service or LLM.
• To update, edit the template, repeat translation process, and commit changes to all files in a single PR.

The majority of the work will be in the initial efforts to create the landing page template. The rest is pretty straight-forward.

@paulhibbitts has kindly drafted some copy for the landing page. Perhaps this is an effort you (and @sy-records?) would like to collaborate on with him?

Maybe we could split those tasks, @sy-records and I could work together on the i18n part of the short term changes, ( nice to have: also we do the sync of zh asap). @sy-records is that okay for u?

My position is that we really want to avoid having maintainers manually sync our localization files--both short- and long-term.

As I mentioned above, even if we get our localized documentation up to date for v5, we don't have a reliable system in place for maintaining it so we'll be almost certainly back in this same position eventually.

Perhaps more importantly, I also believe there is an evaluation to be made regarding the value of offering localized documentation vs. the level of effort required to do so. We can look to our traffic statistics on Google Analytics and the level of interest the new landing pages will (maybe) surface on Discord and GitHub to help inform this evaluation. If we find that an insignificant number of site visitors are viewing our translations and/or that no users have expressed interest in localizations after we launch the new landing pages, we may very well decide offering localized content isn't worth it for our small team. On the other hand, if we determine there's enough interest and we find a free automated localization solution like the ones mentioned above, we can prioritize that work accordingly.

---

I'll close by saying it's exciting to see some movement in this area as well as Docsify in general. I know these threads can get lengthy and there are lots of opinions and options to sort through, but we're making great progress towards modernizing Docsify and publishing the first major release since v4 in 2017. That's awesome. 🥳

[paulhibbitts]

I'd be happy to contribute more work on the text for the English version landing page template and work with @sy-records (or another Maintainer) on where this should be located in the Docsify repo (for example, /docs/_languages/) and then work on a draft PR etc.

Just as a quick example, I've done another pass of the example draft text and using Docify-This you can preview what this page might look like 🙂 https://docsify-this.net/?basePath=https://raw.githubusercontent.com/paulhibbitts/docsify-translations/main&homepage=translation-template.md&hide-credits=true

UPDATE: As an experiment, here is the same landing page in four different languages as translated by ChatGPT 4.0:

https://docsify-this.net/?basePath=https://raw.githubusercontent.com/paulhibbitts/docsify-translations/main/languages&homepage=docs-de.md&hide-credits=true#/

https://docsify-this.net/?basePath=https://raw.githubusercontent.com/paulhibbitts/docsify-translations/main/languages&homepage=docs-es.md&hide-credits=true#/

https://docsify-this.net/?basePath=https://raw.githubusercontent.com/paulhibbitts/docsify-translations/main/languages&homepage=docs-ru.md&hide-credits=true#/

https://docsify-this.net/?basePath=https://raw.githubusercontent.com/paulhibbitts/docsify-translations/main/languages&homepage=docs-zh.md&hide-credits=true#/

Source repo: https://github.com/paulhibbitts/docsify-translations

[jhildenbiddle]

UPDATE: As an experiment, here is the same landing page in four different languages as translated by ChatGPT 4.0

Thanks for these, @paulhibbitts!

After some discussion on Discord, it sounds like we may end up with a hybrid solution for the release of v5:

• Updated ZH docs via CrowdIn
• Landing pages for other translations

This could change, but for those following I believe this is the direction we are currently moving in.