Feat: Support LLMs for i18n Translation + remove CrowdIn ads

[ardigan6]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Description

Add i18n auto-translation support using LLMs via a flexible API.

Has this been requested on Canny?

No response

Motivation

Modern LLMs are better technical translators than the average CrowdIn human.

It would make sense to fully automate the translation flow in CI, and possibly deprecate CrowdIn support mentions in the future.

API design

docusaurus --translation-source-lang=en --translation-target-langs=fr,de --translation-llm-endpoint=[openai-compatible router]

Have you tried building it?

This should be a core feature.

Self-service

• I'd be willing to contribute this feature to Docusaurus myself.

[slorber]

Docusaurus is based on file system paths.

We only document Crowdin usage for historical reasons, because Docusaurus v1 sites used it in 2017, and we wanted to help them upgrade to Docusaurus v2 (including ours and other sites at Meta). But we don't have a tight coupling to Crowdin, nor do we have any integration code for it. You don't have to use Crowdin, and we don't plan to deprecate the integration we don't even have.

The community can build a CLI to automatically translate MDX content. The only requirement is to respect the file system conventions we have.

We could be interested having this a first-class Docusaurus feature, but I'd rather see a successful community implementation first, so that we understand what works and what doesn't.

[ardigan6]

Docusaurus is based on file system paths.

We only document Crowdin usage for historical reasons, because Docusaurus v1 sites used it in 2017, and we wanted to help them upgrade to Docusaurus v2 (including ours and other sites at Meta). But we don't have a tight coupling to Crowdin, nor do we have any integration code for it. You don't have to use Crowdin, and we don't plan to deprecate the integration we don't even have.

The community can build a CLI to automatically translate MDX content. The only requirement is to respect the file system conventions we have.

We could be interested having this a first-class Docusaurus feature, but I'd rather see a successful community implementation first, so that we understand what works and what doesn't.

Yes, that was a bit unclear. I was referring to the README mentions, docs tutorial, etc. Hard to imagine >1% of docusaurus users wanting to use CrowdIn in 2025.

There are several LLM translation scripts available, e.g. https://github.com/moonrailgun/docusaurus-i18n
https://github.com/DanRoscigno/translate_changed_files

My version uses a different approach:

1. parses the mdx,
2. extracts strings only from header/table/string blocks (e.g. ignoring code),
3. replaces any code in the string with $CODEVAR[n], and then
4. chunks those strings into hash-keyed JSON ({"3d653119": "hello"}) for each supported language,
5. sends chunk to LLM endpoint for i18n, then
6. sends the output along with the original English to a second LLM to review, mark any hashes with errors, and provide an alternative translation for them, then
7. copies base files to each language and applies string replacements on top of them.

This works well, enables granular and efficient caching, and avoids many issues with more naive approaches that break mdx formatting or embedded code blocks.

Here is a simplified version of this approach for reference: https://github.com/ardigan6/docusaurus-llm-translator

[slorber]

Hard to imagine >1% of docusaurus users wanting to use CrowdIn in 2025.

We use it for Meta sites and don't plan to migrate away in the short term.

This solution is not mandatory, so if you don't like it, just don't use it. There's no need to criticize it. Adding automatic LLM translations and removing Crowdin from our docs are totally different topics.

---

This works well, enables granular and efficient caching, and avoids many issues with more naive approaches that break mdx formatting or embedded code blocks.

Here is a simplified version of this approach for reference: ardigan6/docusaurus-llm-translator

If you are already able to achieve your need with your own code, what's the value of adding this to Docusaurus core?

[ardigan6]

Hard to imagine >1% of docusaurus users wanting to use CrowdIn in 2025.

We use it for Meta sites and don't plan to migrate away in the short term.

Have you benchmarked this against the latest models? CrowdIn vendor results were uniformly inferior for the languages I speak, and of course much slower than an API call.

If you are already able to achieve your need with your own code, what's the value of adding this to Docusaurus core?

I've seen very few Docusaurus sites in the wild that enable i18n, which is a shame.

Probably this is somewhat related to the difficulty in auto-translating the content compared to a traditional CMS, where structured storage means string replacements at the field level are easy to handle safely.

Building it into core would make it simpler to reuse AST walking rather than my regex strategy, which was quick to write and works in my case but undoubtedly would fail on some docs.