Typesafety for inlang.config.js || allow top level imports from node modules in inlang.config.js

[ivanhofer]

Problem

Regular imports like the following do not work currently:

import { missingKeyRule } from '@inlang/standard-lint-rules'

lint: {
	rules: [
		missingKeyRule(),
	]
}

We need to use the env.$import utility function and then we can only import a file from an url. Regular node_modules imports won't work. This also means that we don't get any TypeScript autocompletions.

Solution

The inlang.config.js could be modified in-memory before it's content get read. The import statements could get transformed and node modules could get resolved, downloaded and the correct entry-point could get found reading the exports map inside package.json.

This should be pretty straightforward and improve DX a lot because there is nothing new to learn.
We should not re-implement the full node resolution algorithm, as all inlang plugins should in theory be "modern", if they use the template repository.

The process could look like this:

1. read inlang.config.js
2. detect all import statements
3. download all node modules and store them on disk (to be able to cache them)
4. detect entry point of node modules
5. rewrite import statement from point nr. 2
6. write modified file to disk
7. read modified file
8. done 🎉

[samuelstroschein]

Finding a better way to import files is welcome!

Step 2 will be the hard one. Detecting all imports in the inlang.config.js file is not enough. Detecting all imports from imported files is necessary too, if we break the rule that imports can only be conducted with env.$import.

[...] downloaded and the correct entry-point could get found reading the exports map inside package.json.

Inlang is supposed to be stack agnostic. Flutter, iOS, Android projects won't have a package.json file and neither would a Deno project.

download all node modules and store them on disk (to be able to cache them)
detect entry point of node modules

If we were to implement import features, I would refrain from anything node-related. ESM solves importing JavaScript and works everywhere.

Using sandboxing to modify import call

The desired effect of using import could be achieved by sandboxing everything in inlang.config.js and thereby controlling native system calls. I intended to implement sandboxing but the effort is high https://github.com/inlang/inlang/pull/129/files#diff-e7505b1903e097a3930ecab74635fff5acaf26cc520f1f34aba31c221edcd224.

[samuelstroschein]

Amusing just after I wrote inlang should work for iOS too [whose projects don't have a package.json], I received the following text:

[ivanhofer]

Step 2 will be the hard one. Detecting all imports in the inlang.config.js file is not enough. Detecting all imports from imported files is necessary too, if we break the rule that imports can only be conducted with env.$import.

We don't need to be compatible with the full JS ecosystem.. just with plugins that were espacially created for inlang. This means, that they will be bundled (=> they don't import other node modules, just regular JavaScript files from a relative path). This makes it relatively easy to implement.

Inlang is supposed to be stack agnostic. Flutter, iOS, Android projects won't have a package.json file and neither would a Deno project

Inlang plugins will always be written in JavaScript or else we won't be able to execute them (without some bridge at least). This proposal is only meant for "top level imports" and not files that ar beeing imported once resource files get loaded from disk. There we can still rely on the env.$import or we just import the file as a raw text blob.

JavaScript projects will mostly have a package.json file and deno has a deno.json, that can be pared to do something similar.

If we were to implement import features, I would refrain from anything node-related. ESM solves importing JavaScript and works everywhere.

ESM is the future, but there are a lot of problems right now:

• Node.js does not support it (needed for CLI)
• you need to use a CDN (no pre-runtime checks if the URL really exists)
• TypeScript will not give you any hints

In my opinion TypeScript should be heavily integrated in every decision. The inlang.config.js file is the central piece of inlang where everything get's set up. Depending on the plugins and extensions someone uses, this file can grow in complexity. Having code completions and static type checks will greatly improve the DX and noone needs to debug their config for half an hour only to realise that they made a typo.
A inlang config validator will not solve that problem.

[samuelstroschein]

We don't need to be compatible with the full JS ecosystem.. just with plugins that were espacially created for inlang. This means, that they will be bundled (=> they don't import other node modules, just regular JavaScript files from a relative path). This makes it relatively easy to implement.

I am referring to plugins that are dynamically importing JavaScript on their own.

A top level import in inlang.config.js would break the semantics of "you must import with env.$import when using inlang". A use case for dynamically importing from a plugin is to read JavaScript files that act as Resources e.g. store translations. Right now, the env objects needs to be prop-drilled. Frankly, not the best solution. But, solving import on the top level only will lead to new problems in the ecosystem chain.

ESM is the future, but there are a lot of problems right now:

• Node.js does not support it (needed for CLI)
• you need to use a CDN (no pre-runtime checks if the URL really exists)

Node 16 (end of life reached September 2023) and above supports ESM including HTTP imports.

Using HTTP imports instead of relying on node module resolutions simplifies our maintenance effort by a wide margin and makes inlang cross platform e.g. one script inlang.config.js is all that is required to make inlang work for Node, Deno, Flutter, iOS, Android, Django projects. No package.json, no import maps, nothing. Just import JS with HTTP imports or from a local path.

• TypeScript will not give you any hints
  In my opinion TypeScript should be heavily integrated in every decision.

I fully agree.

I propose to implement HTTP type imports for TypeScript (like Deno) instead of relying on "hacking" node to somehow get type safety. Thereby we can reduce our maintenance effort and contribute to making TypeScript and ESM better.

[samuelstroschein]

Outdated after #1142.

• the config is now JSON