Watcher introduces plugin logic into sdk.

[NilsJacobsen]

Problem

See the problem in my loom ->
https://www.loom.com/share/cb76b40fa5b846399b8fa5fb7ccc65df?sid=97098f50-5747-4a0f-9c46-a5ebe5194cb9

Proposal 1: Plugin specific stuff should be in plugins

We could add a watcher function with callback to the plugins. The SDK would only access the callback and based on that it would update messages.

SDK:

_resolvedModules.resolvedPluginApi.watch(() => loadAndSetMessages())

Plugins:

export type Plugin<
	ExternalSettings extends Record<keyof ExternalProjectSettings, JSONObject> | unknown = unknown
> = Omit<
	Static<typeof Plugin>,
	"loadMessages" | "saveMessages" | "addCustomApi" | "settingsSchema" | "watch"
> & {
	settingsSchema?: TObject
	loadMessages?: (args: {
		settings: ProjectSettings & ExternalSettings
		nodeishFs: NodeishFilesystemSubset
	}) => Promise<Message[]> | Message[]
	saveMessages?: (args: {
		messages: Message[]
		settings: ProjectSettings & ExternalSettings
		nodeishFs: NodeishFilesystemSubset
	}) => Promise<void> | void
	addCustomApi?: (args: {
		settings: ProjectSettings & ExternalSettings
	}) =>
		| Record<`app.${string}.${string}`, unknown>
		| { "app.inlang.ideExtension": CustomApiInlangIdeExtension }
	watch?: () => () => void
}

Pro:

• easy for SDK and apps

Cons:

• Plugins get harder to make (weakens the ecosystem)
• Extends plugin api

Proposal 2: Let the plugins tell us where to watch

If the plugins expose something like a watchPath. We can use that universal information to put the watcher in the SDK.

SDK:

const abortController = new AbortController()

	;(async () => {
		try {
			const watcher = nodeishFs.watch(
				watchPath,
				{
					signal: abortController.signal,
				}
			)
			//eslint-disable-next-line @typescript-eslint/no-unused-vars
			for await (const event of watcher) {
				loadAndSetMessages()
			}
		} catch (err: any) {
			if (err.name === "AbortError") return
			throw err
		}
	})()

Plugins:

export type Plugin<
	ExternalSettings extends Record<keyof ExternalProjectSettings, JSONObject> | unknown = unknown
> = Omit<
	Static<typeof Plugin>,
	"loadMessages" | "saveMessages" | "addCustomApi" | "settingsSchema" | "watchPath"
> & {
	settingsSchema?: TObject
	loadMessages?: (args: {
		settings: ProjectSettings & ExternalSettings
		nodeishFs: NodeishFilesystemSubset
	}) => Promise<Message[]> | Message[]
	saveMessages?: (args: {
		messages: Message[]
		settings: ProjectSettings & ExternalSettings
		nodeishFs: NodeishFilesystemSubset
	}) => Promise<void> | void
	addCustomApi?: (args: {
		settings: ProjectSettings & ExternalSettings
	}) =>
		| Record<`app.${string}.${string}`, unknown>
		| { "app.inlang.ideExtension": CustomApiInlangIdeExtension }
	watchPath?: string
}

Pro:

• Easy for plugins

Cons:

• Extends plugin API

Proposal 3: No support for legacy plugins

We check if a filePath is there. Old plugins are not getting watcher support.

Pro:

• Easy and fast
• New paraglide user could use it because we push for messageFormat

Cons

• No superspreader support
• Bigger threshold to adopt if you need to change storage solution

@inlang/sdk @inlang/lix @inlang/plugins

[felixhaeberle]

Message Files

As discussed this morning, I'm strongly voting for Proposal 1, watcher in plugin with plugin settings.

Settings File

Please don't forget that we can also do a watcher in the SDK itself for the project.inlang.json file.

[samuelstroschein]

Hey guys, to clarify again:

Plugins should not receive watch APIs! We can't rely on plugin developers to ensure that watching a) is implemented and b) works

Pseudocode

1. loadProject has a wrapped fs that is passed to loadMessages.
2. wrapped fs stores which file paths have been read with fs.readFile
3. wrapped fs uses the file paths to establish fs.watch stuff
4. when fs.watch triggers, the SDK re-runs loadMessages
5. done

SDK consumers will now have an automatic watching mechanism in place because project.query.messages is reactive and will auto update

Pitfalls

Be aware that saveMessages will cause loadMessages to be called again. This is okay for now. We will introduce #1585 soon which will get rid of this problem as we will 100% control how messages are saved