poc: Configurable Web VFS

[stevensJourney]

Overview

This POC adds support for configuring different SQLite virtual filesystems on Web.

WA-SQLite's IDBBatchAtomicVFS is still used as the default VFS.

Synchronous Origin Private FileSystem (OPFS) VFS options are added. See WA-SQLite README for more details.

Notes

Configuration

OPFS can be configured with a WASQLiteOpenFactory. These options could be added to the default web constructor.

export const db = new PowerSyncDatabase({
  schema: AppSchema,
  database: new WASQLiteOpenFactory({
    dbFilename: 'examplsw1se112.db',
    vfs: WASQLiteVFS.AccessHandlePoolVFS,
    flags: {
      enableMultiTabs: typeof SharedWorker !== 'undefined'
    } 
  }),
  flags: {
    enableMultiTabs: typeof SharedWorker !== 'undefined'
  }
});

HTTP Requirements

The use of OPFS requires a secure context and additional HTTP headers when serving a web application. These are already documented for the Dart SDK.

For Vite these headers can be configured in the Vite config file:
vite_config.mts

import { defineConfig } from 'vite';

export default defineConfig({
  server: {
    headers: {
      'Cross-Origin-Opener-Policy': 'same-origin',
      'Cross-Origin-Embedder-Policy': 'require-corp',
    },
  },
});

Multiple Tab Support

The OPFS VFS currently only works in dedicated web workers. The current multiple tab support implementation has been updated in order to cater for this. For IndexDB we still use a single SharedWorker which manages a single SQLite connection between tabs. For OPFS each tab uses its own dedicated Worker which has its own SQLite connection. Table update notifications are shared between tabs via a BroadcastChannel. Exclusive locks between tab connections are maintained through Navigator locks as usual.

The initialisation for Web DBAdapters has been streamlined and extended in order to support OPFS. The abstractions and sharing of implementation is managed by new internal APIs and classes.

Safari Multiple Tab Support

The new code structure seemed to fix the long outstanding Safari multiple tab support issues when using pnpm serve but doing a build and using pnpm preview still shows the same problem.

However, using OPFSCoopSyncVFS now provides multiple tab support for both Safari and Safari iOS.
This is not yet enabled by default but can be enabled with an appropriate flag.

export const db = new PowerSyncDatabase({
  schema: AppSchema,
  database: new WASQLiteOpenFactory({
    dbFilename: 'multitab.db',
    vfs: WASQLiteVFS.OPFSCoopSyncVFS,
    flags: {
      enableMultiTabs: typeof SharedWorker !== 'undefined'
    } 
  }),
  flags: {
    enableMultiTabs: typeof SharedWorker !== 'undefined'
  }
});

OPFS VFS and Multiple tabs

The AccessHandlePoolVFS does not seem to work well when more than 1 dedicated worker accesses the filesystem. OPFSCoopSyncVFS works with multiple tabs.

Performance

OPFS (with AccessHandlePoolVFS) has been observed to be significantly faster than the IndexDB filesystem. The time to sync 23_000 rows was compared between IndexDB and OPFS

IndexDB

OPFS

TODOS

• Better warnings for OPFS if configuration is invalid
• Better support for clearing OPFS storage
• Safari Unit tests

[changeset-bot]

🦋 Changeset detected

Latest commit: 77543e3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@powersync/web	Minor
@powersync/diagnostics-app	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[rhashimoto]

Unsolicited drive-by comment:

The use of OPFS requires a secure context and additional HTTP headers when serving a web application. These are already documented for the Dart SDK.

OPFS itself doesn't require the COOP/COEP headers and associated restrictions. Some SQLite VFS implementations use SharedArrayBuffer to provide a synchronous interface for SQLite, and that is what requires COOP/COEP, not OPFS.

However, no example VFS in wa-sqlite uses SharedArrayBuffer. If you are using any VFS from the wa-sqlite project then COOP/COEP headers are unnecessary.

[stevensJourney]

Unsolicited drive-by comment:

The use of OPFS requires a secure context and additional HTTP headers when serving a web application. These are already documented for the Dart SDK.

OPFS itself doesn't require the COOP/COEP headers and associated restrictions. Some SQLite VFS implementations use SharedArrayBuffer to provide a synchronous interface for SQLite, and that is what requires COOP/COEP, not OPFS.

However, no example VFS in wa-sqlite uses SharedArrayBuffer. If you are using any VFS from the wa-sqlite project then COOP/COEP headers are unnecessary.

Thank you for the tip/catch, @rhashimoto! I copied this from our Dart docs without fully verifying it yet, so this is really helpful. I'll review the relevant sections and update them as needed. It's a personal honor to have you comment on this.