go-pmtiles to WASM

[patrykkalinowski]

I wonder if anyone tried to compile go-pmtiles to Web Assembly?

My end goal is to have extract function available directly in browser, so user can select rectangle on map in browser and download it without relying on external server (user is non technical, so no way of running pmtiles-cli themselves). I'm not sure it's even possible, though.

My findings so far:

• sqlite is not supported in WASM, so convert function cannot be compiled. Removing pmtiles/convert.go and references to it from main.go allows program to compile to WASM
• 'Failed to extract, stat .: not implemented on js' - this is an error I get while trying to use extract function from JS, I didn't figure out how to work around this yet. Looks like os.stat, which is expected as WASM won't have access to filesystem.

My current idea is to try output extract results back to JS as blob, so I can save it in browser storage.

[bdon]

We would need to re-structure the go API to separate I/O, and then implement the I/O using fetch in javascript and pass to WASM. This isn't on the roadmap right now. An easier approach may be to implement the entire extract algorithm in JS.

[bdon]

The main obstacle to implementing extract in JavaScript is there's no implementation of 64-bit Roaring Bitmaps in JavaScript. There is an Emscripten/WASM port but it seems to be 32-bit only.

[prusswan]

This would be useful if it means local pmtiles can be accessed in the browser and used as a map (without requiring range requests or an actual server).

[bdon]

It's already possible to access local PMTiles if you have the entire thing on disk or local storage using the existing JavaScript client and File API. https://pmtiles.io can open local files.

[prusswan]

The thing is, I hope it can be less intrusive (e.g. skip the manual drag and drop and able to register a fixed list of local files via inline JavaScript, like how DuckDB wasm manages it with registerFileHandle: https://duckdb.org/docs/api/wasm/data_ingestion

…

On Thu, Sep 5, 2024, 1:06 PM Brandon Liu ***@***.***> wrote: It's already possible to access local PMTiles if you have the entire thing on disk or local storage using the existing JavaScript client and File API <https://developer.mozilla.org/en-US/docs/Web/API/File_API>. https://pmtiles.io can open local files. — Reply to this email directly, view it on GitHub <#13 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFEBYUA5O2CCQMDAUCGWSLZU7RGBAVCNFSM6AAAAABFIL43ZOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJVGMYTEMI> . You are receiving this because you commented.Message ID: <protomaps/. ***@***.***>

[prusswan]

Okay, I rechecked and realized the user interaction is still needed. No-go for now.

But what if the pmtiles data is part of the WASM binary? It would require the same set of permissions? Then I could use a single WASM with both DuckDB functionality and some bundled vector tiles

[bdon]

It wouldn't be practical to store the data in the WASM blob as there's a size limitation and PMTiles can easily be 10+GB (and client code should work the same for small or large archives). what would be the benefit?

[prusswan]

It wouldn't be practical to store the data in the WASM blob as there's a size limitation and PMTiles can easily be 10+GB (and client code should work the same for small or large archives). what would be the benefit?

If the data is confined to a small area, this can be distributed for users who just want to work on styling without having to care about server or CORS or other network issues. With FileSource, a very crude way is also possible by encoding the PMTiles content into string and loading it from a separate JS file.

[bdon]

If you want to load a WASM blob you need a server and CORS to host the WASM bundle, which is no different than a .pmtiles file.

[prusswan]

If you want to load a WASM blob you need a server and CORS to host the WASM bundle, which is no different than a .pmtiles file.

I mean something like the bare browser setup for duckdb-wasm (https://github.com/duckdb/duckdb-wasm/blob/main/examples/bare-browser/index.html) which works over file://