-
Notifications
You must be signed in to change notification settings - Fork 258
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
✨ Feature request — expose metadata about supported node.js APIs #2097
Comments
// fyi: @petebacondarwin - filing the issue as we discussed. |
Hey! 👋 Is this something you could derive from This function gives you a Cap'n Proto encoded representation of runtime APIs that can be parsed in JavaScript (and maybe converted to a more friendly JSON format ahead-of-time). Lots of the |
thanks for the pointers Brendan! this API could indeed be used to collect the info and produce the metadata file as requested above. I do prefer to have a metadata file as part of the npm package so that we don't need to invoke workerd and compute this info every time we run a build. |
The format should also somehow account for the fact that certain APIs might be exposed purely opt-in without a default on compatibility date. For instance, none of the node.js apis are available unless the {
"dates": {
"2023-05-20": ["buffer#Blob", "assert#AssertionError", "assert#deepEqual", "assert/strict#deepEqual"],
"2024-03-03": ["buffer#Buffer", "path#join"]
},
"flags": {
"foo": ["other#thing"]
}
} |
@jasnell this metadata file assumes that |
Yes, but in the future we might only introduce new node.js APIs behind new opt-in-only flags, those are the ones I'm concerned about. It is also possible to turn on certain apis individually, for instance, there is now a compat flag that makes |
If we introduce node.js apis behind an api-specific opt-in flag, then we can decide on a compat date when this API-specific flag will be on by default when So for the purposes of this metadata list, we only need to know when the API gets enabled along with |
The point is that not all compat flags have default on dates. It's entirely possible that we can add a new API with a flag that never has a default on date. We certainly can set a convention that in the typical case new node.js api additions should have a default-on date, but I don't know if it's a good idea to completely rule it out. |
The use case
As we improve our Node.js compat story, we'd like to dynamically assemble the Node.js compat layer using a mixture of APIs provided via the runtime as well as via polyfills injected at build time (by wrangler or custom build tooling).
For APIs that are already offered by the runtime, the runtime implementation should be used. Since the runtime node.js coverage is expected to grow a little over time, we'd like the build tooling to know what APIs are provided by a given version of runtime and a given compatibility date.
By being able to consider the compatibility date and corresponding Node.js API coverage at build time, we'll be able to fine-time the build-time polyfill to absolute minimum, and also avoid situations where users could build and successfully run an app locally using a runtime that is on npm already but not yet fully rolled out to prod.
Additionally we'll also be able to introduce new Node APIs into the runtime without it becoming a change with unexpected side effects developers.
The ask
We'd like the workerd npm package to contain a metadata file in a format that is easy to parse in javascript (ideally json), that contains information about our node.js coverage enabled using the
--nodejs_compat
flag.The metadata file should group supported node APIs by compat date when they were enabled. The APIs should be listed at the symbol-level granularity.
An example format could look as follows:
nodejs-compat.json
In theory we might want to have a more generic metadata file (e.g.
runtime-compat.json
instead ofnodejs-compat.json
) that lists information about all flags and compat dates which then downstream tooling like wrangler or miniflare could use. I don't have strong opinions about that, as long as such change doesn't expand this scope of this request too much and doesn't result in long delays.Maintenance considerations
Ideally this file is autogenerated rather than manually maintained as part of the bazel build before the package is published.
The text was updated successfully, but these errors were encountered: