-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add web integration document #100
base: master
Are you sure you want to change the base?
Conversation
[`requestVideoFrameCallback()`](https://wicg.github.io/video-rvfc/#dom-htmlvideoelement-requestvideoframecallback) | ||
method [\[VIDEO-RVFC\]](https://wicg.github.io/video-rvfc/) | ||
|
||
### Async completion callbacks |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think Node's legacy APIs also fit into this category - is it worth mentioning them here, despite them not actually being web APIs?
What about a discussion of the transformation from callback to promise to async-await, and the effect that has on the continuation context as a function of the choice of callback snapshot or await restoration? TL;DR as I see it is that our choice here of always restoring callbacks to the registration-time snapshot makes these transformations consistent, but IIUC this is a departure from ALS's behavior, which led to some expectation of flow-through in order to make the transformation behave the same when the callback isn't restored. May or may not make sense to include here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Node.js may choose either way--it can decide for itself whether it feels that the alignment with web or ALS is more important for them. I think it'd be good for someone to work on a separate design document for this, but I don't think it's a Stage 2.7-blocker the same way as web integration is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although this document is specifically about the web integration, because that is a requirement for stage 2.7, I think it should also apply to runtime-specific APIs. Otherwise, that would create an inconsistency in Node.js, since it will have to follow the web API behavior for the web APIs it implements.
I haven't looked at Node.js's APIs in any detail, but if you're talking specifically about async completion callbacks, I don't see how there would be any other possible context for the callbacks other than the time they're passed to the API. After all, the API (let's say fs.readFile()
, for a concrete example) would start an async operation in the background, and the data flow of that operation would go back to the call to fs.readFile()
. I don't think there is any other relevant context here, but @Qard can correct me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still think Node.js can make this tradeoff for themselves, even as I acknowledge the inconsistency risk.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the idea was that fs.readFile
might set some variables internally, and so there's a question of whether those changes would be reflected in the callback. @Qard seemed to suggest that they should, but this is precisely what leads to the inconsistency between callbacks and promises.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is only observable if fs.readFile
does in fact set variables internally. As far as I'm aware, this is not the case for any such APIs in Node.js.
But you're right that a lot of this text doesn't take runtime- (or web platform-) -internal variables into consideration. And things like the WebIDL handling of callbacks is described as simply storing the snapshot and restoring it later, when the needs of scheduler.yield
would mean that very often we'd have to store a different snapshot where the scheduler.yield
variable would change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this document. I overall am happy with the direction of choosing simple registration-time semantics for everything.
My concerns, in descending order of importance:
-
I am worried about the burden this will place on web spec authors.
At first I thought this burden would be minimal, and web spec authors would need to drop down to async snapshot manipulation about as often as they have to drop down to other raw ECMAScript operations: i.e., extremely rarely, only for very special platform features or for patterns that aren't yet codified well into Web IDL. It seemed like auto-capturing for promises and Web IDL callbacks would generally be enough.
But, the section on "Editorial aspects of AsyncContext integration in web specifications" seems to instead say that web spec authors will often have to do these manipulations. Or at least think about whether or not to do these manipulations. Maybe even as often as: every time they queue a task, every time they call a callback that might error, and every time they fire an event.
That seems really bad. I am hopeful I am misunderstanding or there are some confusions here. I think clarifying which of the APIs under "Individual analysis of web APIs and AsyncContext" would need spec patches to use these operations, and what sort of patches they need, would be a good step toward clearing up this confusion.
-
There is a lot of discussion about future extensions to add more context tracking. This kind of unknown future complexity is a bit worrying. How seriously should we be preparing for this future complexity, on the web side? Is there more up-front work to do here, so we can be sure it'll work out? Or would it be OK if we found out later this was unworkable, and we never got anything besides registration-time contexts? (Plus, I guess, a couple that the proposal seems to want to bake in from the start, on
error
andunhandledrejection
events.) -
The document seems to have some editing mistakes per https://github.com/tc39/proposal-async-context/pull/100/files#r1688797847 regarding some
dispatchSnapshot
concept. That is pretty confusing and it'd be good to clear that up. IIUC that concept is gone, but maybe some sections are still discussing it? Similarly there's talk about the rejection context, but the proposal is to only focus on registration time, so why does rejection context matter?
WEB-INTEGRATION.md
Outdated
`AsyncContext.Snapshot` argument to the methods. This might be left for later, | ||
rather than being part of the initial rollout. | ||
|
||
- [`NavigateEvent`](https://html.spec.whatwg.org/multipage/nav-history-apis.html#navigateevent): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not clear what the takeaway is for this item.
WEB-INTEGRATION.md
Outdated
getters of the returned object (e.g. the `processor` method in web audio | ||
worklets, or the `attributeChangedCallback` method of custom elements) also | ||
count as callbacks that these APIs invoke, and which should preserve the | ||
relevant context. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is somewhat related to the old issue whatwg/webidl#701: the way in which these are specified today, by using raw JS machinery instead of a standardized Web IDL pattern, is annoying to get right and error-prone.
Certainly not a blocker, but if someone wanted to shave that yak while they were doing the web integration here, there would be much rejoicing.
WEB-INTEGRATION.md
Outdated
it. Therefore, | ||
[`ErrorEvent`](https://html.spec.whatwg.org/multipage/webappapis.html#errorevent) | ||
will have a `throwSnapshot` property, reflecting the context in which the | ||
exception was thrown. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd appreciate mentioning the alternative to capture the context where the error constructor was called, and allowing developers set any context snapshot appropriate for their cases.
Between this and discussions w/ various frameworks (@rictic on Web Component integration, as well as some colleagues trying to use this both in our internal framework and in Angular), I think the thing to do may be to back off from the simplicity/consistency of "all builtin APIs always snapshot at registration time" to instead have two possible behaviors: (1) registration-time snapshotting, and (2) no context swapping at all, with the latter being the default unless it's more likely to be wrong in most cases (e.g. |
I don't understand how adding an extra choice (registration time vs. not) would make it less burdensome for web spec authors... |
I think in general the "do nothing" default would be correct. |
Are you saying that things like the XHR |
I haven't dug deep into the discussion here, but in case it's useful for y'all, I sketched out the web integration of Task Attribution at https://wicg.github.io/soft-navigations/#sec-task-attribution-algorithms last year. |
I think XHR is one of those cases that will need to be handled specially no matter what we go with as a default. Ultimately, getting the right default matters a lot, because it's important that user code shouldn't need to be aware of AsyncContext at all. It's okay if framework code needs to read snapshots off of event objects (assuming it's got access to them in the right place), but if user code needs to do it, then we've got a problem. I don't know how bad it is to say that XHR events have a different default context than other events. I guess what it would look like is that events would need to either (1) run their handlers in the current context - i.e. no swapping; or (2) run their handlers in a given context that's maybe initialized with the event, and XHR events would set this to the load or send snapshot, error and unhandledrejection events would likely set this to the error/rejection snapshot... unfortunately this does mean we need to figure out all the web events upfront, which is something I was really hoping we could avoid. |
Hey, sorry everyone for the radio silence over here. We've been discussing some of the details of the web integration in the AsyncContext Matrix channel and in our biweekly meetings, and here's a summary of the current state of things: For events, the web integration proposed in this document would call event listeners with the registration context, which would be propagated automatically by WebIDL (through browser- and spec-internal machinery). The dispatch context would be exposed as a property on a limited set of event subclasses, where that set would start intentionally small and could grow over time as more use cases are identified without breaking backwards compatibility. It is in these dispatch context cases where web spec authors would need to manually track the data flow. However, @shicks, @rictic and @jatraman pointed out that using the registration context doesn't work for their use cases, where the AsyncContext user would be a third-party library that needs to track the context through event registrations in first-party code. When the first-party code eventually calls into the third-party library, that library would need to access the event's dispatch context to associate the event with its source, which in the current proposal would need the first-party code to explicitly pass the dispatch context, potentially across multiple function calls – which is the very pain point that AsyncContext aims to prevent in the first place! The alternative option for events would be to always call the listeners with the event's dispatch context if there is one, and to use the initial AsyncContext snapshot otherwise1. For sync dispatch contexts this would not require any extra work in specs or browsers, since the value of the agent's When a web API called from JS eventually causes a task to be queued on the event loop which fires an event, the context that called the web API should be the (async) dispatch context of that event. A common example of this is XHR, where the synchronous call to It's not just XHR, there seem to be many cases where events are caused by a web API but not fired until later. We don't have a full list of such events, because in non-trivial cases, knowing whether an event has an async source requires tracing through the data flow in the spec text or in browser code. So properly tracking the async sources for every event is unfeasible if we want to get AsyncContext shipped anytime soon. @shicks's proposal, as I understand it, is to have a small set of events with async sources which would be handled in the initial rollout, and for any event that isn't handled, the initial AsyncContext snapshot would be used. One big issue with this compared to the current document is that this set would be very hard to extend in the future, since developers might depend on the context for a specific event being the initial one, so changing that context might mean breaking the web. It would also be quite hard for browsers to get telemetry about the usage of such contexts, which makes it more risky to changing the context in the future. It seems like there are no great solutions, and the fact that the users of these events would typically be first-party developers, whereas AsyncContext would be used in third-party libraries, makes it even harder to have a clear idea of the use cases we need to support. TL;DR: We're considering changing the context in which event listeners are called to the dispatch context, not the registration context. Handling this properly for all events would require such amount of work in the specs and browsers that it would probably be unfeasible. But if we only handle a subset of events in the initial rollout, the rest of events might not be able to be fixed in the future without breaking the web. Footnotes
|
As an update: we're currently looking into whether it's possible to hook into the HTML spec's event loop mechanisms, in particular "queue a task" and "in parallel" to automatically propagate the context across any JS code execution in the same agent that is caused, directly or indirectly, by a previous execution. Doing this would automatically make most events that are dispatched asynchronously have the right async dispatch context, so most spec authors would not have to worry about it. However, it is not fully clear to us how hard it would be to implement this in browsers, and whether it can be implemented in a way such that most browser contributors also don't have to worry about it looking forwards. This gets even more complicated when you take into account the difficulty of somehow propagating contexts behind the scenes across cross-thread and cross-process data flows, while making sure the snapshot and the data contained get properly GC'd. |
I updated this document to prefer using the dispatch context, and only use the registration context if the API in question can never have a dispatch context. I replaced most of the event sections with a TODO that @nicolo-ribaudo will fill with their fallback context proposal. I also added the automatic context propagation to the editorial aspects section, which will be added to more in the future as we figure out the details. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me. I think we should merge even if there are TODOs, and then iterate on them as follow-up PRs. This gives a more clear picture of what the current status is, when looking at the proposal repo.
The document on events that @andreubotella mentions is https://hackmd.io/@nicolo-ribaudo/rJhdk8HyJe -- I'll open a PR with it as soon as this is merged.
Given this, for consistency it would be preferable to instead use the | ||
registration context; that is, the context in which the class is constructed. | ||
|
||
- [`MutationObserver`](https://dom.spec.whatwg.org/#mutationobserver) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a possible data point, this is already how console.trace
builds its stack inside MutationObserver, both in Chrome and Firefox.
Co-authored-by: Nicolò Ribaudo <[email protected]>
Co-authored-by: Nicolò Ribaudo <[email protected]>
I still don't understand the desire for this change, from a user that isn't aware of Like for a typical user But under the proposed change if the user say tries to say use some logger: function onHitObject(obj) {
logger.log("Clicked the object", obj);
// ... do whatever for this object
}
logger.file(`my-game-${ new Date().toString() }`, () => {
function onFrame() {
logger.log("Rendering scene");
requestAnimationFrame(onFrame);
}
requestAnimationFrame(onFrame);
canvas.body.addEventListener("click", (event) => {
const hitObject = findHitObject(event);
onHitObject(hitObject);
});
}); then from the users perspective the logging in the event listener just vanishes for no coherent reason and instead happens in the useless fallback context (there's also no way for What will happen is users will google it, find they have to add Like the whole point of |
APIs like
If you imagine an API where these two steps are separate (e.g. let tr;
let et = new EventTarget();
let cb;
let it;
logger.file(`game1`, () => {
tr = createTimerReaction(function() { logger.log("timer") });
et.addEventListener("foo", function() { logger.log("event") });
cb = function () { logger.log("callback") };
it = Iterator.from([1, 2, 3]).map(function () { logger.log("map") });
});
tr.scheduleEveryXms(100);
et.dispatchEvent(new Event("foo"));
cb.call();
it.next(); these three ways of "providing a callback that then gets called" can all be consistent and all log something "empty". The reason let tr;
let et = new EventTarget();
let cb;
let it;
logger.file(`game1`, () => {
tr = createTimerReaction(function() { logger.log("timer") });
tr.scheduleEveryXms(100);
// the above can be "abstracted away" as setInterval(function() { logger.log("timer") }, 100);
et.addEventListener("foo", function() { logger.log("event") });
et.dispatchEvent(new Event("foo"));
cb = function () { logger.log("callback") };
cb.call();
it = Iterator.from([1, 2, 3]).map(function () { logger.log("map") });
it.next();
}); here all of them would log
Actually, the plan is to provide it. I will need to open a PR for it, but it is linked in #100 (review). The "fallback context" needs to be configurable, and it needs to be configurable without explicit opt-in where you call The idea is that the EventTarget fallback is defined through an AsyncContext variable which defaults to the empty context, but it can be set to a different snapshot. When you call |
So the mechanism suggested (
These drawbacks aren't surprising, they come from the fact that I still think the best solution is what already has been suggested, have different types of variables for the different types of flow. (I also wrote a large bit about how registration time is like implicit parameters, but dispatch time is like implicit returns, with an example implementation). |
This unfortunately cuts both ways. Registration context may be what you want for some variables and some events some of the time, but (as the "feedback from multiple parties" indicated) there are also many cases where you want the opposite, and unfortunately there is no clear rule that always gets it right.
I wonder if this hints at a compelling use case for treating fallback contexts as a tree? The
Is it clear that the new
From my experience, I would prefer to get the dispatch context here. As I mentioned above, there's simply no "always right" answer. If a particular framework knows it's more appropriate to use registration context, then it may be possible to have it do the wrapping before it registers the event.
I'm honestly skeptical that this would ever be viable. If we just have two types of variables (i.e. registration vs dispatch) then we're still conflating a bunch of different things together, and will fail to address every use case - there's too many different types of context merge to distill it down to a single axis, e.g. I want something that preserves across |
This is precisely why I argue there should both kinds of variables.
There are use cases that extend beyond just those who have private contact with the proposal authors. And again having both types of variables allows individual use cases to use either or both depending on what they want to do.
This is not a framework, it is a
This is simply not true, the example implementation I made to prove that is possible has the essentially the same characteristics for both in and out flows† (if you replace the † There is a special case with (The implementation is not even particularly complicated, you just have one store for variables that flow inwards, and one for variables that flow outwards). I would also like to make note of how having both flows can improve even the existing use cases. For example in tracing we generally want to know why something happened, by having a variable that has both flows, in-flow and up-flow, we can accurately show both the root task that caused the evaluation AND know who is responsible for the registration: declare namespace $AsyncContext {
export class Variable<T> {
getInFlow(): T | undefined;
getOutFlow(): T | undefined;
// Used for dependency injection use cases
runWithInjectionFlow<R>(value: T, cb: () => R): R;
// I feel most use cases are better that need returns probably want bothFlows
// however some types of tracers may wish to ignore registration time in which
// case they can use this instead
runWithContinuationFlow<R>(value: T, cb: () => R): R;
// Useful for tracing and similar like below
runWithBothFlows<R>(value: T, cb: () => R): R;
// ALSO POSSIBLE, and probably useful:
// sets the continuation synchronously which then bubbles up the stack, useful for
// cases that would need variable.set() or similar
setContinuation<T>(value: T): void;
// clears the continuation, useful if you want a barrier where continuation flow
// doesn't leak out
deleteContinuation(): void
}
}
export default class Tracer {
// Probably more useful as a list, but this example doesn't need it
readonly #tracingVariable = new $AsyncContext.Variable<string>();
span<R>(name: string, start: () => R): R {
return this.#tracingVariable.runWithBothFlows(name, start);
}
trace(): { triggeredBy: string | undefined; registeredBy: string | undefined } {
const triggeredBy = this.#tracingVariable.getOutFlow();
const registeredBy = this.#tracingVariable.getInFlow();
return { triggeredBy, registeredBy };
}
}
const tracer = new Tracer();
const el = document.getElementById("someElement");
tracer.span("my-game", () => {
el?.addEventListener("click", () => {
// Prints { triggeredBy: "user-interaction", registeredBy: "my-game" }
console.log(tracer.trace());
});
});
tracer.span("user-interaction", () => {
el?.click();
}); Other use cases can use multiple flows in similar ways. |
I'm confused by how we are talking about something generic, and not specific to .addEventListener. If it's generic, it's clear to me that one of the two flows is the one that comes from when the function it's called, but it's not at all clear what should the other one be. Is it the one that a function captures lexically? Is it the one from the first time the function is passed as a parameter to another function? Is it the one of the last time it's passed as a parameter to another function? The "out flow" and "in flow" you are describing seem very similar to call-time function parameters and bind-time function parameters: let fn;
myVar.run("foo", () => {
fn = function (inFlow, outFlow) {
console.log({ inFlow, outFlow });
}.bind(null, myVar.get());
});
myVar.run("bar", () => {
fn(myVar.get());
}); logs Maybe this is similar to how |
The reason is because the proposed change to
This is partially correct, a simple model is just to consider it in terms of namespace SyncContext {
let implicitParams = new Map<SyncContext.Variable, any>();
let implicitReturn = new Map<SyncContext.Variable, any>();
export class Variable<T> {
getParam(): T | undefined {
return implicitParams.get(this);
}
getReturn(): T | undefined {
return implicitReturn.get(this);
}
// Basically the same as AsyncContext
runWithImplicitParam<T>(value: T, cb: () => R): R {
const previousImplicitParams = implicitParams;
try {
implicitParams = new Map([...implicitParams, [this, value]));
return cb();
} finally {
implicitParams = previousImplicitParams;
}
}
// This doesn't translate well to all asynchronous behaviours as callbacks
// act as returns, that's why there's runWithContinuationFlow so that it flows
// as a return value into the callback
setImplicitReturn(value: T): void {
implicitReturn = new Map<SyncContext.Variable, any>();
}
// Deletes the implicit return of this value so that it doesn't propogate up
// this can be used as a barrier like in the later example with element.events
// I show later
deleteImplicitReturn(): void {
implicitReturn = new Map<SyncContext.Variable, any>();
}
}
}
const var1 = new SyncContext.Variable<string>();
const var2 = new SyncContext.Variable<string>();
function foo() {
const v1 = var1.get(); // one in the following
var2.setImplicitReturn("two");
return 10;
}
const r = var1.runWithImplicitParam("one", foo);
const v2 = var2.get(); If sync context didn't exist, it would just be like if we had passed params in explictly and returned explictly: function foo(v1: string) {
const v2 = "two";
return [10, v2];
}
const [r, v2] = foo("one") Now what makes asynchronous functions special, is that "returns" are done by passing parameters to callbacks (even for promises, despite // v2 is "returned" from foo by way of the callback
function foo(v1: string, cb: (ret: number, v2: string) => void) {
const v2 = "two";
cb([10, v2]);
} Then async-ifying implicit returns is equivalent to setting a return context before calling the callback, this is exactly the flow that being argued for in the other issue. The In the current design, by using registration time it's basically like as say called To see this more clearly, suppose that asynchronous programming didn't exist in JavaScript and instead events were simply blocking synchronous iterators: startThread {
for (const clickEvent of element.events("click")) {
// do something with clickEvent
}
} If you wanted to return contextual information such as what interaction caused the event, then you obviously can't use implicit parameters, because the iterator returns the event to the for-loop. i.e. The example I gave above where I captured that startThread {
for (const clickEvent of element.events("click")) {
// do something with clickEvent
console.log(tracing.cause()); // either user-interaction or fake-event
}
}
startThread {
sleep(Math.random());
// This would call .setImplicitReturn("fake-event"), call the callback, then
// .deleteImplicitReturn() so that the value doesn't propagate out of this callback
tracing.cause("fake-event", () => {
// Whatever mechanism allows blocking in the other thread would restore the
// the return context of whatever triggers the event, JS doesn't have threads
// but resuming paused async functions is functionally very similar
element.dispatchEvent(clickEvent);
});
} Now an argument you might have, is that in synchronous land we could still write startThread {
element.events("click", (evt) => {
// var.get()
});
} where startThread {
const logFile = new SyncContext.Variable();
logFile.run("./log.txt", () => {
element.events("click", (evt) => {
console.log(logFile.get()); // Still "log.txt"
console.log(hostTracingForWhereThisEventCameFrom.get()); // user-interaction at some time, some place
});
});
} This API is suspiciously similar to the asynchronous case (even though it technically synchronous), and so the same things apply, we could set an implicit return and send it into the event callback without affecting the implicit parameters. Essentially my suggestion is that we implement these asynchronous return flows, this unlocks the use cases for |
No description provided.