Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add web integration document #100

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

andreubotella
Copy link
Member

No description provided.

[`requestVideoFrameCallback()`](https://wicg.github.io/video-rvfc/#dom-htmlvideoelement-requestvideoframecallback)
method [\[VIDEO-RVFC\]](https://wicg.github.io/video-rvfc/)

### Async completion callbacks
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Node's legacy APIs also fit into this category - is it worth mentioning them here, despite them not actually being web APIs?

What about a discussion of the transformation from callback to promise to async-await, and the effect that has on the continuation context as a function of the choice of callback snapshot or await restoration? TL;DR as I see it is that our choice here of always restoring callbacks to the registration-time snapshot makes these transformations consistent, but IIUC this is a departure from ALS's behavior, which led to some expectation of flow-through in order to make the transformation behave the same when the callback isn't restored. May or may not make sense to include here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node.js may choose either way--it can decide for itself whether it feels that the alignment with web or ALS is more important for them. I think it'd be good for someone to work on a separate design document for this, but I don't think it's a Stage 2.7-blocker the same way as web integration is.

Copy link
Member Author

@andreubotella andreubotella Jul 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although this document is specifically about the web integration, because that is a requirement for stage 2.7, I think it should also apply to runtime-specific APIs. Otherwise, that would create an inconsistency in Node.js, since it will have to follow the web API behavior for the web APIs it implements.

I haven't looked at Node.js's APIs in any detail, but if you're talking specifically about async completion callbacks, I don't see how there would be any other possible context for the callbacks other than the time they're passed to the API. After all, the API (let's say fs.readFile(), for a concrete example) would start an async operation in the background, and the data flow of that operation would go back to the call to fs.readFile(). I don't think there is any other relevant context here, but @Qard can correct me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think Node.js can make this tradeoff for themselves, even as I acknowledge the inconsistency risk.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the idea was that fs.readFile might set some variables internally, and so there's a question of whether those changes would be reflected in the callback. @Qard seemed to suggest that they should, but this is precisely what leads to the inconsistency between callbacks and promises.

Copy link
Member Author

@andreubotella andreubotella Aug 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only observable if fs.readFile does in fact set variables internally. As far as I'm aware, this is not the case for any such APIs in Node.js.

But you're right that a lot of this text doesn't take runtime- (or web platform-) -internal variables into consideration. And things like the WebIDL handling of callbacks is described as simply storing the snapshot and restoring it later, when the needs of scheduler.yield would mean that very often we'd have to store a different snapshot where the scheduler.yield variable would change.

WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
Copy link
Member

@domenic domenic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this document. I overall am happy with the direction of choosing simple registration-time semantics for everything.

My concerns, in descending order of importance:

  • I am worried about the burden this will place on web spec authors.

    At first I thought this burden would be minimal, and web spec authors would need to drop down to async snapshot manipulation about as often as they have to drop down to other raw ECMAScript operations: i.e., extremely rarely, only for very special platform features or for patterns that aren't yet codified well into Web IDL. It seemed like auto-capturing for promises and Web IDL callbacks would generally be enough.

    But, the section on "Editorial aspects of AsyncContext integration in web specifications" seems to instead say that web spec authors will often have to do these manipulations. Or at least think about whether or not to do these manipulations. Maybe even as often as: every time they queue a task, every time they call a callback that might error, and every time they fire an event.

    That seems really bad. I am hopeful I am misunderstanding or there are some confusions here. I think clarifying which of the APIs under "Individual analysis of web APIs and AsyncContext" would need spec patches to use these operations, and what sort of patches they need, would be a good step toward clearing up this confusion.

  • There is a lot of discussion about future extensions to add more context tracking. This kind of unknown future complexity is a bit worrying. How seriously should we be preparing for this future complexity, on the web side? Is there more up-front work to do here, so we can be sure it'll work out? Or would it be OK if we found out later this was unworkable, and we never got anything besides registration-time contexts? (Plus, I guess, a couple that the proposal seems to want to bake in from the start, on error and unhandledrejection events.)

  • The document seems to have some editing mistakes per https://github.com/tc39/proposal-async-context/pull/100/files#r1688797847 regarding some dispatchSnapshot concept. That is pretty confusing and it'd be good to clear that up. IIUC that concept is gone, but maybe some sections are still discussing it? Similarly there's talk about the rejection context, but the proposal is to only focus on registration time, so why does rejection context matter?

WEB-INTEGRATION.md Outdated Show resolved Hide resolved
`AsyncContext.Snapshot` argument to the methods. This might be left for later,
rather than being part of the initial rollout.

- [`NavigateEvent`](https://html.spec.whatwg.org/multipage/nav-history-apis.html#navigateevent):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not clear what the takeaway is for this item.

getters of the returned object (e.g. the `processor` method in web audio
worklets, or the `attributeChangedCallback` method of custom elements) also
count as callbacks that these APIs invoke, and which should preserve the
relevant context.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is somewhat related to the old issue whatwg/webidl#701: the way in which these are specified today, by using raw JS machinery instead of a standardized Web IDL pattern, is annoying to get right and error-prone.

Certainly not a blocker, but if someone wanted to shave that yak while they were doing the web integration here, there would be much rejoicing.

WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
it. Therefore,
[`ErrorEvent`](https://html.spec.whatwg.org/multipage/webappapis.html#errorevent)
will have a `throwSnapshot` property, reflecting the context in which the
exception was thrown.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd appreciate mentioning the alternative to capture the context where the error constructor was called, and allowing developers set any context snapshot appropriate for their cases.

@shicks
Copy link
Contributor

shicks commented Aug 6, 2024

@domenic

  • I am worried about the burden this will place on web spec authors.
    At first I thought this burden would be minimal, and web spec authors would need to drop down to async snapshot manipulation about as often as they have to drop down to other raw ECMAScript operations: i.e., extremely rarely, only for very special platform features or for patterns that aren't yet codified well into Web IDL. It seemed like auto-capturing for promises and Web IDL callbacks would generally be enough.
    But, the section on "Editorial aspects of AsyncContext integration in web specifications" seems to instead say that web spec authors will often have to do these manipulations. Or at least think about whether or not to do these manipulations. Maybe even as often as: every time they queue a task, every time they call a callback that might error, and every time they fire an event.
    That seems really bad. I am hopeful I am misunderstanding or there are some confusions here. I think clarifying which of the APIs under "Individual analysis of web APIs and AsyncContext" would need spec patches to use these operations, and what sort of patches they need, would be a good step toward clearing up this confusion.

Between this and discussions w/ various frameworks (@rictic on Web Component integration, as well as some colleagues trying to use this both in our internal framework and in Angular), I think the thing to do may be to back off from the simplicity/consistency of "all builtin APIs always snapshot at registration time" to instead have two possible behaviors: (1) registration-time snapshotting, and (2) no context swapping at all, with the latter being the default unless it's more likely to be wrong in most cases (e.g. setTimeout). This is both more relevant for most APIs, and is less burdensome for web spec authors, since it's what will already happen in most cases, and/or what would fall out from just specifying how the swapping behaves outside of code execution contexts (i.e. in the event loop).

@domenic
Copy link
Member

domenic commented Aug 6, 2024

I don't understand how adding an extra choice (registration time vs. not) would make it less burdensome for web spec authors...

@shicks
Copy link
Contributor

shicks commented Aug 6, 2024

I think in general the "do nothing" default would be correct.

@andreubotella
Copy link
Member Author

andreubotella commented Aug 6, 2024

Between this and discussions w/ various frameworks (@rictic on Web Component integration, as well as some colleagues trying to use this both in our internal framework and in Angular), I think the thing to do may be to back off from the simplicity/consistency of "all builtin APIs always snapshot at registration time" to instead have two possible behaviors: (1) registration-time snapshotting, and (2) no context swapping at all, with the latter being the default unless it's more likely to be wrong in most cases (e.g. setTimeout). This is both more relevant for most APIs, and is less burdensome for web spec authors, since it's what will already happen in most cases, and/or what would fall out from just specifying how the swapping behaves outside of code execution contexts (i.e. in the event loop).

Are you saying that things like the XHR load event should not have any way to get back to the xhr.send() context? Because that is the kind of thing that would not be handled automatically by WebIDL in the current proposal, and that would need to be handled by spec authors. Unless we make it so queueing a new task in the event loop always propagates the snapshot, even when going "in parallel" (i.e. spawning a thread to avoid blocking the event loop), which I suspect is a whole 'nother can of worms.

@yoavweiss
Copy link

I haven't dug deep into the discussion here, but in case it's useful for y'all, I sketched out the web integration of Task Attribution at https://wicg.github.io/soft-navigations/#sec-task-attribution-algorithms last year.

@shicks
Copy link
Contributor

shicks commented Aug 6, 2024

@andreubotella

Are you saying that things like the XHR load event should not have any way to get back to the xhr.send() context? Because that is the kind of thing that would not be handled automatically by WebIDL in the current proposal, and that would need to be handled by spec authors. Unless we make it so queueing a new task in the event loop always propagates the snapshot, even when going "in parallel" (i.e. spawning a thread to avoid blocking the event loop), which I suspect is a whole 'nother can of worms.

I think XHR is one of those cases that will need to be handled specially no matter what we go with as a default. Ultimately, getting the right default matters a lot, because it's important that user code shouldn't need to be aware of AsyncContext at all. It's okay if framework code needs to read snapshots off of event objects (assuming it's got access to them in the right place), but if user code needs to do it, then we've got a problem.

I don't know how bad it is to say that XHR events have a different default context than other events. I guess what it would look like is that events would need to either (1) run their handlers in the current context - i.e. no swapping; or (2) run their handlers in a given context that's maybe initialized with the event, and XHR events would set this to the load or send snapshot, error and unhandledrejection events would likely set this to the error/rejection snapshot... unfortunately this does mean we need to figure out all the web events upfront, which is something I was really hoping we could avoid.

@andreubotella
Copy link
Member Author

andreubotella commented Aug 20, 2024

Hey, sorry everyone for the radio silence over here. We've been discussing some of the details of the web integration in the AsyncContext Matrix channel and in our biweekly meetings, and here's a summary of the current state of things:

For events, the web integration proposed in this document would call event listeners with the registration context, which would be propagated automatically by WebIDL (through browser- and spec-internal machinery). The dispatch context would be exposed as a property on a limited set of event subclasses, where that set would start intentionally small and could grow over time as more use cases are identified without breaking backwards compatibility. It is in these dispatch context cases where web spec authors would need to manually track the data flow.

However, @shicks, @rictic and @jatraman pointed out that using the registration context doesn't work for their use cases, where the AsyncContext user would be a third-party library that needs to track the context through event registrations in first-party code. When the first-party code eventually calls into the third-party library, that library would need to access the event's dispatch context to associate the event with its source, which in the current proposal would need the first-party code to explicitly pass the dispatch context, potentially across multiple function calls – which is the very pain point that AsyncContext aims to prevent in the first place!

The alternative option for events would be to always call the listeners with the event's dispatch context if there is one, and to use the initial AsyncContext snapshot otherwise1. For sync dispatch contexts this would not require any extra work in specs or browsers, since the value of the agent's [[AsyncContextMapping]] field when the event is fired would be correct. Similarly, for event dispatches without a dispatch context (i.e. events not caused by JS, such as a user click), [[AsyncContextMapping]] would be set to its default value, which is the initial snapshot. The problem is with async dispatch contexts.

When a web API called from JS eventually causes a task to be queued on the event loop which fires an event, the context that called the web API should be the (async) dispatch context of that event. A common example of this is XHR, where the synchronous call to xhr.send() starts an asynchronous fetch which eventually fires the e.g. load event. And currently there is no mechanism in web specs or browsers that could automatically propagate the [[AsyncContextMapping]] across async tasks, so the event listener would have the initial AsyncContext snapshot by default, which would not be the right context.

It's not just XHR, there seem to be many cases where events are caused by a web API but not fired until later. We don't have a full list of such events, because in non-trivial cases, knowing whether an event has an async source requires tracing through the data flow in the spec text or in browser code. So properly tracking the async sources for every event is unfeasible if we want to get AsyncContext shipped anytime soon.

@shicks's proposal, as I understand it, is to have a small set of events with async sources which would be handled in the initial rollout, and for any event that isn't handled, the initial AsyncContext snapshot would be used. One big issue with this compared to the current document is that this set would be very hard to extend in the future, since developers might depend on the context for a specific event being the initial one, so changing that context might mean breaking the web. It would also be quite hard for browsers to get telemetry about the usage of such contexts, which makes it more risky to changing the context in the future.

It seems like there are no great solutions, and the fact that the users of these events would typically be first-party developers, whereas AsyncContext would be used in third-party libraries, makes it even harder to have a clear idea of the use cases we need to support.


TL;DR: We're considering changing the context in which event listeners are called to the dispatch context, not the registration context. Handling this properly for all events would require such amount of work in the specs and browsers that it would probably be unfeasible. But if we only handle a subset of events in the initial rollout, the rest of events might not be able to be fixed in the future without breaking the web.

Footnotes

  1. Or alternatively, the registration context could also be stored and used when there is no other context.

@andreubotella
Copy link
Member Author

andreubotella commented Sep 9, 2024

As an update: we're currently looking into whether it's possible to hook into the HTML spec's event loop mechanisms, in particular "queue a task" and "in parallel" to automatically propagate the context across any JS code execution in the same agent that is caused, directly or indirectly, by a previous execution. Doing this would automatically make most events that are dispatched asynchronously have the right async dispatch context, so most spec authors would not have to worry about it.

However, it is not fully clear to us how hard it would be to implement this in browsers, and whether it can be implemented in a way such that most browser contributors also don't have to worry about it looking forwards. This gets even more complicated when you take into account the difficulty of somehow propagating contexts behind the scenes across cross-thread and cross-process data flows, while making sure the snapshot and the data contained get properly GC'd.

@andreubotella
Copy link
Member Author

I updated this document to prefer using the dispatch context, and only use the registration context if the API in question can never have a dispatch context. I replaced most of the event sections with a TODO that @nicolo-ribaudo will fill with their fallback context proposal. I also added the automatic context propagation to the editorial aspects section, which will be added to more in the future as we figure out the details.

Copy link
Member

@nicolo-ribaudo nicolo-ribaudo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I think we should merge even if there are TODOs, and then iterate on them as follow-up PRs. This gives a more clear picture of what the current status is, when looking at the proposal repo.

The document on events that @andreubotella mentions is https://hackmd.io/@nicolo-ribaudo/rJhdk8HyJe -- I'll open a PR with it as soon as this is merged.

WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
WEB-INTEGRATION.md Outdated Show resolved Hide resolved
Given this, for consistency it would be preferable to instead use the
registration context; that is, the context in which the class is constructed.

- [`MutationObserver`](https://dom.spec.whatwg.org/#mutationobserver)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a possible data point, this is already how console.trace builds its stack inside MutationObserver, both in Chrome and Firefox.

andreubotella and others added 2 commits December 2, 2024 04:24
Co-authored-by: Nicolò Ribaudo <[email protected]>
@Jamesernator
Copy link

Jamesernator commented Dec 3, 2024

After feedback from multiple parties, we changed EventTarget to run callbacks in the context that triggered the event (aka "dispatch context")
When there is no dispatch context (e.g. user clicks on a button), it falls back to the root context

I still don't understand the desire for this change, from a user that isn't aware of AsyncContext the mental model is basically incoherent.

Like for a typical user setInterval(() => {}, time) and document.body.addEventListener("click", () => {}) are essentially the same, both take a callback and call it multiple times based on some trigger.

But under the proposed change if the user say tries to say use some logger:

function onHitObject(obj) {
    logger.log("Clicked the object", obj);
    // ... do whatever for this object
}

logger.file(`my-game-${ new Date().toString() }`, () => {
    function onFrame() {
        logger.log("Rendering scene");
        requestAnimationFrame(onFrame);
    }
    
    requestAnimationFrame(onFrame);
    
    canvas.body.addEventListener("click", (event) => {
        const hitObject = findHitObject(event);
        onHitObject(hitObject);
    });
});

then from the users perspective the logging in the event listener just vanishes for no coherent reason and instead happens in the useless fallback context (there's also no way for logger to even repair the context for the user as the registration context is simply lost).

What will happen is users will google it, find they have to add AsyncContext.Snapshot.wrap to the callback and do that, but they still won't have a mental model for what callbacks are affected or not (because well there isn't a coherent mental model, it's just whatever some API decides arbitrarily is the context that should be used), so most likely the user in future will just defensively add AsyncContext.Snapshot.wrap defensively.

Like the whole point of AsyncContext is that it is SUPPOSED TO BE TRANSPARENT for users, if users can add AsyncContext.Snapshot.wrap randomly around the place they can just as easily thread objects like logger around.

@nicolo-ribaudo
Copy link
Member

nicolo-ribaudo commented Dec 3, 2024

APIs like setInterval are a bit confusing in this discussion because they do two things with a single function call:

  • they receive a callback to run later
  • they schedule it to actually be X ms later

If you imagine an API where these two steps are separate (e.g. let timer = createTimerReaction(function() { ... }); timer.scheduleEveryXms(100)), then:

let tr;
let et = new EventTarget();
let cb;
let it;

logger.file(`game1`, () => {
  tr = createTimerReaction(function() { logger.log("timer") });
  et.addEventListener("foo", function() { logger.log("event") });
  cb  = function () { logger.log("callback") };
  it = Iterator.from([1, 2, 3]).map(function () { logger.log("map") });
});

tr.scheduleEveryXms(100);
et.dispatchEvent(new Event("foo"));
cb.call();
it.next();

these three ways of "providing a callback that then gets called" can all be consistent and all log something "empty". The reason setTimeout&co can be confusing is because they force you to put the two actions (provide a callback, and cause it to be called) in the same place. If we do the same with events, then everything still behaves consistently:

let tr;
let et = new EventTarget();
let cb;
let it;

logger.file(`game1`, () => {
  tr = createTimerReaction(function() { logger.log("timer") });
  tr.scheduleEveryXms(100);
  // the above can be "abstracted away" as setInterval(function() { logger.log("timer") }, 100);

  et.addEventListener("foo", function() { logger.log("event") });
  et.dispatchEvent(new Event("foo"));

  cb  = function () { logger.log("callback") };
  cb.call();

  it = Iterator.from([1, 2, 3]).map(function () { logger.log("map") });
  it.next();
});

here all of them would log game1.

there's also no way for logger to even repair the context for the user as the registration context is simply lost

Actually, the plan is to provide it. I will need to open a PR for it, but it is linked in #100 (review). The "fallback context" needs to be configurable, and it needs to be configurable without explicit opt-in where you call .addEventListener (otherwise, you'd just use AsyncContext.wrap).

The idea is that the EventTarget fallback is defined through an AsyncContext variable which defaults to the empty context, but it can be set to a different snapshot. When you call .addEventListener, it reads that AsyncContext variable and stores the fallback together with the callback, so that if there is no proper context to run the callback in it knows which one was defined as the fallback.

@Jamesernator
Copy link

Jamesernator commented Dec 3, 2024

The idea is that the EventTarget fallback is defined through an AsyncContext variable which defaults to the empty context, but it can be set to a different snapshot.

So the mechanism suggested (EventTarget.captureFallbackContext) certainly seems like a clever hack, but it does have some notable draw backs:

  • One, it's not possible to opt in just variables that need the feature, using EventTarget.captureFallbackContext also opts in this behaviour for all other variables even if they don't want or need this context
    • e.g. If we have:
    logger.file = function(logFile, cb) {
        logVariable.run(logFile, () => EventTarget.captureFallbackContext(() => cb()));
    }
    then running this will cause unrelated variables to be in a different context:
    const v = new AsyncContext.Variable();
    // Captures var1="value" for logger.file even though the logger only needed it's own
    // variable captured
    var1.run("value", () => logger.file("log.txt", () => t.addEventListener("e", () => {
        logger.log("Hello");
    })));
  • Two, as a consequence of the previous point the lifetimes of variables can (and will) be accidentally extended
  • Three, there's still no way to capture registration time for synchronous dispatched callbacks, i.e. if a click happens because a webcomponent or such calls element.click() then EventTarget.captureFallbackContext still can't repair the context as it's not used

These drawbacks aren't surprising, they come from the fact that Event.captureFallbackContext is trying to simulate two different kinds of variable flow, but it fails to emulate either flow correctly for all variables.

I still think the best solution is what already has been suggested, have different types of variables for the different types of flow. (I also wrote a large bit about how registration time is like implicit parameters, but dispatch time is like implicit returns, with an example implementation).

@shicks
Copy link
Contributor

shicks commented Dec 4, 2024

Like the whole point of AsyncContext is that it is SUPPOSED TO BE TRANSPARENT for users, if users can add AsyncContext.Snapshot.wrap randomly around the place they can just as easily thread objects like logger around.

This unfortunately cuts both ways. Registration context may be what you want for some variables and some events some of the time, but (as the "feedback from multiple parties" indicated) there are also many cases where you want the opposite, and unfortunately there is no clear rule that always gets it right.

  • One, it's not possible to opt in just variables that need the feature, using EventTarget.captureFallbackContext also opts in this behaviour for all other variables even if they don't want or need this context

I wonder if this hints at a compelling use case for treating fallback contexts as a tree? The captureFallbackContext within the var1.run would do the right thing for its own variable, but shouldn't disturb any other variable that might care (or maybe it's still too coarse-grained, since I guess it would capture any surrounding incidental modifications to other variables).

  • Two, as a consequence of the previous point the lifetimes of variables can (and will) be accidentally extended

Is it clear that the new var1 value is captured incorrectly here? The logger framework is presumably ignorant of it, so the only situation in which it would matter is any listeners passed in from outside, and given that they're registered inside the var1.run, it doesn't seem surprising that it would be captured (though it's admittedly odd that it would change based on the logger.file call).

  • Three, there's still no way to capture registration time for synchronous dispatched callbacks, i.e. if a click happens because a webcomponent or such calls element.click() then EventTarget.captureFallbackContext still can't repair the context as it's not used

From my experience, I would prefer to get the dispatch context here. As I mentioned above, there's simply no "always right" answer. If a particular framework knows it's more appropriate to use registration context, then it may be possible to have it do the wrapping before it registers the event.

These drawbacks aren't surprising, they come from the fact that Event.captureFallbackContext is trying to simulate two different kinds of variable flow, but it fails to emulate either flow correctly for all variables.

I still think the best solution is what already has been suggested, have different types of variables for the different types of flow.

I'm honestly skeptical that this would ever be viable. If we just have two types of variables (i.e. registration vs dispatch) then we're still conflating a bunch of different things together, and will fail to address every use case - there's too many different types of context merge to distill it down to a single axis, e.g. I want something that preserves across await (i.e. current proposal) but goes with dispatch context for events (which would be more consistent with the ContinuationFlow proposal), so neither version would be 100% correct. Alternatively, if there's finer-grained control, then context ends up scaling as O(n) in the number of variables on every context change, and that's going to be a non-starter for implementers (who are very much gatekeepers here).

@Jamesernator
Copy link

Jamesernator commented Dec 5, 2024

This unfortunately cuts both ways. Registration context may be what you want for some variables and some events some of the time, but (as the "feedback from multiple parties" indicated) there are also many cases where you want the opposite, and unfortunately there is no clear rule that always gets it right.

This is precisely why I argue there should both kinds of variables.

"feedback from multiple parties"

There are use cases that extend beyond just those who have private contact with the proposal authors. And again having both types of variables allows individual use cases to use either or both depending on what they want to do.

If a particular framework knows it's more appropriate to use registration context, then it may be possible to have it do the wrapping before it registers the event.

This is not a framework, it is a logger, it knows nothing about where events might be registered or dispatched. This is generally true of all dependency injection style uses of AsyncContext.

I'm honestly skeptical that this would ever be viable.
Alternatively, if there's finer-grained control, then context ends up scaling as O(n) in the number of variables on every context change

This is simply not true, the example implementation I made to prove that is possible has the essentially the same characteristics for both in and out flows† (if you replace the Map with a (peristent) BTreeMap or similar).

† There is a special case with Promise.all, however this doesn't iterate over all variables, it simply captures all continuation contexts and sets that as a list (you get out exactly as many inner contexts as you pass to Promise.all). A tracer could for example check all such returns to see all flows through the program. From the other thread it does seem like a lot of use cases would do fine with just any of the continuation contexts so maybe the extra joining is superfluous.

(The implementation is not even particularly complicated, you just have one store for variables that flow inwards, and one for variables that flow outwards).


I would also like to make note of how having both flows can improve even the existing use cases.

For example in tracing we generally want to know why something happened, by having a variable that has both flows, in-flow and up-flow, we can accurately show both the root task that caused the evaluation AND know who is responsible for the registration:

declare namespace $AsyncContext {
    export class Variable<T> {
        getInFlow(): T | undefined;
        getOutFlow(): T | undefined;
        
        // Used for dependency injection use cases
        runWithInjectionFlow<R>(value: T, cb: () => R): R;
        // I feel most use cases are better that need returns probably want bothFlows
        // however some types of tracers may wish to ignore registration time in which
        // case they can use this instead 
        runWithContinuationFlow<R>(value: T, cb: () => R): R;
        // Useful for tracing and similar like below
        runWithBothFlows<R>(value: T, cb: () => R): R;
        
        // ALSO POSSIBLE, and probably useful:
        // sets the continuation synchronously which then bubbles up the stack, useful for
        // cases that would need variable.set() or similar
        setContinuation<T>(value: T): void;
        // clears the continuation, useful if you want a barrier where continuation flow
        // doesn't leak out
        deleteContinuation(): void
    }
}

export default class Tracer {
    // Probably more useful as a list, but this example doesn't need it
    readonly #tracingVariable = new $AsyncContext.Variable<string>();

    span<R>(name: string, start: () => R): R {
        return this.#tracingVariable.runWithBothFlows(name, start);
    }

    trace(): { triggeredBy: string | undefined; registeredBy: string | undefined } {
        const triggeredBy = this.#tracingVariable.getOutFlow();
        const registeredBy = this.#tracingVariable.getInFlow();
        return { triggeredBy, registeredBy };
    }
}

const tracer = new Tracer();

const el = document.getElementById("someElement");

tracer.span("my-game", () => {
    el?.addEventListener("click", () => {
        // Prints { triggeredBy: "user-interaction", registeredBy: "my-game" }
        console.log(tracer.trace());
    });
});

tracer.span("user-interaction", () => {
    el?.click();
});

Other use cases can use multiple flows in similar ways.

@nicolo-ribaudo
Copy link
Member

nicolo-ribaudo commented Dec 5, 2024

I'm confused by how we are talking about something generic, and not specific to .addEventListener.

If it's generic, it's clear to me that one of the two flows is the one that comes from when the function it's called, but it's not at all clear what should the other one be. Is it the one that a function captures lexically? Is it the one from the first time the function is passed as a parameter to another function? Is it the one of the last time it's passed as a parameter to another function?

The "out flow" and "in flow" you are describing seem very similar to call-time function parameters and bind-time function parameters:

let fn;

myVar.run("foo", () => {
  fn = function (inFlow, outFlow) {
    console.log({ inFlow, outFlow });
  }.bind(null, myVar.get());
});

myVar.run("bar", () => {
  fn(myVar.get());
});

logs { inFlow: "foo", outFlow: "bar" }. The "out flow" is equivalent to implicitly passed function parameters, while the "in flow" is equivalent to implicit .bind calls. The first one is obvious where it happens (when you call the function), while the second is not (it could be either when the function it's declared, or in any of the places the function is referenced).

Maybe this is similar to how function functions get their this from the caller, while => functions capture it regardless of how they are called?

@Jamesernator
Copy link

Jamesernator commented Dec 5, 2024

I'm confused by how we are talking about something generic, and not specific to .addEventListener.

The reason is because the proposed change to .addEventListener basically looks exactly like continuation flow, but this is the only place such a flow is used.

The "out flow" is equivalent to implicitly passed function parameters, while the "in flow" is equivalent to implicit .bind calls.

This is partially correct, a simple model is just to consider it in terms of SyncContext:

namespace SyncContext {
   let implicitParams = new Map<SyncContext.Variable, any>();
   let implicitReturn = new Map<SyncContext.Variable, any>();
   
   export class Variable<T> {
       getParam(): T | undefined {
           return implicitParams.get(this);
       }
       
       getReturn(): T | undefined {
           return implicitReturn.get(this);
       }
   
       // Basically the same as AsyncContext
       runWithImplicitParam<T>(value: T, cb: () => R): R {
           const previousImplicitParams = implicitParams;
           try {
               implicitParams = new Map([...implicitParams, [this, value]));
               return cb();
           } finally {
               implicitParams = previousImplicitParams;
           }
       }
       
       // This doesn't translate well to all asynchronous behaviours as callbacks
       // act as returns, that's why there's runWithContinuationFlow so that it flows
       // as a return value into the callback
       setImplicitReturn(value: T): void {
           implicitReturn = new Map<SyncContext.Variable, any>();
       }
       
       // Deletes the implicit return of this value so that it doesn't propogate up
       // this can be used as a barrier like in the later example with element.events
       // I show later
       deleteImplicitReturn(): void {
           implicitReturn = new Map<SyncContext.Variable, any>();
       }
   }
}

const var1 = new SyncContext.Variable<string>();
const var2 = new SyncContext.Variable<string>();

function foo() {
    const v1 = var1.get(); // one in the following
    var2.setImplicitReturn("two");
    return 10;
}

const r = var1.runWithImplicitParam("one", foo);
const v2 = var2.get();

If sync context didn't exist, it would just be like if we had passed params in explictly and returned explictly:

function foo(v1: string) {
    const v2 = "two";
    return [10, v2];
}

const [r, v2] = foo("one")

Now what makes asynchronous functions special, is that "returns" are done by passing parameters to callbacks (even for promises, despite async-await sugar). Like if foo were a callback based asynchronous function in the above example:

// v2 is "returned" from foo by way of the callback
function foo(v1: string, cb: (ret: number, v2: string) => void) {
    const v2 = "two";
    cb([10, v2]);
}

Then async-ifying implicit returns is equivalent to setting a return context before calling the callback, this is exactly the flow that being argued for in the other issue. The AsyncContext version of sync returns is a bit more complicated with the .runWithContinuation (a.k.a. .setReturnIn my example implementation, I don't know what naming scheme is preferable).

In the current design, by using registration time it's basically like as say called .bind with an implicit param on every single callback passed somewhere, however the proposed change for event listeners explicitly breaks this in that one specific case and makes it more like .dispatchEvent is passing an extra parameter to the callback, i.e. it's a return flow not a parameter flow.

To see this more clearly, suppose that asynchronous programming didn't exist in JavaScript and instead events were simply blocking synchronous iterators:

startThread {
    for (const clickEvent of element.events("click")) {
        // do something with clickEvent
    }
} 

If you wanted to return contextual information such as what interaction caused the event, then you obviously can't use implicit parameters, because the iterator returns the event to the for-loop. i.e. The example I gave above where I captured that "user-interaction" was the trigger for this would be like if internally element.events("click") set an implicit return that this event was triggered by a "user-interaction". The implicit parameters to this loop though are entirely unaffected. i.e.:

startThread {
    for (const clickEvent of element.events("click")) {
        // do something with clickEvent
        console.log(tracing.cause()); // either user-interaction or fake-event
    }
}

startThread {
    sleep(Math.random());
    // This would call .setImplicitReturn("fake-event"), call the callback, then
    // .deleteImplicitReturn() so that the value doesn't propagate out of this callback
    tracing.cause("fake-event", () => {
        // Whatever mechanism allows blocking in the other thread would restore the
        // the return context of whatever triggers the event, JS doesn't have threads
        // but resuming paused async functions is functionally very similar
        element.dispatchEvent(clickEvent);
    });
}

Now an argument you might have, is that in synchronous land we could still write element.events like this:

startThread {
    element.events("click", (evt) => {
        // var.get()
    });
}

where element.events sets some implicit parameters, and yes you would be correct, however element.events in this situation can only affect variables it controls, because it still in the same sync context as it's parent. e.g.:

startThread {
    const logFile = new SyncContext.Variable();
    logFile.run("./log.txt", () => {
        element.events("click", (evt) => {
            console.log(logFile.get()); // Still "log.txt"
            console.log(hostTracingForWhereThisEventCameFrom.get()); // user-interaction at some time, some place
        });
    });
}

This API is suspiciously similar to the asynchronous case (even though it technically synchronous), and so the same things apply, we could set an implicit return and send it into the event callback without affecting the implicit parameters.

Essentially my suggestion is that we implement these asynchronous return flows, this unlocks the use cases for ContinuationFlow and simultaneously allows .addEventListener to pass extra contextual information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants