aws-sdk instrumentation: can direct lambda calls propagate context? #10161
-
Context: One lambda (A) calling another (B) directly via the aws sdk; both are instrumented with
Finding 0: Our instrumentation produces a server span from A, a client span for A calling B, Desired outcome: The configured propagators (e.g., the default Finding 1: Finding 2: When I change that code to inject using all configured propagators, and Finding 3: Inbound instrumentation on B fails to see this context (still get two separate traces). Finding 4: Inbound lambda instrumentation attempts context extraction only if the input object is Finding 5: Based on further debug-printing, nothing in the instrumentable surface of the aws lamba I hesitate to raise this as an Issue since I don't know if it's actually possible with the Question: who can I talk to who has expertise in this to either (a) figure |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 4 replies
-
@rapphil could you help with this |
Beta Was this translation helpful? Give feedback.
-
@johnbley Any chance you could make it to one of the Lambda SIG meetings? (We have one today.) I don't have experience with direct lambda invocation like this and would like to get a better understanding. |
Beta Was this translation helpful? Give feedback.
-
Yes, though I will be a few minutes late. |
Beta Was this translation helpful? Give feedback.
-
We had a discussion at the FaaS SIG meeting where it was suggested that my intuition was right - that the direct call to another lambda through the aws-sdk was being treated as "not through http", and otel propagation headers were being stripped off (but xray was still allowed to work correctly). Based on a discussion there, I tried tunneling the otlp context through xray, which involved modifying the inbound lambda instrumentation to use a modified copy of the It seems that our options are:
Thoughts? |
Beta Was this translation helpful? Give feedback.
-
For cross-referencing purposes, see #11675 |
Beta Was this translation helpful? Give feedback.
For aws-sdk invoke (non-async) API there is actually another option to propagate the trace context by injecting into LambdaContext.ClientContext. the js aws-sdk instrumentation has this already implemented, see here.