Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OTEL_EXPORTER_OTLP_PROTOCOL env variable support #5743

Open
akvlad opened this issue Aug 14, 2023 · 8 comments · May be fixed by #6483
Open

Add OTEL_EXPORTER_OTLP_PROTOCOL env variable support #5743

akvlad opened this issue Aug 14, 2023 · 8 comments · May be fixed by #6483
Labels
feature ⚙️ New feature or request help wanted 🆘 Extra attention is needed

Comments

@akvlad
Copy link

akvlad commented Aug 14, 2023

In the opentelemetry documentation you have link to, there is a configuration of the protocol for traces to send: https://github.com/open-telemetry/opentelemetry-specification/blob/v1.7.0/specification/protocol/exporter.md#specify-protocol. It can be grpc or http/protobuf etc.

As far as I see in the code: https://github.com/caddyserver/caddy/blob/master/modules/caddyhttp/tracing/tracer.go#L61 you have only grpc supported.

It would be great to have other options like http/protobuf supported as well. Some tracing ingesters have the no GRPC support by different reasons.

@francislavoie
Copy link
Member

PRs welcome. Most maintainers don't use tracing or metrics ourselves, so we need to lean on the community to contribute improvements.

@francislavoie francislavoie added feature ⚙️ New feature or request help wanted 🆘 Extra attention is needed labels Aug 14, 2023
@trajano
Copy link

trajano commented Jul 28, 2024

I think this may be the reason Caddy doesn't work with grafana since they only expose the HTTPS port

@trajano trajano linked a pull request Jul 28, 2024 that will close this issue
@hairyhenderson
Copy link
Collaborator

I think this may be the reason Caddy doesn't work with grafana

Can you elaborate? In what way does Caddy "not work with grafana"?

@trajano
Copy link

trajano commented Jul 28, 2024

When you create a free stack at grafana.com and try to set up their OTEL collector it recommends the following environment variables

export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp-gateway-prod-us-east-0.grafana.net/otlp"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic SECRET"

I also added

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="https://otlp-gateway-prod-us-east-0.grafana.net/otlp"

So problems with Caddy are

  1. http/protobuf is the protocol required but as per the OP of this issue, it does not work because it's hard coded to GRPC tracer only. I tried to work around it with a change in my PR
  2. The change unfortunately didn't work as somewhere else in the caddy codebase or one of it's imports it looks at the OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_TRACES_ENDPOINT it parses the value and determines there's no port value (since grafana uses https which has a default port of 443 it doesn't place it)
  3. Adding the port i.e. https://otlp-gateway-prod-us-east-0.grafana.net:443/otlp will make Caddy parse out the port as 443/otlp then there'd be an error saying port 443/otlp is not valid. I am unable to ascertain where that is in the code base

@trajano
Copy link

trajano commented Jul 28, 2024

Maybe it's the way I am testing

RUN git clone https://github.com/trajano/caddy /mnt/
RUN xcaddy build \
    --replace github.com/caddyserver/caddy/v2/modules/caddyhttp/tracing=/mnt/modules/caddyhttp/tracing

FROM caddy
COPY --from=builder /usr/bin/caddy /usr/bin/caddy

@trajano
Copy link

trajano commented Jul 28, 2024

Figured out what was wrong with the Dockerfile

Got a bit further but got

traces export: failed to send to https://otlp-gateway-prod-us-east-0.grafana.net/oltp/v1/traces: 404 Not Found

But posting directly using curl seems to work

@trajano
Copy link

trajano commented Jul 29, 2024

Finally got it working, the last bit was a typo might've accidentally transposed two characters in the otlp in https://otlp-gateway-prod-us-east-0.grafana.net/otlp I see data in Grafana now with my PR.

@trajano
Copy link

trajano commented Aug 27, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature ⚙️ New feature or request help wanted 🆘 Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants