Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Launching with AppHost adds OTEL_EXPORTER_OTLP_PROTOCOL=grpc environment variable which breaks exporting to Seq #6939

Open
1 task done
RonanBurkeGS opened this issue Dec 12, 2024 · 2 comments
Labels
area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication area-telemetry untriaged New issue has not been triaged

Comments

@RonanBurkeGS
Copy link

RonanBurkeGS commented Dec 12, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

When you launch a project with Aspire, Aspire adds lots of "OTEL_" environment variables. One of these is "OTEL_EXPORTER_OTLP_PROTOCOL": "grpc" which in my case broke the manual, non-Aspire'y logging I was doing. This was some existing logging code that broke when Aspire was added to the project and the project was launched using the AppHost project.

I am running Seq (https://docs.datalust.co/docs/getting-started) locally and I am using Serilog to export to it using .WriteTo.OpenTelemetry(x =>.... When launching with Aspire, my project logs nothing to Seq. When setting the API project as the start-up project, it works fine.

It was a lot of debugging, but I found the issue by comparing the env variables with and without running with AppHost. No exceptions are thrown so it was an extremely tough error to track down.

Expected Behavior

  • I don't really know what the expected behavior should be
  • Documentation of this edge case? I think I'd still have run into this problem and it would probably have been hard to find an answer
  • Aspire doesn't set this environment variable? I don't know what the consequences of this would be. Is it possible for Aspire to work well with both protocols?
  • Aspire's Program.cs allows you/requires you to set the protocol
  • Again, not really sure what should be the expected behavior. I guess it was just unexpected that running with AppHost would cause an existing project to break in this way. I don't want to use the Aspire Seq integration just yet and I don't think that should be a requirement personally.

Steps To Reproduce

  1. Locally run Seq
  2. Have this logging code in your API project's program.cs:
        builder.Host.UseSerilog((context, services, configuration) =>
        {
            configuration
                .Enrich.FromLogContext()
                .Enrich.WithSpan()
                .WriteTo.Console()
                .WriteTo.OpenTelemetry(x =>
                {
                    x.Endpoint = "http://localhost:5341/ingest/otlp/v1/logs";
                    x.Protocol = OtlpProtocol.HttpProtobuf;
                    x.Headers = new Dictionary<string, string>
                    {
                        ["X-Seq-ApiKey"] = "MYAPIKEY"
                    };
                    x.ResourceAttributes = new Dictionary<string, object>
                    {
                        ["service.name"] = "ProfileService.Api"
                    };
                }, true);
        });
  1. Launch your app with Aspire AppHost

  2. Observe that in Seq, no events are being logged to its dashboard
    Image

  3. In your AppHost's environment variables, add "OTEL_EXPORTER_OTLP_PROTOCOL": "http/protobuf" environment variable.

  4. Repeat the above steps and observe that logging is now working again

Exceptions (if any)

No response

.NET Version info

PS C:\Users\RonanBurke> dotnet --info
.NET SDK:
Version: 8.0.403
Commit: c64aa40a71
Workload version: 8.0.400-manifests.e0880c5d
MSBuild version: 17.11.9+a69bbaaf5

Runtime Environment:
OS Name: Windows
OS Version: 10.0.26100
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\8.0.403\

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
[android]
Installation Source: SDK 8.0.400, VS 17.11.35327.3
Manifest Version: 34.0.143/8.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.android\34.0.143\WorkloadManifest.json
Install Type: FileBased

[aspire]
Installation Source: SDK 8.0.400, VS 17.11.35327.3
Manifest Version: 8.2.2/8.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.2.2\WorkloadManifest.json
Install Type: FileBased

[ios]
Installation Source: SDK 8.0.400, VS 17.11.35327.3
Manifest Version: 18.0.8303/8.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.ios\18.0.8303\WorkloadManifest.json
Install Type: FileBased

[maccatalyst]
Installation Source: SDK 8.0.400, VS 17.11.35327.3
Manifest Version: 18.0.8303/8.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.maccatalyst\18.0.8303\WorkloadManifest.json
Install Type: FileBased

[maui-windows]
Installation Source: SDK 8.0.400, VS 17.11.35327.3
Manifest Version: 8.0.82/8.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.maui\8.0.82\WorkloadManifest.json
Install Type: FileBased

[wasm-tools]
Installation Source: VS 17.11.35327.3
Manifest Version: 8.0.10/8.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.workload.mono.toolchain.current\8.0.10\WorkloadManifest.json
Install Type: FileBased

Host:
Version: 8.0.10
Architecture: x64
Commit: 81cabf2857

.NET SDKs installed:
8.0.403 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.35 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.35 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.35 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
Not found

Learn more:
https://aka.ms/dotnet/info

Download .NET:
https://aka.ms/dotnet/download

Anything else?

  • ASP.NET Core version: .NET 8
  • .NET Aspire version (e.g. package versions): 9.0.0
  • The IDE (VS / VS Code / Rider / Other) you're running on, and its version (and any extension versions): Visual Studio 2022 Version 17.11.5
@liammclennan
Copy link
Contributor

If your API project includes builder.AddServiceDefaults(); then that is configuring the OpenTelemetry SDK to send telemetry to the Aspire dashboard (see ConfigureOpenTelemetry in the ServiceDefaults project).

Using .WriteTo.OpenTelemetry is also configuring the OpenTelemetry SDK, in the same project and that is probably not going to work well.

I can see two solutions:

  1. Aspire already configures OpenTelemetry. You can modify the ConfigureOpenTelemetry function to add log and trace exporters that export to Seq. In this scenario you have one instance of the OpenTelemetry SDK per project, and they export to both the Aspire Dashboard and Seq. This is what the Aspire Seq component does for you, but you can also do it yourself.

  2. Change the serilog configuration in the API project to use the Seq sink instead of Serilog.Sinks.OpenTelemetry.

If you replace Serilog.Enrichers.Span with serilog-tracing then you will enable tracing features in the Seq UI.

Image

@RonanBurkeGS
Copy link
Author

I initially thought .AddServiceDefaults(); was the root cause of the problem, but commenting it out or commenting out the OpenTelemetry configuration in the service defaults project did not fix the issue. In fact, the API project works fine with these lines enabled! (my intention is to have one place to configure the logging for everything so I don't intend to keep the service defaults the way they are 😊)

The issue only happens when you launch the AppHost project, it is what adds the environment variables that cause the problem.

@joperezr joperezr added untriaged New issue has not been triaged area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication area-telemetry labels Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication area-telemetry untriaged New issue has not been triaged
Projects
None yet
Development

No branches or pull requests

3 participants