Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Send Data Alloy page + Interactive Examples #13367

Open
wants to merge 40 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
34c0640
draft: killercoda enabled alloy doc
Jayclifford345 Jun 21, 2024
2d8ad39
Updated steps
Jayclifford345 Jun 21, 2024
ed0f540
Updated steps
Jayclifford345 Jun 21, 2024
678176d
updated url order
Jayclifford345 Jun 21, 2024
4f009de
fixed dock compose notation
Jayclifford345 Jun 21, 2024
7ee6176
examples
Jayclifford345 Jun 27, 2024
8a1015c
Added Kafka example
Jayclifford345 Jul 1, 2024
8659ea7
Added media
Jayclifford345 Jul 1, 2024
2d30450
Updated links
Jayclifford345 Jul 1, 2024
34b059a
Updated withj new {{< docs/ignore >}} tag
Jayclifford345 Jul 1, 2024
ddf65c8
fixed docs tag and added missing image
Jayclifford345 Jul 1, 2024
c292b5a
Updated include tags and fixed headings
Jayclifford345 Jul 3, 2024
6978cea
Added reason for Kafka label
Jayclifford345 Jul 3, 2024
f8f5eaa
Updated sandbox tip
Jayclifford345 Jul 4, 2024
a36bbc0
Updated sandbox link
Jayclifford345 Jul 4, 2024
6972db1
Merge branch 'main' into docs-send-data-alloy
Jayclifford345 Jul 18, 2024
0b39615
Update docs/sources/send-data/alloy/_index.md
Jayclifford345 Jul 22, 2024
cdfef22
Update docs/sources/send-data/alloy/_index.md
Jayclifford345 Jul 22, 2024
8a88838
Added suggested fixes
Jayclifford345 Jul 22, 2024
50c0435
fixed editor location
Jayclifford345 Jul 22, 2024
6bd94b8
Merge branch 'main' into docs-send-data-alloy
Jayclifford345 Jul 22, 2024
2be81dc
Update docs/sources/send-data/alloy/examples/alloy-kafka-logs.md
Jayclifford345 Jul 24, 2024
8f574a8
Update docs/sources/send-data/alloy/examples/alloy-kafka-logs.md
Jayclifford345 Jul 24, 2024
7b62e81
Update docs/sources/send-data/alloy/examples/alloy-kafka-logs.md
Jayclifford345 Jul 24, 2024
dcc392c
Update docs/sources/send-data/alloy/examples/alloy-kafka-logs.md
Jayclifford345 Jul 24, 2024
7019cc8
Update docs/sources/send-data/alloy/examples/alloy-kafka-logs.md
Jayclifford345 Jul 24, 2024
c4d2afa
Update docs/sources/send-data/alloy/examples/alloy-otel-logs.md
Jayclifford345 Jul 24, 2024
ee78506
fixed images and further corrections
Jayclifford345 Jul 24, 2024
12b032b
Merge branch 'docs-send-data-alloy' of https://github.com/grafana/lok…
Jayclifford345 Jul 24, 2024
103b916
Merge branch 'main' into docs-send-data-alloy
Jayclifford345 Jul 24, 2024
3aa3d1a
Update docs/sources/send-data/alloy/examples/alloy-kafka-logs.md
Jayclifford345 Jul 25, 2024
3252a64
Update docs/sources/send-data/alloy/examples/alloy-kafka-logs.md
Jayclifford345 Jul 25, 2024
31be3a4
Update docs/sources/send-data/alloy/examples/alloy-kafka-logs.md
Jayclifford345 Jul 25, 2024
e830c86
Update docs/sources/send-data/alloy/examples/alloy-otel-logs.md
Jayclifford345 Jul 25, 2024
af3b832
Update docs/sources/send-data/alloy/examples/alloy-otel-logs.md
Jayclifford345 Jul 25, 2024
2b90d10
Update docs/sources/send-data/alloy/examples/alloy-otel-logs.md
Jayclifford345 Jul 25, 2024
3ea3d84
Update docs/sources/send-data/alloy/examples/alloy-otel-logs.md
Jayclifford345 Jul 25, 2024
04ed456
Merge branch 'main' of https://github.com/grafana/loki into docs-send…
Jayclifford345 Jul 25, 2024
9fa414f
added bullet back in
Jayclifford345 Jul 25, 2024
2e71b0d
Removed line
Jayclifford345 Jul 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions docs/sources/send-data/alloy/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
title: Ingesting logs to Loki using Alloy
menuTitle: Grafana Alloy
description: Configuring Grafana Alloy to send logs to Loki.
weight: 250
---


# Ingesting logs to Loki using Alloy

Grafana Alloy is a versatile observability collector that can ingest logs in various formats and send them to Loki. We recommend Alloy as the primary method for sending logs to Loki, as it provides a more robust and feature-rich solution for building a highly scalable and reliable observability pipeline.

{{< figure src="/media/docs/alloy/flow-diagram-small-alloy.png" alt="Alloy flow diagram" >}}

## Installing Alloy

To get started with Grafana Alloy and send logs to Loki, you need to install and configure Alloy. You can follow the [Alloy documentation](https://grafana.com/docs/alloy/latest/get-started/install/) to install Alloy on your preferred platform.

## Components of Alloy for logs

Alloy pipelines are built using components that perform specific functions. For logs these can be broken down into three categories:

- **Collector:** These components collect/receive logs from various sources. This can be scraping logs from a file, receiving logs over HTTP, gRPC or ingesting logs from a message queue.
- **Transformer:** These components can be used to manipulate logs before they are sent to a writer. This can be used to add additional metadata, filter logs, or batch logs before sending them to a writer.
- **Writer:** These components send logs to the desired destination. Our documentation will focus on sending logs to Loki, but Alloy supports sending logs to various destinations.

### Log components in Alloy

Here is a non-exhaustive list of components that can be used to build a log pipeline in Alloy. For a complete list of components, refer to the [components list](https://grafana.com/docs/alloy/latest/reference/components/).

| Type | Component |
|------------|-----------------------------------------------------------------------------------------------------|
| Collector | [loki.source.api](https://grafana.com/docs/alloy/latest/reference/components/loki.source.api/) |
| Collector | [loki.source.awsfirehose](https://grafana.com/docs/alloy/latest/reference/components/loki.source.awsfirehose/) |
| Collector | [loki.source.azure_event_hubs](https://grafana.com/docs/alloy/latest/reference/components/loki.source.azure_event_hubs/) |
| Collector | [loki.source.cloudflare](https://grafana.com/docs/alloy/latest/reference/components/loki.source.cloudflare/) |
| Collector | [loki.source.docker](https://grafana.com/docs/alloy/latest/reference/components/loki.source.docker/) |
| Collector | [loki.source.file](https://grafana.com/docs/alloy/latest/reference/components/loki.source.file/) |
| Collector | [loki.source.gcplog](https://grafana.com/docs/alloy/latest/reference/components/loki.source.gcplog/) |
| Collector | [loki.source.gelf](https://grafana.com/docs/alloy/latest/reference/components/loki.source.gelf/) |
| Collector | [loki.source.heroku](https://grafana.com/docs/alloy/latest/reference/components/loki.source.heroku/) |
| Collector | [loki.source.journal](https://grafana.com/docs/alloy/latest/reference/components/loki.source.journal/) |
| Collector | [loki.source.kafka](https://grafana.com/docs/alloy/latest/reference/components/loki.source.kafka/) |
| Collector | [loki.source.kubernetes](https://grafana.com/docs/alloy/latest/reference/components/loki.source.kubernetes/) |
| Collector | [loki.source.kubernetes_events](https://grafana.com/docs/alloy/latest/reference/components/loki.source.kubernetes_events/) |
| Collector | [loki.source.podlogs](https://grafana.com/docs/alloy/latest/reference/components/loki.source.podlogs/) |
| Collector | [loki.source.syslog](https://grafana.com/docs/alloy/latest/reference/components/loki.source.syslog/) |
| Collector | [loki.source.windowsevent](https://grafana.com/docs/alloy/latest/reference/components/loki.source.windowsevent/) |
| Collector | [otelcol.receiver.loki](https://grafana.com/docs/alloy/latest/reference/components/otelcol.receiver.loki/) |
| Transformer| [loki.relabel](https://grafana.com/docs/alloy/latest/reference/components/loki.relabel/) |
| Transformer| [loki.process](https://grafana.com/docs/alloy/latest/reference/components/loki.process/) |
| Writer | [loki.write](https://grafana.com/docs/alloy/latest/reference/components/loki.write/) |
| Writer | [otelcol.exporter.loki](https://grafana.com/docs/alloy/latest/reference/components/otelcol.exporter.loki/) |
| Writer | [otelcol.exporter.logging](https://grafana.com/docs/alloy/latest/reference/components/otelcol.exporter.logging/) |


## Interactive Tutorials

To learn more about how to configure Alloy to send logs to Loki within different scenarios, follow these interactive tutorials:

- [Sending OpenTelemetry logs to Loki using Alloy]({{< relref "./examples/alloy-otel-logs" >}})
- [Sending logs over Kafka to Loki using Alloy]({{< relref "./examples/alloy-kafka-logs" >}})


Loading
Loading