Skip to content

Commit

Permalink
Miscellanous compliance fixes (#63)
Browse files Browse the repository at this point in the history
* Rename function_calls_count to function_calls

* Add unit to histogram duration

* Replace `caller` with `caller.(function|module)`

* Add `service.name` label

* Update CHANGELOG.md

* Fix tests

* Fix service name not being exported

* Update code with API changes outside of docker builds

* Tweak demo apps to fail faster with cleaner logs

* Update autometrics-shared rules
  • Loading branch information
gagbo authored Sep 4, 2023
1 parent dea8bbf commit 0a62cf8
Show file tree
Hide file tree
Showing 19 changed files with 196 additions and 121 deletions.
10 changes: 9 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,19 @@ versioning](https://go.dev/doc/modules/version-numbers).
- The Go generator now removes all the old defer statements in function bodies before re-adding
only the necessary ones. This means calling `go generate` on a file that has no annotation
at all effectively cleans up the whole file from autometrics.

- All metrics now have a `service_name` label, which can either be compiled in `Init` call, or
filled at runtime from environment variables (in order of precedence):
+ `AUTOMETRICS_SERVICE_NAME`
+ `OTEL_SERVICE_NAME`

### Changed

- Instead of returning an error when the go generator does not find the autometrics import
in a file, it will add the needed import itself in the file.
- Function calls metric has been renamed from `function_calls_count_total` to `function_calls_total`
- Function calls duration histogram has been renamed from `function_calls_duration`
to `function_calls_duration_seconds`
- Function caller label has been split from `caller` to `caller_function` and `caller_label`

### Fixed

Expand Down
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,14 +96,19 @@ And then in your main function initialize the metrics
autometrics.Init(
nil,
autometrics.DefBuckets,
autometrics.BuildInfo{Version: "0.4.0", Commit: "anySHA", Branch: ""},
autometrics.BuildInfo{Version: "0.4.0", Commit: "anySHA", Branch: "", Service: "myApp"},
)
```

Everything in `BuildInfo` is optional. It will add relevant information on the
metrics for better intelligence. You can use any string variable whose value is
injected at build time by `ldflags` for example, or use environment variables.

> **Note**
> Instead of hardcoding the service in the code, you can simply have environment variables set to fill the "Service" name.
`AUTOMETRICS_SERVICE_NAME` will be used if set, otherwise `OTEL_SERVICE_NAME` will be attempted (so OpenTelemetry
compatibility comes out of the box).

### 3. Add directives for each function you want to instrument

> **Warning**
Expand Down Expand Up @@ -235,7 +240,7 @@ func main() {
autometrics.Init(
nil,
autometrics.DefBuckets,
autometrics.BuildInfo{Version: "0.4.0", Commit: "anySHA", Branch: ""},
autometrics.BuildInfo{Version: "0.4.0", Commit: "anySHA", Branch: "", Service: "myApp"},
)
http.Handle("/metrics", promhttp.Handler())
}
Expand Down Expand Up @@ -345,7 +350,7 @@ metric. You can use the name of the application or its version for example
- nil,
+ "myApp/v2/prod",
autometrics.DefBuckets,
autometrics.BuildInfo{ Version: "2.1.37", Commit: "anySHA", Branch: "" },
autometrics.BuildInfo{ Version: "2.1.37", Commit: "anySHA", Branch: "", Service: "myApp" },
)
```

Expand Down
2 changes: 2 additions & 0 deletions docker-compose.open-telemetry-example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ services:
build:
context: .
dockerfile: examples/otel/Dockerfile
environment:
AUTOMETRICS_SERVICE_NAME: open-telemetry-autometrics-demo
container_name: web-server-otel
restart: unless-stopped
expose:
Expand Down
2 changes: 2 additions & 0 deletions docker-compose.prometheus-example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ services:
build:
context: .
dockerfile: examples/web/Dockerfile
environment:
AUTOMETRICS_SERVICE_NAME: prometheus-autometrics-demo
container_name: web-server-prom
restart: unless-stopped
expose:
Expand Down
41 changes: 22 additions & 19 deletions examples/otel/cmd/main.go

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions examples/otel/scripts/poll_server
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ TARGET_HOST="${TARGET_HOST:-localhost}"

while true
do
if [ "$(($RANDOM % 2))" == "0" ]; then
curl "http://${TARGET_HOST}:62086/random-error"
if [ "$(($RANDOM % 3))" == "0" ]; then
curl -s "http://${TARGET_HOST}:62086/random-error"
fi
if [ "$(($RANDOM % 4))" == "0" ]; then
curl "http://${TARGET_HOST}:62086/"
curl -s "http://${TARGET_HOST}:62086/"
fi
sleep 0.2
sleep 0.05
done
11 changes: 6 additions & 5 deletions examples/web/cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,21 +89,22 @@ func main() {
// [Request Rate Callee]: http://localhost:9090/graph?g0.expr=%23+Rate+of+function+calls+emanating+from+%60indexHandler%60+function+per+second%2C+averaged+over+5+minute+windows%0A%0Asum+by+%28function%2C+module%2C+version%2C+commit%29+%28rate%28function_calls_count%7Bcaller%3D%22main.indexHandler%22%7D%5B5m%5D%29+%2A+on+%28instance%2C+job%29+group_left%28version%2C+commit%29+last_over_time%28build_info%5B1s%5D%29%29&g0.tab=0
// [Error Ratio Callee]: http://localhost:9090/graph?g0.expr=%23+Percentage+of+function+emanating+from+%60indexHandler%60+function+that+return+errors%2C+averaged+over+5+minute+windows%0A%0A%28sum+by+%28function%2C+module%2C+version%2C+commit%29+%28rate%28function_calls_count%7Bcaller%3D%22main.indexHandler%22%2Cresult%3D%22error%22%7D%5B5m%5D%29+%2A+on+%28instance%2C+job%29+group_left%28version%2C+commit%29+last_over_time%28build_info%5B1s%5D%29%29%29+%2F+%28sum+by+%28function%2C+module%2C+version%2C+commit%29+%28rate%28function_calls_count%7Bcaller%3D%22main.indexHandler%22%7D%5B5m%5D%29+%2A+on+%28instance%2C+job%29+group_left%28version%2C+commit%29+last_over_time%28build_info%5B1s%5D%29%29%29&g0.tab=0
//
//autometrics:inst --slo "API" --latency-target 99 --latency-ms 100
//autometrics:inst --slo "API" --latency-target 99 --latency-ms 5
func indexHandler(w http.ResponseWriter, r *http.Request) error {
defer autometrics.Instrument(autometrics.PreInstrument(autometrics.NewContext(
r.Context(),
autometrics.WithConcurrentCalls(true),
autometrics.WithCallerName(true),
autometrics.WithSloName("API"),
autometrics.WithAlertLatency(100000000*time.Nanosecond, 99),
autometrics.WithAlertLatency(5000000*time.Nanosecond, 99),
)), nil) //autometrics:defer

time.Sleep(time.Duration(rand.Intn(200)) * time.Millisecond)
msSleep := rand.Intn(200)
time.Sleep(time.Duration(msSleep) * time.Millisecond)

fmt.Fprintf(w, "Hello, World!\n")
_, err := fmt.Fprintf(w, "Slept %v ms\n", msSleep)

return nil
return err
}

var handlerError = errors.New("failed to handle request")
Expand Down
9 changes: 5 additions & 4 deletions examples/web/cmd/main.go.orig
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,15 @@ func main() {
//
// It always succeeds and says hello.
//
//autometrics:inst --slo "API" --latency-target 99 --latency-ms 100
//autometrics:inst --slo "API" --latency-target 99 --latency-ms 5
func indexHandler(w http.ResponseWriter, r *http.Request) error {

time.Sleep(time.Duration(rand.Intn(200)) * time.Millisecond)
msSleep := rand.Intn(200)
time.Sleep(time.Duration(msSleep) * time.Millisecond)

fmt.Fprintf(w, "Hello, World!\n")
_, err := fmt.Fprintf(w, "Slept %v ms\n", msSleep)

return nil
return err
}

var handlerError = errors.New("failed to handle request")
Expand Down
8 changes: 4 additions & 4 deletions examples/web/scripts/poll_server
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ TARGET_HOST="${TARGET_HOST:-localhost}"

while true
do
if [ "$(($RANDOM % 2))" == "0" ]; then
curl "http://${TARGET_HOST}:62086/random-error"
if [ "$(($RANDOM % 3))" == "0" ]; then
curl -s "http://${TARGET_HOST}:62086/random-error"
fi
if [ "$(($RANDOM % 4))" == "0" ]; then
curl "http://${TARGET_HOST}:62086/"
curl -s "http://${TARGET_HOST}:62086/"
fi
sleep 0.2
sleep 0.05
done
16 changes: 10 additions & 6 deletions internal/autometrics/prometheus_link_gen.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,10 @@ func addBuildInfoLabels() string {
}

func requestRateQuery(counterName, labelKey, labelValue string) string {
return fmt.Sprintf("sum by (%s, %s, %s, %s) (rate(%s{%s=\"%s\"}[5m]) %s)",
return fmt.Sprintf("sum by (%s, %s, %s, %s, %s) (rate(%s{%s=\"%s\"}[5m]) %s)",
prometheus.FunctionLabel,
prometheus.ModuleLabel,
prometheus.ServiceNameLabel,
prometheus.VersionLabel,
prometheus.CommitLabel,
counterName,
Expand All @@ -68,9 +69,10 @@ func requestRateQuery(counterName, labelKey, labelValue string) string {
}

func errorRatioQuery(counterName, labelKey, labelValue string) string {
return fmt.Sprintf("(sum by (%s, %s, %s, %s) (rate(%s{%s=\"%s\",%s=\"error\"}[5m]) %s)) / (%s)",
return fmt.Sprintf("(sum by (%s, %s, %s, %s, %s) (rate(%s{%s=\"%s\",%s=\"error\"}[5m]) %s)) / (%s)",
prometheus.FunctionLabel,
prometheus.ModuleLabel,
prometheus.ServiceNameLabel,
prometheus.VersionLabel,
prometheus.CommitLabel,
counterName,
Expand All @@ -83,9 +85,10 @@ func errorRatioQuery(counterName, labelKey, labelValue string) string {
}

func latencyQuery(bucketName, labelKey, labelValue string) string {
latency := fmt.Sprintf("sum by (le, %s, %s, %s, %s) (rate(%s_bucket{%s=\"%s\"}[5m]) %s)",
latency := fmt.Sprintf("sum by (le, %s, %s, %s, %s, %s) (rate(%s_bucket{%s=\"%s\"}[5m]) %s)",
prometheus.FunctionLabel,
prometheus.ModuleLabel,
prometheus.ServiceNameLabel,
prometheus.VersionLabel,
prometheus.CommitLabel,
bucketName,
Expand All @@ -103,9 +106,10 @@ func latencyQuery(bucketName, labelKey, labelValue string) string {
}

func concurrentCallsQuery(gaugeName, labelKey, labelValue string) string {
return fmt.Sprintf("sum by (%s, %s, %s, %s) (%s{%s=\"%s\"} %s)",
return fmt.Sprintf("sum by (%s, %s, %s, %s, %s) (%s{%s=\"%s\"} %s)",
prometheus.FunctionLabel,
prometheus.ModuleLabel,
prometheus.ServiceNameLabel,
prometheus.VersionLabel,
prometheus.CommitLabel,
gaugeName,
Expand All @@ -119,11 +123,11 @@ func (p Prometheus) GenerateAutometricsComment(ctx GeneratorContext, funcName, m
requestRateUrl := p.makePrometheusUrl(
requestRateQuery(prometheus.FunctionCallsCountName, prometheus.FunctionLabel, funcName), fmt.Sprintf("Rate of calls to the `%s` function per second, averaged over 5 minute windows", funcName))
calleeRequestRateUrl := p.makePrometheusUrl(
requestRateQuery(prometheus.FunctionCallsCountName, prometheus.CallerLabel, fmt.Sprintf("%s.%s", moduleName, funcName)), fmt.Sprintf("Rate of function calls emanating from `%s` function per second, averaged over 5 minute windows", funcName))
requestRateQuery(prometheus.FunctionCallsCountName, prometheus.CallerFunctionLabel, funcName), fmt.Sprintf("Rate of function calls emanating from `%s` function per second, averaged over 5 minute windows", funcName))
errorRatioUrl := p.makePrometheusUrl(
errorRatioQuery(prometheus.FunctionCallsCountName, prometheus.FunctionLabel, funcName), fmt.Sprintf("Percentage of calls to the `%s` function that return errors, averaged over 5 minute windows", funcName))
calleeErrorRatioUrl := p.makePrometheusUrl(
errorRatioQuery(prometheus.FunctionCallsCountName, prometheus.CallerLabel, fmt.Sprintf("%s.%s", moduleName, funcName)), fmt.Sprintf("Percentage of function emanating from `%s` function that return errors, averaged over 5 minute windows", funcName))
errorRatioQuery(prometheus.FunctionCallsCountName, prometheus.CallerFunctionLabel, funcName), fmt.Sprintf("Percentage of function emanating from `%s` function that return errors, averaged over 5 minute windows", funcName))
latencyUrl := p.makePrometheusUrl(
latencyQuery(prometheus.FunctionCallsDurationName, prometheus.FunctionLabel, funcName), fmt.Sprintf("95th and 99th percentile latencies (in seconds) for the `%s` function", funcName))
concurrentCallsUrl := p.makePrometheusUrl(
Expand Down
Loading

0 comments on commit 0a62cf8

Please sign in to comment.