From 8e64a01f4a16250f5ee9a5547d014bb2c8723ea6 Mon Sep 17 00:00:00 2001 From: Jamil Date: Sun, 26 Jan 2025 18:57:07 -0800 Subject: [PATCH] chore(infra): Disable debug log for otel (#7874) In the relay's `cloud-init.yaml`, we've overridden the `telemetry` service log filter to be `debug`. This results in this log being printed to Cloud Logging every 1s, for _every_ relay: ``` 2025-01-26T23:00:35.066Z debug memorylimiter/memorylimiter.go:200 Currently used memory. {"kind": "processor", "name": "memory_limiter", "pipeline": "logs", "cur_mem_mib": 31} ``` These logs are consuming over half of our total log count, which accounts for over half our Cloud Monitoring cost -- the second highest cost in our GCP account. This PR removes the override so that the relay app has the same `otel-collector` log level as the Elixir, the default (presumably `info`). --- .../modules/google-cloud/apps/relay/templates/cloud-init.yaml | 3 --- 1 file changed, 3 deletions(-) diff --git a/terraform/modules/google-cloud/apps/relay/templates/cloud-init.yaml b/terraform/modules/google-cloud/apps/relay/templates/cloud-init.yaml index 1373de5c4..c309cc452 100644 --- a/terraform/modules/google-cloud/apps/relay/templates/cloud-init.yaml +++ b/terraform/modules/google-cloud/apps/relay/templates/cloud-init.yaml @@ -56,9 +56,6 @@ write_files: - set(attributes["exported_instrumentation_version"], attributes["instrumentation_version"]) - delete_key(attributes, "instrumentation_version") service: - telemetry: - logs: - level: "debug" pipelines: traces: receivers: [otlp]