From e7339418cb5ee6feae112feb25434aafad2eb543 Mon Sep 17 00:00:00 2001 From: Rachael Graham Date: Tue, 6 Oct 2026 16:59:00 -0500 Subject: [PATCH] Refresh observability docs for chart updates This commit switches observability examples to shared Agent Substrate Helm snippet and version files and adds chart-version support for both 0.x and 1.x. It also updates tracing documentation to explain the underscore-converted invoke_agent span names and the controller behavior for invalid telemetry settings: the affected signal is disabled with a warning instead of failing the upgrade. Signed-off-by: Rachael Graham --- .../assets/kagent-docs/snippets/helm-substrate.md | 1 + .../kagent-docs/versions/agent-substrate-chart.md | 1 + .../1.x/observability/lightweight-otel-stack.md | 8 ++++---- .../content/kagent/1.x/observability/metrics.md | 4 ++-- .../content/kagent/1.x/observability/otel-stack.md | 8 ++++---- .../content/kagent/1.x/observability/tracing.md | 12 +++++++----- 6 files changed, 19 insertions(+), 15 deletions(-) create mode 100644 docs-site/assets/kagent-docs/snippets/helm-substrate.md create mode 100644 docs-site/assets/kagent-docs/versions/agent-substrate-chart.md diff --git a/docs-site/assets/kagent-docs/snippets/helm-substrate.md b/docs-site/assets/kagent-docs/snippets/helm-substrate.md new file mode 100644 index 00000000..d1825bbb --- /dev/null +++ b/docs-site/assets/kagent-docs/snippets/helm-substrate.md @@ -0,0 +1 @@ +oci://ghcr.io/kagent-dev/substrate/helm/substrate \ No newline at end of file diff --git a/docs-site/assets/kagent-docs/versions/agent-substrate-chart.md b/docs-site/assets/kagent-docs/versions/agent-substrate-chart.md new file mode 100644 index 00000000..3b705c38 --- /dev/null +++ b/docs-site/assets/kagent-docs/versions/agent-substrate-chart.md @@ -0,0 +1 @@ +{{< version include-if="0.x" >}}0.0.9{{< /version >}}{{< version include-if="1.x" >}}0.4.0-alpha1{{< /version >}} \ No newline at end of file diff --git a/docs-site/content/kagent/1.x/observability/lightweight-otel-stack.md b/docs-site/content/kagent/1.x/observability/lightweight-otel-stack.md index a6a3bbfb..10aa2b9c 100644 --- a/docs-site/content/kagent/1.x/observability/lightweight-otel-stack.md +++ b/docs-site/content/kagent/1.x/observability/lightweight-otel-stack.md @@ -248,8 +248,8 @@ Upgrade the Agent Substrate Helm release. A single `otel.endpoint` setting turns ```bash helm upgrade substrate \ - oci://ghcr.io/kagent-dev/substrate/helm/substrate \ - --version {{< reuse "kagent-docs/versions/agent-substrate.md" >}} \ + {{< reuse "kagent-docs/snippets/helm-substrate.md" >}} \ + --version {{< reuse "kagent-docs/versions/agent-substrate-chart.md" >}} \ --namespace ate-system \ --reuse-values \ --set otel.endpoint=http://otel-collector.telemetry.svc.cluster.local:4317 \ @@ -321,8 +321,8 @@ Check each signal in turn: traces in Jaeger, metrics in Prometheus, and logs in --set otel.logs.enabled=false \ --set controller.metrics.enabled=false helm upgrade substrate \ - oci://ghcr.io/kagent-dev/substrate/helm/substrate \ - --version {{< reuse "kagent-docs/versions/agent-substrate.md" >}} \ + {{< reuse "kagent-docs/snippets/helm-substrate.md" >}} \ + --version {{< reuse "kagent-docs/versions/agent-substrate-chart.md" >}} \ --namespace ate-system --reuse-values \ --set otel.endpoint="" \ --set otel.traces.samplingRatio=0.01 diff --git a/docs-site/content/kagent/1.x/observability/metrics.md b/docs-site/content/kagent/1.x/observability/metrics.md index f99ba2bd..7f8733ee 100644 --- a/docs-site/content/kagent/1.x/observability/metrics.md +++ b/docs-site/content/kagent/1.x/observability/metrics.md @@ -51,8 +51,8 @@ Agent Substrate exports metrics over OTLP when its Helm release has an OTLP endp ```bash helm upgrade substrate \ - oci://ghcr.io/kagent-dev/substrate/helm/substrate \ - --version {{< reuse "kagent-docs/versions/agent-substrate.md" >}} \ + {{< reuse "kagent-docs/snippets/helm-substrate.md" >}} \ + --version {{< reuse "kagent-docs/versions/agent-substrate-chart.md" >}} \ --namespace ate-system \ --reuse-values \ --set otel.endpoint=http://otel-collector.telemetry.svc.cluster.local:4317 diff --git a/docs-site/content/kagent/1.x/observability/otel-stack.md b/docs-site/content/kagent/1.x/observability/otel-stack.md index e00f3a13..3a10ba86 100644 --- a/docs-site/content/kagent/1.x/observability/otel-stack.md +++ b/docs-site/content/kagent/1.x/observability/otel-stack.md @@ -392,8 +392,8 @@ Point Agent Substrate at the collector. A single `otel.endpoint` setting turns o 1. Upgrade the Agent Substrate Helm release. ```bash helm upgrade substrate \ - oci://ghcr.io/kagent-dev/substrate/helm/substrate \ - --version {{< reuse "kagent-docs/versions/agent-substrate.md" >}} \ + {{< reuse "kagent-docs/snippets/helm-substrate.md" >}} \ + --version {{< reuse "kagent-docs/versions/agent-substrate-chart.md" >}} \ --namespace ate-system \ --reuse-values \ --set otel.endpoint=http://otel-collector.telemetry.svc.cluster.local:4317 \ @@ -500,8 +500,8 @@ Log in to Grafana, and query each backend from the **Explore** view. --set controller.metrics.enabled=false \ --set controller.metrics.serviceMonitor.enabled=false helm upgrade substrate \ - oci://ghcr.io/kagent-dev/substrate/helm/substrate \ - --version {{< reuse "kagent-docs/versions/agent-substrate.md" >}} \ + {{< reuse "kagent-docs/snippets/helm-substrate.md" >}} \ + --version {{< reuse "kagent-docs/versions/agent-substrate-chart.md" >}} \ --namespace ate-system --reuse-values \ --set otel.endpoint="" \ --set otel.traces.samplingRatio=0.01 diff --git a/docs-site/content/kagent/1.x/observability/tracing.md b/docs-site/content/kagent/1.x/observability/tracing.md index f6739eda..b53d805b 100644 --- a/docs-site/content/kagent/1.x/observability/tracing.md +++ b/docs-site/content/kagent/1.x/observability/tracing.md @@ -56,13 +56,13 @@ Each hop reports itself as a separate OpenTelemetry (OTel) service. A tracing ba ### Spans -The `kagent` runtime creates the same spans for every agent, and most span names describe the operation rather than the agent. The `invoke_agent` span is the exception, because its name carries the service name of the agent that ran. To narrow a search to one agent, filter by service name rather than by span name. The following spans appear in nesting order, from the span that accepts the request down to the model and tool calls that serve it. +The `kagent` runtime creates the same spans for every agent, and most span names describe the operation rather than the agent. The `invoke_agent` span is the exception, because its name carries the name of the agent that ran. To narrow a search to one agent, filter by service name rather than by span name. The following spans appear in nesting order, from the span that accepts the request down to the model and tool calls that serve it. | Span | When it is created | | ---- | ------------------ | | `lf.a2a.v1.A2AService/SendMessage` | Once per request, as the root of the runtime's half of the trace. The runtime creates it when it accepts the A2A call from the controller. The controller reports spans of the same name for its own side of the call. | | `a2a.request` | Once per request. Records the A2A method and the final state of the task in the `a2a.method` and `a2a.task.state` attributes. | -| `invoke_agent ` | Once per request, named for the {{< gloss "Agent" >}}Agent{{< /gloss >}} that serves it, such as `invoke_agent my-first-agent`. The name matches the runtime's service name. | +| `invoke_agent ` | Once per request, named for the {{< gloss "Agent" >}}Agent{{< /gloss >}} that serves it, with each hyphen replaced by an underscore. The Agent `my-first-agent` produces `invoke_agent my_first_agent`, while its service name keeps the hyphens, so the two spellings differ. | | `generate_content ` | Once per model call, named for the model that was called. | | `execute_tool ` | Once per tool call, named for the tool that was called. Records the call's arguments and the tool's reply in the `gcp.vertex.agent.tool_call_args` and `gcp.vertex.agent.tool_response` attributes. | | `execute_tool (merged)` | Once per model turn that calls more than one tool, as the parent of that turn's `execute_tool` spans. A turn that calls a single tool creates no merged span. | @@ -133,12 +133,14 @@ Tracing is off by default. Turning it on is a Helm change, because the controlle | Field | Description | | ----- | ----------- | | `traces.enabled` | Whether to export traces at all. Defaults to `false`. | - | `exporter.otlp.endpoint` | The OTLP endpoint that every signal exports to, as an `http://` or `https://` URL. An `http://` endpoint sends plaintext. Empty by default. When a signal is enabled and neither this setting nor its per-signal counterpart holds an endpoint, the controller rejects the configuration with `OTLP traces endpoint is required when traces export is enabled`. | + | `exporter.otlp.endpoint` | The OTLP endpoint that every signal exports to, as an `http://` or `https://` URL. An `http://` endpoint sends plaintext. Empty by default. When a signal is enabled and neither this setting nor its per-signal counterpart holds an endpoint, the controller disables that signal and reports `OTLP traces endpoint is required when traces export is enabled`. | | `exporter.otlp.protocol` | `grpc` or `http/protobuf`. Defaults to `grpc`, which matches the port `4317` in the example endpoint. Point `http/protobuf` at port `4318` instead. | | `exporter.otlp.timeout` | The export timeout in milliseconds. Empty by default, which keeps the OTel SDK default. | | `traces.endpoint`, `traces.protocol` | Send traces somewhere other than the other signals. Each one overrides its `exporter.otlp` counterpart for traces alone. Both are empty by default. | - An endpoint is an absolute `http://` or `https://` URL, and the controller rejects one that carries credentials, a query, or a fragment. The two endpoint settings differ in how the controller treats the path. A per-signal endpoint such as `traces.endpoint` is used exactly as you write it. The shared `exporter.otlp.endpoint` is used as written on the `grpc` protocol, and gains a `/v1/traces` suffix on `http/protobuf`, so point the shared setting at the collector's root rather than at a signal path. + An endpoint is an absolute `http://` or `https://` URL, and one that carries credentials, a query, or a fragment leaves its signal disabled in the same way. An invalid telemetry setting never fails the upgrade or the Harness. The controller reports each one as a warning when it starts, as the log record `invalid agent telemetry configuration; disabling signal`, and turns off only the signal that the setting belongs to. Check that log after you change these values, because no other surface reports the problem and an agent with a disabled signal runs normally. + + The two endpoint settings differ in how the controller treats the path. A per-signal endpoint such as `traces.endpoint` is used exactly as you write it. The shared `exporter.otlp.endpoint` is used as written on the `grpc` protocol, and gains a `/v1/traces` suffix on `http/protobuf`, so point the shared setting at the collector's root rather than at a signal path. 4. Upgrade the kagent Helm release. ```bash @@ -219,7 +221,7 @@ Send a request to a new Session, then find its trace in the backend that you set POST agentgateway lf.a2a.v1.A2AService/SendMessage my-first-agent a2a.request my-first-agent - invoke_agent my-first-agent my-first-agent + invoke_agent my_first_agent my-first-agent generate_content gpt-4.1-mini my-first-agent HTTP POST my-first-agent ```