OpenTelemetry signal reference

This reference catalogs all OpenTelemetry metrics and trace spans emitted by Posit Connect. Use this alongside the operational guide and health and performance guide to understand what signals are available for monitoring and troubleshooting.

Metrics

Application metrics

Metric	Type	Description	Key Dimensions
`application.count`	Gauge	Running applications (deduplicated across workers)	`application.type`
`associations.count`	Gauge	Content items associated with integrations	`integration.template`, `integration.auth_type`
`bundles.uploaded`	Gauge	Uploaded bundles over time windows	`window` (24h/7d/30d/1y/all)
`content.count`	Gauge	Total number of content items	`content.type`, `runtime.language`, `runtime.version`, `access.type`
`integrations.count`	Gauge	integrations	`integration.template`, `integration.auth_type`
`process.count`	UpDownCounter	Running processes	`process.tag`
`schedule.count`	Gauge	Schedules by status	`schedule.type`, `schedule.status` (queued/running/success/failure)
`session.ping.duration`	Histogram (ms)	Session ping request duration	—
`users.active`	Gauge	Distinct active users over time windows	`window` (24h/7d/30d/1y), `role` (administrator/publisher/viewer)

Job and queue metrics

Metric	Type	Description	Key Dimensions
`job.cancelled`	Counter	Jobs that were cancelled	`job.cancel.reason`
`job.completion`	Counter	Completed jobs by status	`content.guid`, `job.status` (success/failure), `job.exit_code`
`job.duration`	Histogram (s)	Job execution duration. Buckets: [1, 5, 15, 30, 60, 300, 900, 1800, 3600, 10800, 21600, 43200, 86400] seconds	`content.guid`, `job.status`, `job.exit_code`
`job.timeout`	Counter	Jobs that timed out	`content.guid`
`queue.items.active_duration`	Gauge (s)	Duration of oldest in-flight job	`queue.name`, `job.hostname`
`queue.items.age`	Gauge (s)	Age of oldest item in queue	`queue.name`
`queue.items.completed`	Counter	Queue items completed	`job.status`, `queue.name`
`queue.items.size`	Gauge	Items in queue	`queue.name`

Database metrics

Connection pool metrics

Metric	Type	Description
`db.sql.connection.closed_max_idle`	Counter	Connections closed due to SetMaxIdleConns
`db.sql.connection.closed_max_idle_time`	Counter	Connections closed due to SetConnMaxIdleTime
`db.sql.connection.closed_max_lifetime`	Counter	Connections closed due to SetConnMaxLifetime
`db.sql.connection.max_open`	Gauge	Maximum open connections to database
`db.sql.connection.open`	Gauge	Current open connections (idle and in-use)
`db.sql.connection.wait`	Counter	Total connections waited for
`db.sql.connection.wait_duration`	Counter (ms)	Total time blocked waiting for connections

Key dimensions: pool.name, db.system, status (idle/inuse for db.sql.connection.open)

Query latency metrics

Metric	Type	Description
`db.sql.latency`	Histogram (ms)	Database operation latency

Key dimensions: method, status

HTTP metrics

Metric	Type	Description	Key Dimensions
`http.server.request.body.size`	Histogram (bytes)	HTTP request body size	`http.request.method`, `http.response.status_code`
`http.server.request.duration`	Histogram (s)	HTTP request duration	`http.request.method`, `http.response.status_code`
`http.server.response.body.size`	Histogram (bytes)	HTTP response body size	`http.request.method`, `http.response.status_code`

Launcher metrics (off-host execution)

Metric	Type	Description	Key Dimensions
`launcher.client.request.duration`	Histogram (s)	Launcher HTTP request duration including retries	`http.request.method`, `http.response.status_code`
`launcher.client.requests`	Counter	HTTP requests to Launcher including retries	`http.request.method`, `http.response.status_code`
`launcher.client.retries`	Counter	Retry attempts for Launcher requests	`http.request.method`
`launcher.client.retry.exhausted`	Counter	Requests that exhausted all retries	`http.request.method`

Worker pool metrics

Metric	Type	Description	Key Dimensions
`worker.pool.busy`	Gauge	Workers currently processing jobs	`application.type`
`worker.pool.size`	Gauge	Total workers available	`application.type`
`worker.pool.utilization`	Gauge	Worker utilization ratio (0-1)	`application.type`

License metrics

Metric	Type	Description	Key Dimensions
`license.expiration.days_remaining`	Gauge (days)	Days until license expiration (negative when expired)	—
`license.shiny_users.current`	Gauge	Concurrent Shiny users	—
`license.shiny_users.limit`	Gauge	Licensed limit on concurrent Shiny users	—
`license.users.current`	Gauge	Named users counting against license limit	—
`license.users.limit`	Gauge	Licensed limit on named users	—

Request rejection metrics

Metric	Type	Description	Key Dimensions
`requests.rejected`	Counter	Requests rejected due to capacity or other limits	`rejection.reason` (capacity/license/auth), `rejection.type` (shiny/api/content type)

MCP server metrics

Connect records this metric for requests to its built-in Model Context Protocol (MCP) server at /mcp. The OpenTelemetry.MCPInstrumentation setting does not turn off this metric.

Metric	Type	Description	Key Dimensions
`mcp.server.operation.duration`	Histogram (s)	Duration of requests to the Connect MCP server at `/mcp`	`mcp.method.name`, `gen_ai.operation.name`, `gen_ai.tool.name`, `rpc.response.status_code`, `error.type`

Traces

Connect exports high-level lifecycle traces to your APM system, providing visibility into Connect operations. These traces show what Connect is doing without exposing internal implementation details.

See the operational guide for details on trace visibility.

HTTP and API spans

Span Name	Description	Key Attributes
`api.service.request.handle`	Service request handling	`service.name`
`api.variant.render`	Variant render request	`content.guid`, `content.variant.key`, `execution.reason`, `execution.should_activate`, `execution.should_send_email`
`HTTP {method} {route}`	Automatic HTTP request tracing	`http.method`, `http.route`, `http.status_code`, `url.path`

Schedule execution spans

Span Name	Description	Key Attributes
`schedule.render`	Root span for schedule execution	`schedule.id`, `schedule.next_time`, `schedule.planned_run`, `schedule.delay_diff`, `schedule.is_stale`, `content.id`, `content.guid`

Queue operations spans

Span Name	Description	Key Attributes
`queue.item.enqueue`	Adding item to queue	`queue.name`, `queue.item.type`
`queue.item.process`	Queue consumer processing work item	`queue.item.id`, `queue.name`, `queue.item.type`, `queue.wait_time.ms`, `queue.processing_duration.ms`, `heartbeat.success_count`, `heartbeat.failure_count`

Worker lifecycle spans

Span Name	Description	Key Attributes
`worker.connection.await`	Waiting for worker to accept connections	`worker.id`, `worker.host`, `worker.port`, `worker.init_timeout.ms`, `worker.connection_duration.ms`
`worker.lifecycle`	Full worker lifecycle from creation to termination	`worker.id`, `content.guid`
`worker.process.startup`	Worker process startup	`runtime.type` (python, shiny, plumber_api, tensorflow, container), `worker.id`, `content.guid`, `job.key`
`worker.provision`	Worker provisioning and creation	`worker.id`

Report execution spans

Span Name	Description	Key Attributes
`report.execute`	Report execution	`report.type` (quarto, jupyter, rmarkdown, rmarkdown_site), `content.id`, `content.guid`, `content.bundle.id`, `execution.reason`
`report.process.start`	Report process startup (separates startup from execution)	`content.guid`, `job.key`, `runtime.type`
`report.setup`	Report setup and preparation	`report.type`, `content.id`, `content.guid`

Launcher spans (off-host execution with Launcher)

Span Name	Description	Key Attributes
`launcher.job.start`	Launcher job setup and submission (Launcher)	`content.id`, `content.guid`, `content.variant.id`, `content.bundle.id`, `job.key`, `job.tag`, `job.cluster`

Kubernetes spans (off-host execution)

Span Name	Description	Key Attributes
`kubernetes.job.start`	Kubernetes job setup and submission	`content.id`, `content.guid`, `content.variant.id`, `content.bundle.id`, `job.key`, `job.tag`, `job.cluster`, `job.queue.name`, `job.queue.time`, `kubernetes.job.name`

Deployment spans

Span Name	Description	Key Attributes
`deploy.content.launch`	Content deployment and launch	`content.id`, `content.guid`, `content.bundle.id`

Email spans

Span Name	Description	Key Attributes
`email.send`	Email send operation	`email.trigger`, `content.guid`

MCP server spans

When OpenTelemetry.Enabled and OpenTelemetry.MCPInstrumentation are both true, Connect records one server span for each request to the Connect MCP server at /mcp. These spans follow the OpenTelemetry semantic conventions for MCP, with these exceptions:

Spans do not record jsonrpc.request.id, mcp.session.id, client.address, or client.port. The HTTP request span records the client address.
By default, the status of a failed span has no error message, because the message can contain client input.

Connect names a tool call span tools/call {tool} and uses the MCP method as the name of every other span.

When the MCP client sends its trace context in the params._meta field of a request, the client span is the parent of the Connect span. The HTTP request span is then a link. Without client trace context, the HTTP request span is the parent.

A span for a failed request has an error status and an error.type of tool_error, a JSON Remote Procedure Call (JSON-RPC) error code, or _OTHER. A JSON-RPC error that the client caused, such as an unknown tool or method, is not a failure. Connect records its code in rpc.response.status_code and leaves the span status unset.

Spans do not record successful tool results. A tool call span has content.guid and content.bundle.id only when the tool acts on that content or bundle.

Span Name	Description	Key Attributes
`tools/call {tool}`	Connect MCP tool call	`mcp.method.name`, `mcp.protocol.version`, `gen_ai.operation.name`, `gen_ai.tool.name`, `gen_ai.tool.type`, `content.guid`, `content.bundle.id`, `rpc.response.status_code`, `error.type`
`{mcp.method.name}`	Other Connect MCP requests, such as `tools/list`	`mcp.method.name`, `mcp.protocol.version`, `rpc.response.status_code`, `error.type`

Common span attributes

These attributes appear across multiple span types:

Attribute	Description
`content.bundle.id`	Bundle ID for content version
`content.guid`	GUID of content item
`content.owner.guid`	GUID of content owner
`content.id`	Database ID of content item
`content.name`	Short name of content
`content.type` / `content.mode`	Content type (shiny, python_app, rmd, etc.)
`content.variant.id`	Variant ID for content configuration
`content.variant.key`	Human-readable variant key
`email.trigger`	What triggered the email (e.g., report completion)
`error.type`	Error type when operation fails
`execution.reason`	Why content is executing (scheduled/manual/api)
`heartbeat.failure_count`	Failed heartbeat count
`heartbeat.success_count`	Successful heartbeat count
`job.id`	Database ID of job record
`job.key`	Unique job tracking key
`job.queue.name`	Name of the queue the job was placed in
`job.queue.time`	Time when the job was queued (RFC 3339)
`job.tag`	Job type tag
`kubernetes.job.exit_code`	Exit code when a direct Kubernetes job completes
`kubernetes.job.name`	Name of the Kubernetes Job resource
`kubernetes.pod.phase`	Kubernetes pod phase reported on job completion
`launcher.job.exit_code`	Exit code when job completes
`launcher.job.id`	Launcher’s job identifier
`launcher.job.status`	Job status from launcher
`process.pid`	Operating system process ID
`queue.item.id`	Queue item database ID
`queue.item.type`	Type of queued work
`queue.name`	Queue identifier
`queue.processing_duration.ms`	Time spent processing queue item
`queue.wait_time.ms`	Time item waited in queue before processing
`rebuild.required`	Whether package rebuild was needed
`report.type`	Report format type: quarto, jupyter, rmarkdown, rmarkdown_site
`runtime.type`	Runtime type: python, shiny, plumber_api, tensorflow, container
`schedule.delay_diff`	Delay from planned time
`schedule.id`	Schedule database ID
`schedule.is_stale`	Whether schedule is stale
`schedule.next_time`	Next scheduled run time
`schedule.planned_run`	Originally planned run time
`worker.connection_duration.ms`	Time taken to connect to worker
`worker.host` / `worker.port`	Worker connection information
`user.email`	User email address (added by trace user enrichment)
`user.guid`	User GUID (added by trace user enrichment when resolved from `content.guid`)
`user.username`	Username (added by trace user enrichment)
`worker.id`	Unique worker identifier
`worker.init_timeout.ms`	Configured worker initialization timeout

Span events

These events may appear within spans to mark significant moments:

Event Name	Context
`exception.appLocked`	Content is locked
`exception.licenseInvalid`	License check failed
`heartbeat.had_failures`	Heartbeat failures occurred during processing
`noWorkFound`	No schedule ready to run
`process started`	Worker process started successfully
`rebuild.triggered`	Package rebuild was triggered
`status.queued`	Item successfully queued
`status.skipDuplicate`	Duplicate item not queued
`workFound`	Schedule found and queued

Resource attributes

All signals include these resource attributes:

Attribute	Description
`service.namespace`	From `OpenTelemetry.ServiceNamespace` (defaults to `posit-connect-ns`). Identifies the deployment or environment, and is shared across all of a deployment’s signals.
`connect.node.name`	From `Server.NodeName`. Identifies the individual replica within a deployment.
`host.name`	System hostname
`service.name`	Always “posit-connect”

To distinguish signals from multiple Connect deployments, group by service.namespace; use connect.node.name to identify a replica within a deployment. See running multiple instances on a shared cluster.