Prometheus Metrics

Workbench | Enhanced Advanced

Prometheus Metrics

Prometheus is a third-party, open source system monitoring and alerting system that stores metrics data as time series. Prometheus can be configured to:

  • Scrape one or more targets that expose metrics data on http endpoints
  • Store metrics data on http endpoints
  • Allow metrics data querying through its own http interface
  • Serve as a data source in a further layer of integration with visualization software such as Grafana

You can configure Posit Workbench to publish metrics to an http endpoint that Prometheus can access.

Prometheus installation

You can install Prometheus in several ways, depending on your infrastructure. Consult the Prometheus installation documentation for installation procedures.

Configuration

Configuring Workbench to publish metrics

To configure Posit Workbench to publish metrics:

  1. Set metrics-enabled=1 in rserver.conf.
  2. Restart Workbench with rstudio-server restart.

By default, the /metrics endpoint is available at the configured www-address and port 8989. In the configuration example below, metrics are retrieved from https://my.example.workbench.org:8989/metrics:

# /etc/rstudio/rserver.conf
www-address=my.example.workbench.org
ssl-enabled=1
#...
metrics-enabled=1

Below is the full set of configuration options:

Config Option Description Possible Values Default Value
metrics-enabled Whether or not to enable Prometheus metrics endpoint. 0 or 1 0
metrics-port The port to use for the Prometheus metrics endpoint. Any valid Posix port that is not already in use, usually from 1024 - 65535 8989
metrics-address The address to use for the prometheus metrics endpoint. When not present, the value of www-address is used, and TLS will be determined using ssl-enabled. When present, TLS will be inferred from the HTTP(S) scheme provided. If no HTTP(S) scheme is provided, HTTP will be used. A valid network address. The value of www-address

It is strongly recommended to use the above settings to control how Prometheus metrics are exposed by Workbench. However, if more advanced control is needed, a custom NGNIX directive block can be provided by editing the /etc/rstudio/nginx.metrics-directives.conf file. The location of this file can by modified with XDG_CONFIG_DIRS or RSTUDIO_CONFIG_DIR, as with other configuration files, or it can be set explicitly by adding server-nginx-metrics-directives-path=/path/to/desired/file.conf to rserver.conf. When this file is used, metrics-port and metrics-address will be ignored. Additionally, TLS must be explicitly included in the server directive block. When this file is used, it is not possible to configure Workbench to supply metrics directly from <www-address>/metrics.

To use this file, a server directive containing the location at which the metrics information should be served. Below is an example where the directive has been modified so that the Prometheus metrics can be accessed directly from http://127.0.0.1:

# /etc/rstudio/nginx.metrics-directives.conf
server {
    listen 127.0.0.1:80;
    location = / {
        proxy_pass http://unix:/var/run/rstudio-server/rstudio-rserver/rserver.socket:/metrics/;
        proxy_set_header Host $http_host;
        add_header Strict-Transport-Security "max-age=86400";
        proxy_redirect http://unix:/var/run/rstudio-server/rstudio-rserver/rserver.socket:/metrics/ $scheme://$host:80/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_read_timeout 20d;
    }
}

Configuring Prometheus

Consult the Prometheus configuration documentation for information on how to configure Prometheus.

You need to add a target to the Prometheus configuration that matches the endpoint you configured in Workbench. Using the previous example Workbench configuration, here is an example of how to configure Prometheus:

/etc/prometheus/prometheus.yml
global:
  scrape_interval: 15s
scrape_configs:
  - job_name: 'PWB'
    metrics_path: /metrics
    scheme: https
    static_configs:
      - targets: ['my.example.workbench.org:8989']
Prometheus and load-balanced Workbench configurations

If Workbench is running in a load-balanced configuration, each Workbench node publishes metrics on its own /metrics endpoint. To configure Prometheus to aggregate all Workbench instance metrics, add each Workbench node address to the targets list in prometheus.yml:

/etc/prometheus/prometheus.yml
      - targets:
        - 'my.node1.workbench.org:8989'
        - 'my.node2.workbench.org:8989'
        - 'my.node3.workbench.org:8989'

Metric data

Workbench publishes three types of Prometheus metrics: counters, histograms, and gauges. Here’s a brief definition of each type:

Counter
This metric maintains a steadily increasing count. It is reset each time the Workbench rserver process is restarted (typically with rstudio-server restart). An example counter is the metric pwb_sessions_launched_total.
Histogram
Maintains a set of buckets, each of which provided ascending threshold values. The histogram counts the number of samples below each threshold value for each bucket. An example histogram metric is: pwb_session_startup_duration_seconds. As with Counters, the counts are reset when the rstudio-server service is restarted.
Gauge
Provides the current value of a given metric for each sample. An example gauge metric is pwb_sessions.

Each metric may have labels attached to it, allowing the metric data to be queried by each label. For example, session metrics have a type label that specified the name of the IDE used: rstudio-pro, vscode, jupyter, or positron. When labels are attached to a metric, the metric value is maintained for each combination of values for each label.

Workbench http metrics

Metric Type Description Labels
pwb_http_requests_inflight Gauge Number of HTTP requests in progress.
pwb_http_handlers_inflight Gauge Number of HTTP requests active on a thread.
pwb_http_response_size_bytes_total Counter Size of all HTTP responses from this server. code, method, uri
pwb_http_requests_total Counter Number of HTTP requests served. code, method, uri
pwb_http_response_duration_seconds Histogram Duration of HTTP requests served, in seconds code, method, uri

IDE session metrics

Session metrics provide information about sessions that are currently active as well as amalgamated session data that relates to the current run of Workbench. To see historical data for sessions that are no longer active, see the Workbench API documentation

Metric Type Description Labels
pwb_sessions_launched_total Counter Cumulative IDE sessions launched from this server. type
pwb_sessions_failed_total Counter Binary requests, by distro. type
pwb_session_startup_duration_seconds Histogram Duration in seconds for session starts. type
pwb_session_startup_and_connect_duration_seconds Histogram Duration of the start including time to join. type
pwb_sessions Gauge Count of active workbench sessions by status. cluster, type, status
pwb_last_active_session_time Gauge Timestamp of the last active Workbench session.

Job metrics

Metric Type Description Labels
pwb_jobs_launched_total Counter Cumulative Workbench jobs launched from this server.

See the Workbench Jobs section of the User Guide for more information about Workbench Jobs.

System metrics

Metric Type Description Labels
pwb_build_info Gauge Workbench version, release name, and metrics version
pwb_active_user_sessions Gauge Current number of active user logins on this server.
pwb_license_expiry Gauge License expiry time in seconds since Epoch
pwb_license_user_seats Gauge Number of users available under this node’s product license
pwb_license_active_users Gauge Number of registered users in the internal database, used for licensing.

Launcher metrics

These metrics apply to the Posit Job Launcher, which launches sessions and Workbench jobs. The Posit Job Launcher makes requests to various plugins to launch the different types of sessions and jobs.

Metric Type Description Labels
pjl_build_info Gauge Job Launcher build version information Version
pjl_http_requests_total Counter Number of HTTP Job Launcher requests code, method, uri
pjl_http_response_duration_seconds Histogram Duration of HTTP requests served, in seconds code, method, uri
pjl_http_response_size_bytes_total Counter Size of all HTTP responses from the Job Launcher code, method, uri
pjl_http_requests_inflight Gauge Number of HTTP requests to the Job Launcher currently being served
pjl_http_handlers_inflight Gauge Number of Launcher HTTP handlers occupying worker thrreads
pjl_http_response_duration_seconds Histogram Duration of Job Launcher HTTP requests served, in seconds code, method, uri
pjl_launcher_plugin_requests_total Counter Number of Job Launcher plugin requests cluster_name, type
pjl_launcher_cpu_percent Gauge CPU used by the Job Launcher
pjl_launcher_memory_bytes Gauge Memory used by the Job Launcher
pjl_launcher_plugin_manager_status Gauge Status of the Job Launcher’s Plugin Manager: Green (0), Yellow(1), Red (2)

Metrics details

Normally the /metrics URL is requested by a Prometheus server, but you can also load this URL into a web browser to see what information Workbench is publishing.

For each metric, there is a # HELP comment and a # TYPE comment in the metrics output. Below is an example of what some metrics may look like:

# HELP pwb_http_requests_total Total HTTP requests served.
# TYPE pwb_http_requests_total counter
pwb_http_requests_total{code="200",method="GET",uri="/metrics"} 4
pwb_http_requests_total{code="200",method="POST",uri="/log"} 2
pwb_http_requests_total{code="200",method="POST",uri="/events"} 225
pwb_http_requests_total{code="200",method="POST",uri="/rpc"} 413
# HELP pwb_license_active_users Named users actually consuming the user seats available under this node's product license.
# TYPE pwb_license_active_users gauge
pwb_license_active_users 3

Visualizing metrics with Grafana

While Prometheus provides basic visualization capabilities, Grafana offers more powerful dashboarding and visualization features for monitoring Workbench metrics. Workbench ships with an example Grafana dashboard that can help you get started with visualizing performance metrics.

Setting up Grafana for Workbench

  1. Install Grafana by following the steps in the Grafana installation guide. Install Grafana somewhere accessible to both your browser and Prometheus server. These steps assume you’ve installed both on localhost.

  2. Access the Grafana web interface (default: http://localhost:3000) and set up an admin account.

  3. Add Prometheus as a data source:

    • Navigate to Configuration > Data Sources
    • Select Add data source > Prometheus
    • Set the URL to your Prometheus server (typically http://localhost:9090)
    • Select Save & Test to verify the connection

  4. Import the Workbench dashboard:

    • Navigate to Dashboards > Import
    • Upload or paste the contents of the example dashboard JSON file located at /usr/lib/rstudio-server/extras/grafana/workbench-dashboard.json
    • Select your Prometheus data source in the import options
    • Select Import

Sample visualizations

The Workbench dashboard includes panels for metrics such as:

  • Active user sessions over time
  • Session duration statistics
  • HTTP requests by endpoint
  • Session startup times
  • License usage metrics

You can customize these dashboards to focus on the metrics most relevant to your environment, or create new dashboards to visualize specific aspects of Workbench performance.

Visualizing load-balanced metrics

If you’ve configured Prometheus to aggregate metrics across multiple load-balanced Workbench nodes, we recommend that you customize the example Grafana dashboard to fit your organization’s needs. The example dashboard is a starting point and only considers single-node metrics. While it will function with aggregated metrics, it might display metrics in unexpected formats.

Back to top