Logging

Posit Connect provides a structured logging implementation. Structured logging provides well-defined log formatting and flexible configuration, making it well-suited for log aggregators like Datadog or Splunk.

Note

Content logs are accessible via the Posit Connect dashboard and jobs API. See the jobs cookbook examples for more details.

Logging configuration is managed by the Logging configuration section and the examples below illustrate possible logging configurations.

In this example, service logs will be emitted as JSON to STDOUT:

; /etc/rstudio-connect/rstudio-connect.gcfg
[Logging]
ServiceLogFormat = JSON
ServiceLog = STDOUT

In this example, service logs will be emitted as TEXT to the specified logfile, and access logs will be emitted as JSON to STDERR.

; /etc/rstudio-connect/rstudio-connect.gcfg
[Logging]
ServiceLogFormat = TEXT
ServiceLog = /var/log/logfile.log
AccessLogFormat = JSON
AccessLog = STDERR

A full list of structured logging configuration options is provided in the Appendix.

Using environment variables to configure logging

Similar to how it works for other configuration sections, logging can be configured using special environment variables.

For logging, however, Posit Connect follows a different variable precedence such that the environment variables specified at runtime will take precedence over the variables defined in a configuration file that gets passed to Posit Connect at runtime. This is done in order to properly configure logging in Posit Connect very early during startup.

The following example sets the respective environment variables needed to change output, format and level of the Service Log:

CONNECT_LOGGING_SERVICELOG=service.log
CONNECT_LOGGING_SERVICELOGFORMAT=JSON
CONNECT_LOGGING_SERVICELOGLEVEL=WARN

A more detailed description on how to use environment variables for configuring Posit Connect is provided in the Appendix.

Logging to `STDOUT`/`STDERR`

Logging to STDOUT and/or STDERR may be useful when using Kubernetes to forward logs to a centralized location, or when using systemd to aggregate logs in a single-host environment.

An example configuration which outputs logs to STDOUT and/or STDERR is provided below. In this example, access logs are emitted to STDERR:

; /etc/rstudio-connect/rstudio-connect.gcfg
[Logging]
AccessLog = STDERR

Logging to a file

Migrating to structured logging using a logfile requires either:

Updating aggregation tooling to ingest logfiles from the new default locations listed in the Appendix.
Configuring Posit Connect to write logfiles to the basic logging locations, allowing aggregation tooling to continue to ingest logs without modification.

An example configuration which specifies a non-default logfile is provided below. In this example, access logs are emitted to a logfile at /var/logs/non-default.log:

; /etc/rstudio-connect/rstudio-connect.gcfg
[Logging]
AccessLog = /var/logs/non-default.log

Logrotate

If logrotate is available when Posit Connect is installed, a logrotate configuration will also be installed.

Each log type has a different default rotation frequency and retention period:

Log Type	Default Rotation Frequency	Default Retention Period
Service Logs	Daily	30 Days
Access Logs	Daily	30 Days
Audit Logs	Monthly	12 Months

When a logfile is rotated, the old logfile will be compressed and stored alongside the original log file with a .1.gz extension (then .2.gz, etc.). Compressed logs are then maintained for the retention period.

The default logrotate configuration is provided below:

/var/log/rstudio/rstudio-connect/rstudio-connect.log {
       rotate 30
       daily
       copytruncate
       compress
       delaycompress
       notifempty
       missingok
       su root root
}
/var/log/rstudio/rstudio-connect/rstudio-connect.access.log {
       rotate 30
       daily
       copytruncate
       compress
       delaycompress
       notifempty
       missingok
       su root root
}
/var/log/rstudio/rstudio-connect/rstudio-connect.audit.log {
       rotate 12
       monthly
       copytruncate
       compress
       delaycompress
       missingok
       su root root
}

If you configure Posit Connect to write to a different file, you will need to update the logrotate configuration accordingly.

Permissions

For some log aggregators (such as Datadog or CloudWatch), file permissions may need to be adjusted. Logfile(s) must be at least 0644 (group/world readable) and their enclosing directory must be at least 0755 (group/world searchable).

Alternatively, the aggregation agent could be added to a specific group or ACL, but this approach will be particular to the aggregation agent and distro.

Structured Logs

Structured logging emits three types of log events: Service Logs, Access Logs, and Audit Logs. Debug Logs are troubleshooting-oriented service logs that can be enabled for specific services. Application Logs are not affected when structured logging is enabled.

Warning

When modifying the format of structured logs, be sure to clear the output target location to avoid mixing log formats. For example, you may want to clear the service logfile before switching from TEXT to JSON or vice versa.

Service Logs

Note

If you are using the official Posit Connect container image, the server logs are written to stdout by default and are not available at the default log path /var/log/rstudio/rstudio-connect/rstudio-connect.log. If you prefer to log to a file, you will need to modify the value of the Logging.ServiceLog setting.

Service logs capture the operating behavior of Posit Connect.

By default, service logs are written to /var/log/rstudio/rstudio-connect/rstudio-connect.log in TEXT format. The default logfile is owned by root with permissions 0600.

Warning

In Posit Connect editions prior to 2022.06, basic logging emitted service logs to /var/log/rstudio-connect.log by default. This differs from the current default for both basic and structured logging.

The following service log configuration options are available. See the Appendix for further details.

Configuration Parameter	Description
`Logging.ServiceLog`	Output destination
`Logging.ServiceLogLevel`	Minimum severity level for logging messages
`Logging.ServiceLogFormat`	Output format

Service Log Format

Regardless of format, service logs contain the following mandatory fields:

Field	Description
`level`	Log level (`info`, `warn`, etc)
`msg`	Context-specific content describing the state of the service
`time`	Timestamp indicating when the log was generated

Additional fields can be attached to service logs to provide service-specific contextual information.

An example service log in JSON format is provided below:

{
  "level":"info",
  "msg":"Startup complete (~4.548667s)",
  "time":"2022-04-07T15:38:39.472Z"
}

Here is an equivalent service log in TEXT format, where the log entry’s fields are space-delimited <key>=<value> pairs:

time="2022-04-11T17:01:34.465Z" level=info msg="Startup complete (~4.8366369s)"

Service Log Level

The service log level determines the minimum severity of service log entries. It is configured with the Logging.ServiceLogLevel property. The default log level is INFO, and available log level settings (in order of severity) are INFO, WARN and ERROR.

Debug Logs

Debug logs provide troubleshooting information about specific services. They are enabled by setting the Logging property to true within the service’s configuration section. Available debug log properties are provided below:

TableauIntegration.Logging
LDAP.Logging
PAM.Logging
ProxyAuth.Logging
OAuth2.Logging
SAML.Logging

For example, the following configuration would enable debug logging for an LDAP auth workflow:

; /etc/rstudio-connect/rstudio-connect.gcfg
[LDAP]
Logging = true

Debug logs are a specific subset of service logs with the field level set to debug. Therefore, their formatting and output location are configured by Logging.ServiceLogFormat and Logging.ServiceLog, respectively.

Note

Setting a debug log overrides any explicitly set Logging.ServiceLogLevel configuration and forces the log level to DEBUG. The DEBUG log level is highly permissive and will record INFO, WARN, and ERROR logs.

Access Logs

Access logs capture information about client requests received by Posit Connect.

Note

Access log timestamps are generated when a request is received by Posit Connect, but access events are not written to the log stream until the response completes. Cross-comparing an access log stream’s sequential ordering with the timestamp of each logged event may appear non-chronological. This is an expected side effect of unique requests requiring different amounts of processing time.

By default, access logs are written to /var/log/rstudio/rstudio-connect/rstudio-connect.access.log in COMMON format. The default logfile is owned by root with permissions 0600.

The following access log configuration options are available. See the Appendix for further details.

Configuration Parameter	Description
`Logging.AccessLog`	Output destination
`Logging.AccessLogFormat`	Output format

Access Log Format

Posit Connect can format access logs according to three standards:

Structured JSON
Common Format
Combined Format

Structured JSON Format

Access logs which are formatted as structured JSON contain the following fields:

Field	Description
`type`	Log type (access)
`level`	Log level (`info`, `warn`, etc)
`time`	Timestamp indicating when request was received by Posit Connect
`endpoint`	Requested API endpoint
`method`	HTTP method attached to request
`size`	Response size
`status`	HTTP response status code

An example access log in Posit Connect’s structured JSON format is provided below:

{
  "type":"access",
  "level":"info",
  "time":"2022-04-07T18:29:07.8109901Z",
  "endpoint":"/connect/",
  "method":"GET",
  "size":2306,
  "status":200
}

Common Format

See Apache’s Common Log Format documentation for a full description of the COMMON log formatting standard.

An example COMMON access log is provided below:

172.17.0.1 - - [11/Apr/2022:17:08:38 +0000] "GET /connect/ HTTP/1.1" 200 2306

Combined Format

See Apache’s Combined Log Format documentation for a full description of the COMBINED log formatting standard.

An example COMBINED access log is provided below:

172.17.0.1 - - [11/Apr/2022:17:10:33 +0000] "GET /connect HTTP/1.1" 307 45 "" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36"

Audit Logs

Audit logs capture information about the set of events defined by Posit Connect’s auditing system.

Note

Audit logfiles contain actions performed through a specific Posit Connect server. In a clustered installation, the auditing files from each server need to be combined.

The Connect dashboard, Server API, and usermanager tool present all auditing information, not per-server records.

Note

The auditing system cannot be disabled and always records changes to the Posit Connect database (even when an optional log file is configured). Database audit entries are maintained indefinitely.

By default, audit logs are only recorded in the database. Admin users are able to view audit logs from the database in the dashboard.

The following audit log configuration options are available. See the Appendix for further details.

Configuration Parameter	Description
`Logging.AuditLog`	Output destination
`Logging.AuditLogFormat`	Output format

Audit Log Format

When logged to a file, audit logs can be in either TEXT or JSON formats. See the Appendix for more details about audit logs format types.

Note that different formats will always show the same fields. See the Audited Events page for more details about all available fields.

An example audit log in JSON format is provided below:

{
  "type":"audit",
  "level":"info",
  "action":"user_login",
  "entry_id": 1,
  "msg":"New session by user: 123-456-789 (user@posit.com)",
  "time":"2022-04-07T18:47:21.769Z",
  "user_description":"user_description",
  "user_id":1
}

Here is an equivalent audit log in TEXT format, where fields are space-delimited <key>=<value> pairs:

time="2022-04-11T17:05:33.887Z" level=info msg="New session by user: 123-456-789 (user@posit.com)" action=user_login entry_id=1 type=audit user_description="user_description" user_id=1

Using environment variables to configure logging

Logging to STDOUT/STDERR

Logging to a file

Logrotate

Permissions

Structured Logs

Service Logs

Service Log Format

Service Log Level

Debug Logs

Access Logs

Access Log Format

Structured JSON Format

Common Format

Combined Format

Audit Logs

Audit Log Format

Logging to `STDOUT`/`STDERR`