Job queue

Posit Connect uses an internal job queue to schedule and execute background tasks. These tasks include rendering scheduled reports, polling Git repositories for updates, syncing user and group membership from external identity providers, and cleaning up after completed jobs.

Administrators can view and manage the queue from the System view under the Queue tab, or programmatically through the Connect Server API.

Queue types

The job queue handles four categories of work:

Default queue (default)

The default queue processes scheduled content renders. When a report, R Markdown document, Jupyter notebook, or other content has a schedule defined, Connect adds a queue item at the scheduled time. A worker picks up the item and executes the render.

Git queue (git)

The Git queue processes updates for Git-backed content. Connect periodically checks Git repositories for new commits based on the Git.PollingFrequency setting. When a repository has changes, Connect adds a queue item to pull the latest content and redeploy it.

Memberships queue (memberships)

The memberships queue handles syncing user and group information from external identity providers such as LDAP or SAML. When Connect synchronizes group memberships, it adds items to this queue. The LDAP.MembershipUpdateInterval setting controls the sync interval for LDAP. If your identity provider has many groups or users, this queue can contain a large number of items during a sync cycle.

Job finalizer queue (job-finalizer)

The job finalizer queue handles cleanup tasks after jobs finish. These tasks include recording job outcomes, updating metadata, and releasing resources associated with completed jobs.

Viewing the queue

Dashboard

Navigate to System > Queue to view all items in the queue. This view is only available to administrators. The Queue view displays the following information for each item:

Column Description
Created Time When the item was added to the queue.
Queue Which queue the item belongs to (default, git, memberships, or job-finalizer).
Type The type of work: ScheduledRender, GitFetch, MembershipUpdater, or JobFinalizer.
Active Whether a worker is processing the item.
Payload A JSON object with details about the work to be done. Click View to inspect it.
Hostname The Connect node processing the item (relevant in clustered deployments).

The table supports sorting by column. Click a column header to change the sort order.

API

Use the GET /v1/queue endpoint to list queue items programmatically. You can filter by queue name and paginate results:

Terminal
curl -H "Authorization: Key ${CONNECT_API_KEY}" \
  "${CONNECT_SERVER}__api__/v1/queue?queue_names=memberships&page_size=50"

See the API Reference for full endpoint documentation.

Managing queue items

Deleting individual items

You can delete a queue item that has not yet started processing:

  • Dashboard: Click the trash icon next to an inactive item in the Queue view.

  • API: Send a DELETE request to the unversioned queue endpoint:

    Terminal
    curl -X DELETE \
  -H "Authorization: Key ${CONNECT_API_KEY}" \
  "${CONNECT_SERVER}__api__/queue/{id}"

    Replace {id} with the numeric ID of the queue item.

Note

You cannot delete an item that a worker is actively processing. The Active column in the dashboard indicates whether a worker has started the item.

Bulk deletion through the API

The dashboard does not support bulk deletion. To remove multiple items at once, script against the API. For example, to delete all items in the memberships queue:

Terminal
# Fetch all memberships queue item IDs, then delete each one
curl -s -H "Authorization: Key ${CONNECT_API_KEY}" \
  "${CONNECT_SERVER}__api__/v1/queue?queue_names=memberships&page_size=500" \
  | jq -r '.results[].id' \
  | while read id; do
      curl -s -X DELETE \
        -H "Authorization: Key ${CONNECT_API_KEY}" \
        "${CONNECT_SERVER}__api__/queue/${id}"
    done
Warning

Deleting queue items removes pending work. Deleted scheduled renders will not run until the next scheduled time. Deleted membership syncs skip the current sync cycle. Only delete items that you are confident are no longer needed.

Troubleshooting

Old items in the queue

Items with old Created Time values that are not marked Active might indicate one of the following:

  • Worker capacity: All available workers are busy processing other items. The queue has a backlog but processes items when workers become available.
  • Stalled processing: A worker started the item but failed silently. This can happen if the Connect process restarted while processing work.
  • Duplicate prevention: Connect prevents duplicate items from being added to the queue. An old item can represent a recurring task that no worker has picked up because it is still queued from a prior cycle.

To investigate:

  1. Check the Active column. If an old item shows Active: Yes but appears stalled, the worker processing it might have encountered an issue.
  2. Review the Connect server logs for errors related to the queue type in question.
  3. If old items are no longer needed, delete them through the dashboard or API so that Connect can queue fresh items.

Large memberships queue

A large number of items in the memberships queue typically occurs when Connect syncs group memberships from an external identity provider with many groups. This is normal during a sync cycle. If the items persist after the sync cycle completes:

  1. Verify that the identity provider is accessible and responding.
  2. Check Connect server logs for LDAP or SAML errors.
  3. If the stale items are from a previous sync cycle that did not complete, delete them to allow a fresh sync.

Queue monitoring with OpenTelemetry

You can use OpenTelemetry metrics and traces to monitor queue health over time. The Job queue operations guide provides detailed metric queries for:

  • Detecting whether scheduled jobs are running
  • Identifying long-running or slow jobs
  • Monitoring queue backups and worker capacity