Content Execution Environments

In Posit Connect, an execution environment refers to the metadata that defines the characteristics of a container image. Once Connect selects an execution environment, Connect uses the image associated with that environment to build and execute your content on Kubernetes.

This appendix explains:

The concept of execution environments.
How to declaratively manage execution environments via a configuration file.
How to use the Environments tab in the Connect UI to manage execution environments.
How execution environments are selected to build and execute content.
How to build images for use as execution environments.

Overview

Important

Execution environments in Connect should not be confused with environment variables. Environment variables let you provide named values used when running your content and are configured in the Advanced tab of the Content Settings panel.

Execution environments describe the capabilities of a given container image. The capabilities of an image refer to the available Python, Quarto, and R installations for the image, and the version of each installation. Connect uses the capabilities of the image when determining whether an environment is compatible for building and executing content. The process used by Connect to select a compatible environment is discussed later in this appendix.

There is a one-to-many relationship between an image name and execution environments in Connect. For example, the image named rstudio/content-base:r4.1.0-py3.9.2-jammy may be referenced by multiple execution environments. Because the Python and R package caches are tied to the container image name, this also means that execution environments which reference the same container image will also share a package cache.

When content is first deployed, Connect uses the selected environment to create a build container, which installs the Python and R packages required by your content. Each time your content runs, Connect uses that same environment to create an execution container. Execution environments do not share Python and R package caches.

Users can interact with execution environments by visiting the System > Environments tab in the Connect UI, or by interacting with the Connect server API.

Environment supervisors

Under the local execution model, it is possible to configure Posit Connect to invoke a program supervisor before building or executing content. The supervisor is a script or program that must be available and executable on the Connect server.

Under the off-host execution model, you can still use a supervisor, but it is necessary to configure it per-environment rather than globally (as the same script or program might not be installed on every image).

Please see the Program Supervisors section of the Posit Connect Admin Guide to read more about the uses and requirements of supervisors.

Note

It is still possible to configure a global supervisor in the Connect configuration file when using off-host execution; it is only used when invoking processes local to the Connect server (like git processes). A warning is printed at startup to remind you that the supervisor will not be invoked when building or executing content.

Note

Unlike the global supervisor, per-environment supervisors cannot be validated at startup. Please ensure that any supervisor you configure is available and executable in the relevant image and that it conforms to the rules described in the Admin Guide.

The configured supervisor script or program must exist and be executable within the image you are describing. One way to accomplish this is to add the supervisor script to the image when it is built.

Matching strategy

An environment’s matching strategy tells Connect if/how the environment is available for selection. The default value, any, means that Connect can consider the environment for selection if one is not explicitly requested by the content. The value exact indicates that the environment should never be selected automatically by Connect, and it can only be used if it is requested by the content. See the section on Publisher environment selection for details about how to request a specific environment for a given piece of content. The value none means that the environment is disabled and cannot be used.

Declarative management

Connect supports declarative management of execution environments through a YAML configuration file. When configured, Connect reads environment definitions from the file on startup and polls it for changes, syncing environments to the database automatically without requiring a restart.

This approach replaces the legacy bootstrapping mechanism, which only applies on the first startup when no environments exist.

Getting started

To enable declarative management, add an executionEnvironments list to your Helm chart values.yaml. Each entry uses the same fields as the Environments API. At minimum, each entry requires a name field specifying the container image:

values.yaml

executionEnvironments:
  - name: ghcr.io/my-org/connect-runtime:r4.4-py3.11
    title: "R 4.4 / Python 3.11"
    matching: any
    python:
      installations:
        - version: "3.11.3"
          path: /opt/python/3.11.3/bin/python3
    r:
      installations:
        - version: "4.4.2"
          path: /opt/R/4.4.2/bin/R

  - name: ghcr.io/my-org/connect-runtime:r4.3-py3.10
    title: "R 4.3 / Python 3.10"
    matching: any
    python:
      installations:
        - version: "3.10.12"
          path: /opt/python/3.10.12/bin/python3
    r:
      installations:
        - version: "4.3.3"
          path: /opt/R/4.3.3/bin/R

Run helm upgrade to apply the changes. The chart renders the list into a ConfigMap, mounts it into the Connect pod, and sets the ExecutionEnvironments.ConfigFilePath configuration automatically. Unlike the legacy launcher.customRuntimeYaml, changes take effect on every helm upgrade without requiring a pod restart or database reset. The kubelet updates the mounted file automatically when the ConfigMap changes, and Connect detects the update within seconds.

See the Execution environments section of the Helm chart README for additional configuration options.

Note

The configuration file also accepts JSON. You can save a response from the Environments API directly as the configuration file to bootstrap your definitions from existing environments.

After the upgrade completes, environments from the file appear in the Environments tab and are available for content execution.

How sync works

Connect syncs the environments file to the database using the following process:

On startup, Connect reads the file and syncs all definitions to the database. If the file cannot be read or contains invalid definitions, Connect fails to start.
While running, Connect polls the file for changes every few seconds. When a change is detected, Connect re-reads the file and syncs again. If errors occur during a runtime sync, Connect logs the error and retries on the next poll without modifying the database.
All operations within a single sync are transactional. If any definition fails validation, the entire sync rolls back and the database remains in its previous state.
If the file is removed, Connect removes all file-managed environments from the database.

Note

An empty file is valid and removes all file-managed environments, just like removing the file.

Environment matching and adoption

When syncing, Connect matches each definition in the file to an existing environment in the database using a two-phase process:

Globally Unique Identifier (GUID) matching: If the definition includes a guid field, Connect matches by GUID only. If no environment with that GUID exists, Connect creates a new one with that GUID. Connect does not fall back to name matching when a GUID is provided.
Name matching: If no guid is provided, Connect matches by name. It first looks for an environment already managed by the file with that name. If none is found, it looks for an unmanaged environment with that name to adopt.

If you already have environments created through the UI or API and you enable declarative management, Connect adopts each existing environment whose name matches a definition in the file. Connect only adopts environments with a matching name. Environments that do not appear in the file remain unmanaged, and the sync process does not affect them.

During adoption, Connect preserves the existing environment’s GUID so that content deployments and permission assignments that reference it continue to work. After adoption, the environment is managed by the file and subsequent syncs match it directly.

When to use explicit GUIDs

In most cases, name matching handles everything and you do not need to specify GUIDs. You only need an explicit guid when:

The name you want to use already exists in the database from another config entry or an environment created through the API, causing ambiguity.
Multiple environments in the file share the same name, for example two entries for different tags of the same image. Connect rejects duplicate names in the file unless each entry has an explicit GUID.

GUIDs must be valid RFC 4122 v4 Universally Unique Identifiers (UUIDs). You can generate one using a standard UUID tool:

Terminal

uuidgen

Then add it to the definition:

values.yaml

executionEnvironments:
  - guid: "a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d"
    name: ghcr.io/my-org/connect-runtime:r4.4-py3.11
    title: "R 4.4 / Python 3.11"

Note

If multiple environments in the database share the same name and your definition does not include a GUID, Connect cannot determine which one to adopt. The sync fails with an error listing the matching GUIDs. To resolve this, add the GUID of the environment you want to manage to your definition. For example, if the error lists GUIDs aaa... and bbb... and you want to manage aaa..., add guid: "aaa..." to the corresponding entry in your values.yaml.

Mixed management

File-managed environments and environments created through the UI or API coexist in Connect. The sync process only modifies environments that it manages:

Adding an environment to the file creates or adopts it as file-managed.
Updating a definition in the file updates the corresponding file-managed environment in the database.
Removing an environment from the file deletes it from the database.
The sync process never modifies or deletes environments created through the UI or API.

Note

The API and UI do not allow edits to file-managed environments. To update a file-managed environment, edit the values.yaml and run helm upgrade.

Limitations

Name is immutable. Connect does not allow changes to the name field after creating an environment. The name is the container image reference. To change the image associated with an environment, remove the old entry from the file and add a new one.
Access control lists are not supported in the configuration file. Manage environment permissions through the API or the Environments tab. Permissions set through the API or UI are preserved across syncs. See Execution environment permissions for details.

Migrating from legacy bootstrapping

If you are currently using launcher.customRuntimeYaml in your Helm chart values, you can migrate to declarative management:

Move the environment definitions from launcher.customRuntimeYaml.images to executionEnvironments. Each images entry becomes a top-level list item in executionEnvironments, without the name: "Kubernetes" and images: wrapper.
Run helm upgrade. On the first sync, Connect adopts existing environments by name so that content and permission references remain valid.

Note

When both executionEnvironments and launcher.customRuntimeYaml are set, only the declarative management runs. The legacy bootstrapping path is skipped.

Bootstrapping execution environments (deprecated)

Warning

This bootstrapping approach is deprecated. It only runs on the first startup when no environments exist and silently ignores subsequent changes. Use declarative management instead, which syncs environments on every startup and detects file changes without requiring a restart.

If Connect was installed using the official Helm Chart, then a default set of execution environments is created by default the first time the server starts. If any execution environments already exist in Connect, then no environments are created or modified on subsequent restarts. This allows users to bootstrap new installations by providing a set of execution environments which are created automatically the first time the server starts. To customize the set of environments that are created automatically, server administrators can modify the YAML configuration file specified by the Launcher.ClusterDefinitions setting.

This can be accomplished by modifying the values.yaml for your installation. If used for a new installation, the following values.yaml creates a single execution environment with three installations (one Python, one R, and one Quarto) with the image my-org/content-base:r4.1-py3.10.

values.yaml

launcher:
  customRuntimeYaml:
    name: "Kubernetes"                             # (required) this is always "Kubernetes"
    images:
      - name: my-org/content-base:r4.1-py3.10      # (required) a valid container image name

        matching: any                              # (optional) one of [none, any, exact] defines the image selection
                                                   # behavior for this environment

        supervisor: /scripts/connect-supervisor.sh # (optional) defines a supervisor script for this environment

        python:                                    # (optional) defines a set of python installations
          installations:
            - path: /opt/python/3.10.4/bin/python3 # (required) the absolute path to the executable
              version: 3.10.4                      # (required) semantic version of the installation

        r:                                         # (optional) defines a set of R installations
          installations:
            - path: /opt/R/4.1.3/bin/R
              version: 4.1.3

        quarto:                                    # (optional) defines a set of Quarto installations
          installations:
            - path: /opt/quarto/bin/quarto
              version: 1.0.35

Warning

The Launcher.ClusterDefinitions YAML file is only read when no environments exist in the database. Subsequent changes to the file are silently ignored. For ongoing management of environments via a configuration file, use declarative management instead.

Generating `Launcher.ClusterDefinitions` YAML (deprecated)

Note

This section applies to the deprecated Launcher.ClusterDefinitions bootstrapping approach. For the recommended approach, see declarative management.

The build-image-yaml.sh and examine-image.sh scripts can help generate YAML for your images. These scripts are available from the content/scripts subdirectory of the rstudio/rstudio-docker-products repository.

These scripts:

Use docker run to inspect a named image. Images not already in the local Docker cache are downloaded. Images present in the local cache are not pulled again.
Look for R installations at /opt/R/<VERSION> and /opt/local/R/<VERSION>. If none are found, they look for R using the PATH.
Look for Python installations at /opt/python/<VERSION> and /opt/local/python/<VERSION>. If none are found, they look for python, python2, and python3 using the PATH.

The following command produces YAML describing three images.

Terminal

./scripts/build-image-yaml.sh \
    rstudio/content-base:r3.6.3-py3.8.8-jammy \
    rstudio/content-base:r4.0.5-py3.8.8-jammy \
    rstudio/content-base:r4.1.0-py3.9.2-jammy \
    > runtimes.yaml

Review the generated YAML to confirm that the expected R and Python installations are included. Remove any detected installations that you do not want in use.

Environments tab

To view and manage available environments in the Connect UI, click System from the top menu bar, and then Environments. Capabilites for the Environments tab vary by role:

Administrators: create, update, or remove environments, and configure permissions and volume mounts for specific execution environments
Publishers: view available environments created by an Administrator
Viewers: no access to view available environments

Searching environments

You can use the Environments tab to search for execution environments based on the environment’s capabilities. You can search the list of environments by title, description, image name, image GUID, supervisor script path, or installation versions. You can use the column headers to sort the results.

Screenshot of the execution environments search bar with a search query for a version number — Environments Search

Adding new environments

To add a new execution environment:

Click + Add Environment.
An Add Environment window appears. Fill in the fields as appropriate:

Include a human-readable title and description for the environment, which is useful for Publishers when they select an execution environment for their content.
Add an installation. You must enter the semantic version of the interpreter and the absolute path to the interpreter’s executable.

Installation Add
Add Volume Mounts. See the Volume Mounts section of this guide for more information.
Add users or groups in the Access Control section. See the Environment Permissions section of this guide for more information.
If needed, click Advanced to add a matching strategy and/or a supervisor for the environment.
Example configuration:

Environment Add

Environment selection

When your content is deployed, Connect can automatically determine the most appropriate environment based on the R and Python version constraints specified in the manifest.json file.

Example manifest.json file segment:

manifest.json

"environment": {
    "python": {
        "requires": ">=3.8,<4.0"
    },
    "r": {
        "requires": ">=4.2.0,<5.0"
    }
}

With this information, Connect discards images that do not have compatible R and Python installations. From the remaining images, the best candidate is chosen by performing a pairwise comparison of the R and Python version components.

For content without an environment section, Connect uses the configured R and Python matching algorithms to do the image selection.

The R.VersionMatching setting controls which R installations are considered compatible with your content. We recommend a setting of major-minor rather than the default nearest configuration. The Connect Admin Guide contains more information about R version matching.

The Python.VersionMatching setting controls which Python installations are considered compatible with your content. The default of major-minor is appropriate for most installations. The Connect Admin Guide contains additional details about Python version matching.

Once an environment and appropriate R and Python installations are selected, Connect uses that environment to install R packages and Python packages before using that same environment to run your content.

Publisher selection

Publishers can select a specific environment for their content through the content settings in the dashboard, or the bundle’s manifest.json file. Selection is limited to the environments that they have access to, as determined by execution environment permissions.

Dashboard selection

To select an environment through the dashboard, navigate to the content settings. Under the Runtime tab, the Execution Environment section allows Publishers to select an environment to use the next time their content builds and executes. Publishers can choose between Automatic and Use a specific environment.

Note

Modifying this selection will cause your content to rebuild the next time it runs.

If Automatic is selected, then Connect will attempt to choose an environment that meets the requirements of your content using the strategy described above. If Use a specific environment is selected, then the Publisher is presented with a window where they must select an environment.

The environment selection window provides the same filter and sort functionality as the main Environments dashboard. Publishers can use this window to select a compatible environment for their content.

Note

The environment selected must satisfy the Python and R version requirements of your content using the same version-matching criteria listed above. If the environment is not compatible, then Connect cannot run the content.

Select environment — Publisher Environment Select

The environment selected through the content settings serves as the default environment that will be used when building and executing the content. If an image is specified by the bundle’s manifest.json, then the manifest selection is used instead. This means that bundle activation continues to work, even when the manifest selects a different image than the content’s default execution environment.

Manifest selection

Publishers can optionally set the environment.image field in the content’s manifest.json to select a container image to be used for building and executing content. This option provides a code-first approach to deployments and allows Publishers to have fine-grained control over the image used to build and execute content. It also supports image selection with git-backed deployments.

The Python client package, rsconnect-python (1.8.0+), supports an --image parameter when deploying content. Similarly, the rsconnect (0.8.26+) R package supports an image argument so that users can specify an image when deploying content to Posit Connect.

Note

The image selected by the manifest must also have a corresponding environment entry in Connect, and the Python and R version requirements of your content must be satisfied using the same version-matching criteria listed above. If the environment is not compatible, then Connect cannot run the content.

Note

Push-button deployment from the IDE does not currently support image selection.

About automatic environment selection

In some cases, it is useful to exclude an image from automatic selection. For example, a highly customized content image that is only suitable for running a very specific content item might not be generally suitable to run most content. In this scenario, users can set the matching field in the runtimes.yaml configuration to control image selection.

There are three possible values accepted by the matching field:

any (default) indicates that the image can be selected automatically by Connect, or it can be explicitly targeted by a bundle’s manifest.json.
none indicates that the image is effectively disabled. It will never be considered for selection.
exact indicates that the image can only be used when it is explicitly targeted by the bundle’s manifest.json. It will not be selected automatically by Connect, even if it is compatible with the content.

Building images for use as execution environments

Images

You probably want a selection of container images that reflect the versions of Python and R used by your organization. We recommend images which:

Cover each MAJOR.MINOR version of R.

This probably means images with R-3.6.X, R-4.0.X, R-4.1.X, R-4.2.X, and R-4.3.X, where X is the latest patch for each release.
Cover each MAJOR.MINOR version of Python.

This probably means images with Python-3.8.X, Python-3.9.X, Python-3.10.X and Python-3.11.X, where X is the latest patch for each release.

Posit has created a set of basic images that are available to use in your environment. Each image uses the Ubuntu 22.04 LTS (Jammy Jellyfish) release and includes single versions of Python, R, and Quarto, and system libraries commonly used by Python and R packages.

The Python installation is available at /opt/python/<VERSION>
The R installation is available at /opt/R/<VERSION>
The Quarto installation is available at /opt/quarto/<VERSION>

Available images are provided in the table below. New images may be added as new Python, R, and Quarto releases occur. If these images do not suit your organization, you can create and use custom images.

These images are available from GitHub Container Registry.

For content-base variants, see: https://github.com/rstudio/rstudio-docker-products/pkgs/container/content-base

For content-pro variants, see: https://github.com/rstudio/rstudio-docker-products/pkgs/container/content-pro

The source that builds these images is available from GitHub: https://github.com/rstudio/rstudio-docker-products/tree/main/content

Custom images

You are welcome to create your own images by extending the rstudio/content-base images or by creating new images from scratch.

Some general guidance when creating your own images:

Prefer smaller images where possible. Large images lead to increased launch times, as images may need to be downloaded from their hosting repository.
An image should contain at least one Python, Quarto, or R installation.
Images should include the system dependencies of the Python and R packages you use.
Consider having Posit Connect use different, smaller images than you use during development.
You do not need to include both Python and R in every image; choose combinations that mirror what you use during development.
You do not need to produce an image for every patch release of Python and R the most recent patch for each MAJOR.MINOR release is usually sufficient.

For further information, see Environment Management.

Runtime components

It’s possible that these images do not contain any runtime components of Posit Connect. When the job is launched, the container is augmented with a handful of required runtime components using a separate, auxiliary Posit Connect init-container. The use of an init-container to prepare the runtime components means that the content image does not need to be rebuilt for a particular version of Posit Connect. For more details about the init-container, see the Runtime Init-Container section.

Overview

Environment supervisors

Matching strategy

Declarative management

Getting started

How sync works

Environment matching and adoption

When to use explicit GUIDs

Mixed management

Limitations

Migrating from legacy bootstrapping

Bootstrapping execution environments (deprecated)

Generating Launcher.ClusterDefinitions YAML (deprecated)

Environments tab

Searching environments

Adding new environments

Environment selection

Publisher selection

Dashboard selection

Manifest selection

About automatic environment selection

Building images for use as execution environments

Images

Custom images

Runtime components

Generating `Launcher.ClusterDefinitions` YAML (deprecated)