Building Git Packages

Advanced

Package Manager can build R and Python source packages from a Git repository. There are two Git sources types used for building from a Git repository:

This document provides information for how to create and edit Git builders for both R and Python packages. Most of the information applies to both git and git-python sources, specific notes will be provided in each section if that is not the case.

Git Builder Prerequisites

R Prerequisites

Package Manager requires an R installation on the system to build R packages from Git repositories.

Package Manager will only build source tarballs for R packages, not binary packages, so no other system dependencies are required unless you would like to optionally build package vignettes.

R Installation

In most cases, Posit recommends you install R from pre-compiled binaries.

To install from pre-compiled binaries, follow the instructions at Install R.

Alternatively, you can install R from source by following the instructions at Install R from source.

Package Manager supports two ways of discovering R installations: through explicit configuration and through automatic detection.

Automatically Detecting R

If Server.RVersion is not set, Package Manager attempts to automatically detect an R installation on the server. Automatic R detection will be disabled if Server.RVersion is configured.

Python Prerequisites

Python packages have several formats for describing the metadata of a package. This is often stored in a setup.py or pyproject.toml file and needs to be parsed to properly build the package. Because of this, Package Manager requires the following when using a git-python source with Python package repositories:

  • A Python installation on the system
  • The build and virtualenv modules for building the source and/or binary distributions

Package Manager will build both the source distribution (sdist) and binary distribution (wheel) for a Python package if possible. You may need to install additional system dependencies to build wheels, especially for packages with compiled code. If a wheel build fails, Package Manager will fall back to building only the source distribution.

Wheels with compiled code will be specific to the operating system or distribution that Package Manager is running on. Users on other operating systems or distributions may still need to compile the source distribution when installing Git-built packages.

Python Installation

In most cases, Posit recommends you install Python from pre-compiled binaries.

To install from pre-compiled binaries, follow the instructions at Install Python.

Alternatively, you can install Python from source by following the instructions at Install Python from source.

Package Manager supports two ways of discovering Python installations: through explicit configuration and through automatic detection.

Automatically Detecting Python

If Server.PythonVersion is not set, Package Manager attempts to automatically detect a Python installation on the server. Automatic detection will be disabled if Server.PythonVersion is configured.

Required Modules

The build and virtualenv modules are required to be installed so a Git builder can fetch necessary dependencies and build the package in an isolated environment. They can be installed with:

Terminal
pip install build virtualenv

If they are not installed in a location where python can call them with python -m build or python -m virtualenv, then Git builders for Python will be disabled.

Using Git Builders

Package Manager defines a git-builder as an entity that watches a Git endpoint for changes, and builds R or Python packages.

When referring to “Git” sources in Package Manager, this broadly refers to both git and git-python source types which are used depending on if a Git package repository points to an R or Python package.

An administrator follows these steps:

  1. Ensure the necessary prerequisites for R and/or Python have been configured, as described in Git builder prerequisites above.
  2. Create a git (R) or git-python (Python) source.
  3. Create a git-builder for the source, specifying whether to watch for commits to a Git branch or tags in a Git repository. The endpoint can be: HTTP, SSH (see below), or a local file path (see below). See the rspm create git-builder command for full details, e.g., how to track a specific branch.
  4. Based on the selection specified with the rspm create git-builder command, Package Manager clones the Git endpoint and runs a job to transform the Git repository into an R or Python package. The package is made available to any repositories subscribing to the source.
  5. Package Manager polls the Git endpoint to watch for either new commits or new tags (based on the selection specified with the rspm create git-builder command). If an update is available, Package Manager automatically pulls the latest changes and builds new packages in the background. For R packages, previous versions are archived. All Python package versions remain available in the source.
  6. Users can now install the packages with the proper installation tool for the corresponding language. For R, packages are installed from the repository via install.packages not devtools. Python packages are installed via pip.

For a specific example building an R package, reference the Git Builders for R Quickstart Guide. For a specific example building a Python package, reference the Git Builders for Python Quickstart Guide.

Building Package Vignettes for R

R packages with git sources have the optional ability to attempt building a package vignette. By default, the Git builders for R packages use the --no-build-vignettes option to build packages and bypass the complexity associated with additional software and system dependencies. If you want to enable vignette building and are willing to manage the required dependencies yourself, set the Git.AttemptVignettes option to true. When enabled, the builders will attempt to build the vignettes for R packages.

Even if the Git.AttemptVignettes setting is enabled, but the required dependencies are missing or any other issue occurs during the vignette building process, the builders will automatically fall back to using the --no-build-vignettes option. This ensures that the build process will continue, even if vignettes cannot be built.

Pulling Python Dependencies from a PyPI Repository

By default git-python Git builders will pull the necessary dependencies for building a Python package from https://pypi.org. If pulling from a different PyPI repository is desired, the Git.PyPIRepoURL config option can be used:

/etc/rstudio-pm/rstudio-pm.gcfg
[Git]
PyPIRepoURL = localhost:4242/pypi/latest/simple

When setting this, Git builders will pull the Python package dependencies from the defined URL instead of PyPI. This is also helpful if you would like to pull from a specific snapshot date:

/etc/rstudio-pm/rstudio-pm.gcfg
[Git]
PyPIRepoURL = localhost:4242/pypi/2023-12-01/simple

Git Logging on the Server

Server log messages related to this component can be shown by enabling debug logging. More information about activating debug logging is in the configuration appendix.

Accessing Restricted Git Endpoints Using Git Credentials

If Git builders require authentication to read a repository, Package Manager can use SSH keys, HTTPS credentials, or OAuth2 federated credentials to authenticate against the endpoint.

Choosing the Right Authentication Method

Package Manager supports four types of authentication for Git repositories:

  1. SSH Keys - Traditional public/private key authentication. Best for repositories that support SSH access and when you can manage SSH keys.

  2. HTTPS Credentials - Username and password/token authentication over HTTPS. Best for repositories that require Personal Access Tokens (PATs) or when SSH is not available.

  3. OAuth2 Federated Credentials - Modern OAuth2-based authentication that automatically obtains access tokens. Best for cloud-hosted repositories like Azure DevOps and Bitbucket that support OAuth2, and ideal for automated environments where token rotation is important.

  4. GitHub App Credentials - GitHub’s native application authentication using short-lived access tokens. Best alternative to SSH keys and HTTPS credentials for GitHub repositories, providing enhanced security, fine-grained permissions, and better integration with GitHub’s permission model.

For Azure DevOps users, we recommend OAuth2 federated credentials as they provide:

  • Automatic token rotation and management
  • Better security through short-lived tokens
  • Integration with Azure Active Directory
  • Support for both client credentials and federated identity authentication

For Bitbucket users, we recommend OAuth2 federated credentials as they provide:

  • Automatic token rotation and management
  • Better security through short-lived tokens
  • Simple setup using OAuth consumers

For GitHub users, we recommend GitHub App credentials over SSH keys and HTTPS credentials as they provide:

  • Enhanced security through short-lived access tokens (1 hour expiration) vs. long-lived SSH keys or PATs
  • Fine-grained repository permissions vs. broad account-level access
  • No dependency on individual user accounts or managing SSH key files
  • Automatic token rotation vs. manual SSH key or PAT management

Importing an SSH key

Begin by creating an SSH key and granting the SSH key access to the Git endpoint. Although Package Manager allows the use of SSH keys with no passphrase, it is still recommended to use a strong SSH key with a passphrase. The specific steps for granting access will depend on your Git provider.

Once you have the path to the SSH key, use the rspm import ssh-key command to name and securely store the SSH key for later use by Package Manager. If desired, you can now remove the SSH key file from your file system, as it is loaded into the Package Manager database as encrypted text. Multiple keys can be imported, but they must have unique names to refer to. This is the --name argument you pass to the rspm import ssh-key command, not the filename of the key.

Because the use of a password for the key requires writing the password to a file on disk, there is some risk of leaking the password while it is on disk. To mitigate this risk, encrypt the password using the rspm encrypt command prior to writing it in a file. Package Manager will understand passwords in the file as either plaintext or encrypted text, and will decrypt the password as necessary to unlock your SSH key.

To use the newly imported SSH key with a new Git builder, specify the key name with the --credential flag in the rspm create git-builder command.

Importing an HTTPS credential

HTTPS credentials consist of a username and password. In applications where two-factor authentication is used, you will often need a PAT that serves as the password. To import an HTTPS credential, use the rspm import https-credential command.

Note that using this command requires you to input the password/PAT on the command line, which may be stored in your command history. To avoid this possible credential leak, encrypt your password using the rspm encrypt command, and pass this encrypted string to the rspm import https-credential command instead.

To use the newly imported credential with a new Git builder, specify the credential name with the --credential flag in the rspm create git-builder command.

Importing OAuth2 Federated Credentials

OAuth2 federated credentials provide modern, secure authentication using short-lived access tokens that are automatically obtained from an OAuth2 provider. This method is particularly well-suited for cloud-hosted Git repositories like Azure DevOps and Bitbucket.

Package Manager supports OAuth2 federated credentials for multiple providers. The specific setup process varies by provider, but all use the same underlying OAuth2 client credentials flow.

Supported Providers

Package Manager supports OAuth2 federated credentials for:

  • Azure DevOps - Using Microsoft Entra ID (Azure Active Directory) with support for both client credentials and federated identity flows
  • Bitbucket - Using Bitbucket OAuth consumers with client credentials flow
Tip

Choosing Your OAuth2 Provider Setup:

  • Azure DevOps: Offers both client credentials (simpler) and federated identity (more secure, no long-lived secrets) flows. Choose federated identity for enhanced security in production environments.
  • Bitbucket: Uses a straightforward OAuth consumer setup with client credentials flow. Simpler to configure than Azure DevOps but requires managing a client secret.

Azure DevOps Setup

Using a Client Secret with Azure DevOps:

  1. Create a service principal in Azure Active Directory following Microsoft’s guide
  2. Configure the service principal with access to your Azure DevOps organization following Microsoft Entra ID OAuth documentation
  3. Note the client ID, client secret, and tenant ID

To import a federated credential using client credentials:

Terminal
# Import the federated credential with client secret
rspm import federated-credential \
  --name=azure-devops-cred \
  --client-id=your-client-id \
  --client-secret=your-client-secret \
  --token-endpoint=https://login.microsoftonline.com/your-tenant-id/oauth2/v2.0/token \
  --scope=https://app.vssps.visualstudio.com/.default

Using Federated Identity with OpenID Connect (OIDC) for Azure DevOps:

Federated Identity uses OpenID Connect (OIDC) tokens instead of client secrets. This provides enhanced security by eliminating long-lived secrets and is ideal for automated environments like CI/CD pipelines. This method requires that the external provider (e.g., Azure) can access your Package Manager instance to validate Package Manager ID tokens.

For Azure DevOps with Federated Identity:

  1. Set up federated identity authentication following Microsoft’s federated identity guide
  2. Configure your Azure AD application to trust tokens from your Package Manager environment
  3. Note the client ID, tenant ID, and configure the subject and audience claims

To import a federated credential using federated identity:

Terminal
# Import the federated credential with OIDC
rspm import federated-credential \
  --name=azure-devops-oidc \
  --client-id=your-client-id \
  --subject=repo:your-org/your-repo:ref:refs/heads/main \
  --audience=https://dev.azure.com/your-org \
  --token-endpoint=https://login.microsoftonline.com/your-tenant-id/oauth2/v2.0/token \
  --scope=https://app.vssps.visualstudio.com/.default
Note

The subject and audience values depend on your identity provider and use case:

  • Subject: Often represents the entity requesting access (e.g., repository, user, or service)
  • Audience: Typically the target service or resource you’re accessing
  • If not specified, Package Manager will use defaults: audience packagemanager and subject matching the git builder name

Bitbucket Setup

Using OAuth Consumers with Bitbucket:

Bitbucket uses OAuth consumers for application authentication. This is simpler than Azure DevOps setup and only requires creating an OAuth consumer with the appropriate permissions.

  1. Navigate to your Bitbucket workspace settings.
  2. Go to OAuth consumers under the “Apps and Features” section.
  3. Click Add consumer to create a new OAuth consumer.
  4. Configure the consumer:
    • Name: Give your consumer a descriptive name (e.g., “Package Manager”).
    • Callback URL: You can use a placeholder URL since Package Manager uses client credentials flow.
    • This is a private consumer: Important - Check this box to enable client credentials grants.
    • Permissions: Grant the minimum required permissions:
      • Repositories: Read access to the repositories Package Manager needs to access
  5. Save the consumer and note the Key and Secret values.

To import a Bitbucket federated credential:

Terminal
# Import the Bitbucket federated credential
rspm import federated-credential \
  --name=bitbucket-cred \
  --client-id=YOUR_CONSUMER_KEY \
  --client-secret=YOUR_CONSUMER_SECRET \
  --token-endpoint=https://bitbucket.org/site/oauth2/access_token \
  --scope=repository

Replace YOUR_CONSUMER_KEY with the Key from your OAuth consumer and YOUR_CONSUMER_SECRET with the Secret.

Then create a Git builder using the credential:

Terminal
# Create a Git builder for Bitbucket repository
rspm create git-builder \
  --url=https://bitbucket.org/your-workspace/your-repository.git \
  --source=your-source-name \
  --credential=bitbucket-cred

Using Your Federated Credentials

To use the newly imported federated credential with a new Git builder, specify the credential name with the --credential flag in the rspm create git-builder command.

Importing GitHub App Credentials

GitHub App credentials provide secure, automated authentication to GitHub repositories using GitHub’s native application authentication. This method serves as a modern alternative to SSH keys and HTTPS credentials (Personal Access Tokens) for GitHub repositories, offering enhanced security through time-limited access tokens and fine-grained permissions.

GitHub App authentication offers several advantages over SSH keys and Personal Access Tokens (PATs):

  • Enhanced Security: Uses short-lived access tokens (1 hour expiration) instead of long-lived SSH keys or PATs
  • Fine-grained Permissions: Apps can be granted only the specific repository access needed, unlike SSH keys which typically have broad access
  • Audit Trail: Better tracking and monitoring of access through GitHub’s app installation logs
  • No User Dependency: Does not depend on individual user accounts or managing SSH key files
  • Automatic Token Management: No need to manually rotate SSH keys or PATs when they expire
  • Centralized Access Control: Easier to manage access across multiple repositories from a single app installation

Setting up a GitHub App:

Tip

Understanding GitHub Apps: GitHub Apps work in two phases. First, you create and configure the app in your organization or personal account’s developer settings (this defines what the app can do). Second, you install that app to specific repositories or organizations (this grants the app access to those repositories). You’ll need information from both phases to configure Package Manager.

Phase 1: Create and Configure the GitHub App

  1. Navigate to your GitHub organization or personal account’s Settings → Developer settings → GitHub Apps.
  2. Click New GitHub App and create a new app following GitHub’s documentation.
  3. Configure the app with the following permissions:
    • Repository permissions: Contents (Read) - Required to access repository content
    • Repository permissions: Metadata (Read) - Required for basic repository information
  4. Generate and download a private key for the app from the app’s configuration page.
  5. Note the Client ID (found on the app’s configuration page - you’ll need this when importing the credential).

Phase 2: Install the GitHub App

  1. From the app’s configuration page, click Install App in the left sidebar.
  2. Select the organization or account where you want to install the app.
  3. Choose whether to install the app on all repositories or select specific repositories that Package Manager needs to access.
  4. Complete the installation process.
  5. Note the Installation ID from the installation URL (e.g., https://github.com/settings/installations/12345678 - the number at the end is your Installation ID).

Importing the GitHub App Credential into Package Manager:

Now that you have your App ID, Installation ID, and private key file, you can import the credential:

Terminal
# Import the GitHub App credential using the values from setup
rspm import github-app \
  --name=my-github-app \
  --client-id=YOUR_CLIENT_ID \
  --installation-id=YOUR_INSTALLATION_ID \
  --private-key="$(cat /path/to/your-private-key.pem)"

Replace YOUR_CLIENT_ID with the Client ID from step 5, YOUR_INSTALLATION_ID with the Installation ID from step 10, and /path/to/your-private-key.pem with the path to the private key file you downloaded in step 4.

Note

GitHub App vs Other Authentication Methods:

For GitHub repositories, you have several authentication options:

  • GitHub App Credentials: Designed specifically for GitHub repositories with GitHub’s native app authentication. Provides the best integration with GitHub’s permission model, automatic token management, and audit features. Recommended for most GitHub use cases.
  • SSH Keys: Traditional authentication method that works well for simple setups but requires manual key management and provides broad account-level access.
  • HTTPS Credentials (PATs): Simple to set up but use long-lived tokens that require manual rotation and may have broader permissions than needed.

For GitHub repositories, GitHub App credentials are generally the recommended approach over SSH keys or HTTPS credentials.

To use the newly imported GitHub App credential with a new Git builder, specify the credential name with the --credential flag in the rspm create git-builder command.

Git Credential Security

Package Manager encrypts and stores imported credentials in the metadata database. Any person (by default, members of the rstudio-pm unix group) with access to the admin CLI can:

Users cannot access the contents of the credential, nor is the credential available for arbitrary actions. We recommend granting SSH keys, HTTPS credentials, OAuth2 credentials, and GitHub App credentials imported to Package Manager limited read-only access to only the endpoints you wish to expose as packages.

OAuth2 Security Benefits:

  • OAuth2 federated credentials provide enhanced security through short-lived access tokens that are automatically refreshed
  • Client secrets are stored encrypted and never exposed to git operations
  • Federated identity authentication eliminates long-lived secrets entirely by using OIDC tokens

GitHub App Security Benefits:

  • GitHub App credentials use short-lived access tokens (1 hour expiration) that are automatically renewed, unlike long-lived SSH keys or PATs
  • Private keys are stored encrypted and used only for JWT generation, eliminating the need to manage SSH key files
  • Fine-grained permissions allow access to only the repositories and operations needed, unlike SSH keys which typically provide broad access
  • Enhanced audit trails through GitHub’s app installation logs provide better visibility than SSH key usage
  • No risk of accidentally exposing SSH private keys or PATs in configuration files or logs

Changing credentials for a Git builder

Credentials may be rotated by either creating a new credential and editing an existing git-builder:

Terminal
rspm import ssh-key --name=[key name] --path=[/path/to/key]
rspm edit git-builder --name=[git-builder name] --source=[git source] --new-credential=[key name]

Or you can update a credential by running the rspm edit ssh-key, rspm edit https-credential, rspm edit federated-credential, or rspm edit github-app commands. Note that this will change the credential for all the Git builders using the key name.

If you would like to change the URL type of a Git builder from https to git or vice-versa, ensure that you pass a credential that matches the new URL type.

Terminal
rspm edit git-builder --name=[git-builder name] --source=[git source] --new-url=git@github.com:somebody/something --new-credential=[name of ssh key]

Package Manager does not change credentials aside from manually specified commands. Therefore, when changing a URL without changing the credential, Package Manager will continue using any prior credential. To remove the association of the credential with the git-builder, use the rspm edit git-builder --remove-credential command:

Terminal
rspm edit git-builder --name=[git-builder name] --source=[git source] --remove-credential

Accessing Git Endpoints on a Local File System

If a Git repository is available locally, it is possible to point a Git builder to watch this location and update when changes are made on a branch or tag (based on the selection specified with the rspm create git-builder command).

To do this, the Git.AllowFileURLs configuration option must first be set to true. Once the option is set, the following command can be used to create a Git builder pointed to a local Git repository:

Terminal
rspm create git-builder --source git --url file:///path/to/local/git/repo
Note

Git must be installed on the server if you need support for file:// URLs.

Commits vs Tags

A package based on a Git endpoint can can be configured to watch one of two types of changes: “commits” or “tags.” In short, “commits” watches for changes to a specified Git branch, where “tags” watches for new tags in the whole Git repository.

Commit mode is recommended for bleeding edge repositories, whereas tag mode is suitable for exposing stable releases of packages.

A Git source can support different packages with different modes. However, a given package can only have one mode in a source. If you would like to surface the same package in both commit and tag mode, you must create two Git sources.

Using commits and tags has slightly different behavior for R and Python, outlined below.

Commits and Tags Behavior for R Packages

Commits

Package Manager will update the package any time new commits are discovered in a branch. In this mode, Package Manager automatically modifies the package’s version, assigning a unique version number to each build. The version number is created based on the commit time-stamp and is designed to avoid conflicts with the version scheme used by the package author. For example, if the Description file for a package indicates a version of 1.1-3, the automatic version number would be: 1.1-3.0.0.0.1537204599. If the author updates the package with a new commit, but keeps the version in the Description file the same, the new automatic version number would reflect the new commit time-stamp, e.g. 1.1-3.0.0.0.1537218677. This process ensures that users of the package always get the correct behavior from install.packages, with newer commits being associated with a semantically higher version number.

Tip

The above version behavior for “commits” triggers may be overridden by using the Git.ForceDescriptionVersion configuration option. This will force all packages built by commits in a branch to use the exact version in the DESCRIPTION file.

Tags

Package Manager will update the package any time a new Git tag is discovered. In this mode, Package Manager retains the version specified in the package’s Description file. This mode is designed to work when a Git tag is used to indicate a package release. Note: The name of the tag must match the version in the Description file. For example, if your package’s Description file has Version: 5.4.2, your tag must be either 5.4.2 or v5.4.2. If two tags reference the same version, preference is given to the newer tag. If a newer tag references an older version than a prior tag, the new tag is built as an archived package. If a tag is removed from a Git endpoint, any packages already built for that tag remain.

To enable tag matching, set the AllowTagVersionMismatch and TagMatching configuration settings in the [Git] section of the configuration file. AllowTagVersionMismatch tells Package Manager to build packages even when the Git tag does not match the Description file version. The TagMatching setting accepts a Golang formatted regex pattern to use for matching Git tags and will instruct Package Manager to only build packages for Git tags that match the pattern. For example, to match any Git tags, the following setting values will be used:

/etc/rstudio-pm/rstudio-pm.gcfg
[Git]
AllowTagVersionMismatch = true
TagMatching = .*

The default value for the TagMatching setting is the following regex pattern:

# Starts with an optional 'v' followed a number and up to 3 additional numeric segments separated by dots or hyphens.
^v?\d+([\.-]\d+){0,3}$

Some other common patterns for matching Git tags include the following:

/etc/rstudio-pm/rstudio-pm.gcfg
# Match a specific version (loosely)
TagMatching = "v1\\.3\\.1"

# Match a specific version (exactly)
TagMatching = "^v1\\.3\\.1$"

# Match any version beginning with 'v1.2.'
TagMatching = "^v1\\.2\\..+"
Tip

If you wish to build packages where the Description file version does not match the tag, use the Git.AllowTagVersionMismatch configuration option. When Git.AllowTagVersionMismatch is true, the version from the Description file will be used.

Commits and Tags Behavior for Python Packages

  • Commits: Package Manager will update the package any time new commits are discovered in a branch. In this mode, Package Manager will use the version that is built from python -m build and overwrite it any time a new commit is pushed. For example, if a package has a version 1.4.4 and a new commit is pushed, the current package will be replaced for version 1.4.4 and overwritten with the latest commit.

  • Tags: Package Manager will update the package any time a new Git tag is discovered. In this mode, Package Manager retains the version built from python -m build. Unlike Git builders for R packages, the tag is not required to match the version being built for Python packages. Because of this, the Git.AllowTagVersionMismatch does not do anything for Python Git builders. The highest version built will always be the latest available. If a new tag references an old version, it will be built and can be downloaded, but will not be served as the latest available version.

Git Cloning Depth

By default, Git builders will shallow clone a repository to make cloning more efficient. If a package requires cloning a deeper commit depth, use the --clone-depth=[depth] flag when creating or editing the Git builder. Set the flag value to the required cloning depth. The default cloning depth is 1, which means that only the latest commit will be cloned. If a full deep cloning is necessary, setting the --clone-depth flag to 0 will clone all the commits on the branch.

Tip

If a Python package is not building the latest version, try setting --clone-depth=0. Some Python packages require the whole repository to be cloned to determine the version to build, setting clone-depth to 0 will resolve this issue by cloning the entire depth of the repository.

Git Submodules

If your Git repository includes Git submodules that are required for building the R package, use the --recurse-submodules=[depth] flag when creating or editing the Git builder. Set the flag value to the required submodule recursion depth. The default recursion depth is 0, which means that no submodules will be included when cloning. If the Git builder includes credentials, the same credentials will be used for checking out the Git submodules.

Git Builder Naming and Multiple Builders for the same Git Repo

For git sources, when you create a Git builder without providing a name, the Git builder is named after the R package name by default. For git-python sources, Python packages require the --name flag as the Python package name cannot be determined prior to building.

If you need to add multiple Git builders for the same Git repo to the same Git source, use the --name flag to provide a custom name for each additional Git builder. For example, you could create Git builders to follow two branches:

Terminal
rspm create source --name=git --type=git
rspm create repo --name=git
rspm subscribe --repo=git --source=git

# Build "main" branch
rspm create git-builder --source=git --url https://github.com/rstudio/plumber --branch=main --wait

# Build "v044" branch
rspm create git-builder --source=git --url https://github.com/rstudio/plumber --branch=v044 --name=plumber-044 --wait
Note

When multiple Git builders are enabled for the same repository using the same git source for R packages, the latest version will always be marked as “latest”, and all others will be considered “archived,” regardless of when the Git builder was created or when the build was run.

Git Directories

By default, packages are built from the root directory of a Git repository. If the package instead exists in a subdirectory, specify the location with the --sub-dir flag when adding the git builder.

For example, to add arrow-adbc/r/adbcsnowflake as a package, the following syntax is used.

Terminal
rspm create git-builder --source=git --url=https://github.com/apache/arrow-adbc.git --sub-dir=r/adbcsnowflake/

The value https://github.com/apache/arrow-adbc.git is supplied to the --url flag because that is the URL used to clone the repository. The value /r/adbcsnowflake is supplied to the --sub-dir flag since the package is located in the r/adbcsnowflake subdirectory of the repository.

Managing Packages from Git

The git-builder (described above) watches the Git endpoint for changes, automatically handling package updates and archives. There might be cases where you wish to remove packages, or to stop package building altogether.

Packages can be removed at any time using the rspm remove command.

To stop automatic package building, but keep the existing packages, use the rspm delete git-builder command. To resume package building, simply create a new git-builder with the same metadata.

Terminal
# To remove previously-built packages from git:
rspm remove --source=[name of source] --name=[name of package and scope]

# To stop automatic package building, but keep the packages:
rspm delete git-builder --name=[name of package] --source=[name of source]

To view information about the current Git endpoints that are being tracked, use:

Terminal
rspm list git-builders

Editing Git builders

Git builders have a few fields that may be edited: credentials, URLs, sub directories, and branches (for “commits” triggers). The Git builders cannot be changed from SSH to HTTP URLs or vice versa.

Terminal
rspm edit git-builder --name=[git-builder name] --source=[git source name] --new-url=[HTTP/SSH URL]
rspm edit git-builder --name=[git-builder name] --source=[git source name] --new-credential=[credential name]
rspm edit git-builder --name=[git-builder name] --source=[git source name] --new-branch=[branch name]
rspm edit git-builder --name=[git-builder name] --source=[git source name] --new-sub-dir=[sub directory path]
rspm edit git-builder --name=[git-builder name] --source=[git source name] --remove-credential
rspm edit git-builder --name=[git-builder name] --source=[git source name] --remove-sub-dir

Combining Packages from Git with Other Package Sources

Local packages cannot be added manually to a git source, but a repository can surface packages from a git source alongside local packages and CRAN packages by subscribing to multiple sources. Take care when managing a repository’s subscriptions, as order is important. Refer to the Multiple Sources section for more details.

Polling Frequency

You can control how frequently Package Manager checks for updates using the Git.PollInterval configuration field. If multiple commits occur between checks, Package Manager will create a single version representing all of the changes. If multiple tags are created or removed between checks, Package Manager will build each tag individually, automatically archiving tags representing older versions of the package.

Repository Versioning is identical in all source types, including git sources.

Tracking Changes and Errors

If a repository subscribes to a Git source, you can view the Git source’s history in the Activity Log. For a git source, the Activity Log will identify each change to an R package including the new version, and a message will indicate the associated Git tag or commit as appropriate. For git-python sources, an entry will only show that a package was built, but not the specific package and version.

If an error is encountered attempting to clone, poll, or build a package, the Activity Log will record the attempt and include a message with the CLI command to be run to view a full error log.

You can also use the following Package Manager CLI commands to quickly check your active Git builders and view the logs:

Terminal
$ rspm list git-builders
Git Builders:
- [git package name]
    Source: [source name]
    URL: [source url]
    Trigger: [git package trigger]
    Key: none
Terminal
$ rspm list git-builds --source=[source name] --name=[git package name]
Git Builds:
- [git package name]
    Transaction ID: [transaction ID]
    SHA: [SHA]
    Tag: [tag]
    Status: [job status]
    Time: [time of run]
Only showing latest build, for more builds use the --count and --page flags
For more information run: rspm logs --transaction-id=[transaction ID]
Terminal
$ rspm logs --transaction-id=[transaction ID]
...
[git package run logs]
...

Package Manager automatically tries to build updates from a Git source three times. If the build fails more than three times, the update causing the failure is ignored. New updates are still discovered and built.

To retry a failed update, or to force a Git builder to rebuild the latest package version, use the rspm rerun command:

Terminal
rspm rerun git-builder \
    --name=[package name]  \
    --source=[source name] \
    --tag=[tag to rebuild, only required if the build trigger is tags]

You can also rerun commit-based Git builders to rebuild the package based on a specific Git SHA:

Terminal
rspm rerun git-builder \
    --name=[package name]  \
    --source=[source name] \
    --commit-sha=[Git SHA for commit to rebuild]

To aid in debugging, it can help to view output from the Git commands that are run as well as output from the SSH connection when applicable. To enable debugging, refer to the Logging.SystemLogLevel configuration property in the configuration appendix. To enable the debug log temporarily without restarting the server use the rspm config log command:

Terminal
rspm config log --level=DEBUG

Process Management

Refer to the Process Management section for information on how Package Manager securely runs R and Python processes when building packages for Git sources.

Back to top