Configuration Encryption

Package Manager supports the encryption of sensitive configuration options. For example, the Postgres.Password, Proxy.Password, and Manifest.Password settings all support plain text or encrypted values.

Generate an Encryption Key

Although Package Manager automatically generates an encryption key if one does not exist, there are situations where you might need to create a key manually before starting the application. To generate an encryption key, run the following command in your terminal

Terminal
$ rspm encrypt generate
0616a5a7445f4c0b8b9f31a840f22a152f7621c5c9cc1febcb9f647183193c8e9f60...

This can be stored in the PACKAGEMANAGER_ENCRYPTION_KEY environment variable or written to the persistent storage location (either file or s3).

If [Storage].Persistent is set to s3, the key needs to be manually uploaded to the correct location in the S3 bucket. This process looks like the following:

Terminal
# generate the encryption key locally
rspm encrypt generate > rstudio-pm.key

# push it to S3 in the `/persistent/encryption` directory.
aws s3 cp rstudio-pm.key s3://<s3_bucket_name>/persistent/encryption/rstudio-pm.key

Now when Package Manager starts, it will read the encryption key from the S3 bucket.

Encrypt a setting

To encrypt a sensitive configuration setting, use the rspm encrypt command. For example:

Terminal
$ rspm encrypt
Encryption: Enter the plain text value below.
Qu0lI/gridhu85sqChwFtP2wFkqCcWt9owBpxFjAhKFaU2ZraBB2LM62Ieo=
Note

Only settings that have the type of encrypted-string support encryption.

Note

If [Storage].Persistent is set to s3, the key file must be present in the S3 bucket at the location /persistent/encryption/rstudio-pm.key and your AWS credentials must be configured correctly to access the key.

Key file

The rspm encrypt command creates a key file called rstudio-pm.key. This should be placed on [Storage].Persistent location at /persistent/encryption/rstudio-pm.key. This key must not be deleted for the Package Manager server to properly read the configuration file.

Note that the PACKAGEMANAGER_ENCRYPTION_KEY environment variable can be used to specify the encryption key to rspm encrypt in place of the key file, which may be preferable to managing the file directly in some cases.

Moving to a New Deployment

When you move Package Manager to a new deployment, or point an existing installation at new storage, the key file must come with it. It is named rstudio-pm.key and lives in the encryption directory within the [Storage].Persistent location, at /persistent/encryption/rstudio-pm.key.

If the key is not present in the new location, Package Manager does not stop and does not report an error. It generates a new key, saves it, and starts normally. The only signal is a single Info-level line in the log:

A new PPM encryption key was saved to /persistent/encryption/rstudio-pm.key at the [Storage].Persistent location.

That line reads like routine first-time setup, so it is easy to miss on a server that has been running for some time. The one exception is a key rotation that was started but never finished: Package Manager then stops during startup with an error about migrating encrypted database values, which is the better outcome, because an error can be recovered from while a silently generated key cannot.

When a new key is generated, everything encrypted with the previous key can no longer be decrypted, including both the configuration values and the database values listed under Overview. Those values cannot be recovered without the original key. If the key was lost during a move, restore it from a backup. Once the correct key is in place, the encrypted configuration values can be set again with rspm encrypt, but a stored Git credential cannot be reconstructed and must be entered again.

Note

If you supply the key through the PACKAGEMANAGER_ENCRYPTION_KEY environment variable, Package Manager does not read or create the key file, so there is no file to move. Carry the value of the environment variable to the new deployment instead.

Refer to the Changing Database Provider section for the other files that must move with a deployment.

Encryption Key Rotation

Warning

Be sure to backup your data before initiating a key rotation. If the key rotation fails in an incomplete state, you will need to restore the server from a backup to recover.

Overview

Package Manager uses an encryption key to encrypt sensitive data like config values and Git credentials. The encryption key is stored in the rstudio-pm.key file in the /persistent/encryption directory, on S3 or disk depending on your storage configuration. The encryption key is generated on the first start-up of Package Manager and is used to encrypt sensitive data. Package Manager encrypts the following data:

  • Encrypted configuration values:
    • Postgres.Password
    • Postgres.UsageDataPassword
    • Postgres.AzureClientSecret
    • Proxy.Password
    • Manifest.Password
    • OpenIDConnect.ClientSecret
  • Encrypted values stored in a file:
  • Encrypted database values:
    • git_credentials
    • key_rotations
Warning

The encryption key is also used to sign and verify API tokens. If the encryption key is rotated, all existing API tokens will be invalidated as they can only be verified with the old key. Admins will need to generate new API tokens for all users.

Key Rotation

To initiate an encryption key rotation, run the following command:

Terminal
rspm encrypt rotate
Warning

Using the PACKAGEMANAGER_ENCRYPTION_KEY environment variable is not supported for encryption key rotation. If you are using this environment variable, you must store the key at /persistent/encryption/rstudio-pm.key before initiating a rotation. This is so Package Manager owns the lifecycle of the old and new key during the rotation. After the rotation is complete, you may set the new key as the PACKAGEMANAGER_ENCRYPTION_KEY environment variable. If you try to rotate the key while using the PACKAGEMANAGER_ENCRYPTION_KEY environment variable, the command will return an error:

Terminal
encryption key rotation is prohibited for keys defined via the PACKAGEMANAGER_ENCRYPTION_KEY environment variable, only keys stored on persistent storage may be rotated

The rspm encrypt rotate command will generate a new encryption key and return which values need to be updated in the configuration file. The command output will look like this:

Terminal
Key rotation with ID '1' in progress. The following values are encrypted in the server's configuration and have been re-encrypted with the new encryption key:

Postgres.Password = jW1dMjLZuZMwQz7gBD01i7m8XZBlQUSt2qjRiCQm17kKogEJqNFFvzDc1Ho=
Postgres.UsageDataPassword = dznNHxRYtSlel1nUe2kCMWlVUBLDWQbF1gxiSWmmdUPTDOUQ5ahjgIwBUdQ=
Manifest.Password = /xssgdd3z5icZFXGWNlOnEfhAK445+donSMiHK56Pk8Y73q3xuMR+9SM7EU=
Proxy.Password = ntmE57lm4olrnWKmcU7G1u0s5TIERfQWyVUw6F4bdayqXRlq2PiOYewvPHI=

Please update these configuration values wherever they are stored and restart the server. Restarting the server will verify the new encryption key against the encrypted configuration values and rotate any encrypted database values.

The command output will list only the configuration values that it detects are currently encrypted with the old key. If they are stored in plain-text or are not set at all, it will not list them. If no values are currently encrypted in the configuration file, the command will return the following message:

Terminal
Key rotation with ID '1' in progress. No encrypted configuration values were found, nothing to manually rotate. Please restart the server. Restarting the server will rotate any encrypted database values.

After initiating a key rotation, you must update any encrypted configuration settings manually because Package Manager doesn’t control where these settings are defined. They can be stored:

  • In the .gcfg file,
  • As environment variables,
  • In a values.yaml file, or
  • In a combination of the above items.

After updating the configuration, restart the server to automatically verify the new encryption key against the encrypted configuration values and rotate any encrypted database values.

Values Stored in a File

A secret does not have to be written in the configuration itself. The OpenID Connect client secret can instead be held in a file, with OpenIDConnect.ClientSecretFile naming the path, and the contents of that file can be encrypted just as OpenIDConnect.ClientSecret can.

A key rotation re-encrypts those contents as well, and reports them in a separate section so that it is clear where the new value belongs:

Terminal
Key rotation with ID '1' in progress. No encrypted configuration values were found.

The following values are encrypted in files rather than in the server's configuration and have been re-encrypted with the new encryption key. Replace the entire contents of each file with the new value shown, rather than adding the value to the configuration:

OpenIDConnect.ClientSecretFile (/etc/rstudio-pm/oidc-client-secret) = ntmE57lm4olrnWKmcU7G1u0s5TIERfQWyVUw6F4bdayqXRlq2PiOYewvPHI=

Then restart the server. Restarting the server will verify the new encryption key against the encrypted configuration values and rotate any encrypted database values.

Replace the entire contents of the named file with the new value. Do not add the value to the configuration file. Package Manager reads the secret from the file, so it keeps using the old contents, which the new key cannot decrypt.

Package Manager does not write the file for you. The rotation runs inside the server process, under the rstudio-pm service account. Another account usually owns these files, or a secret manager supplies them, or configuration management would overwrite whatever the server wrote.

Note

If Package Manager cannot decrypt the file contents with the current key, it reports no value for that file. It also writes a warning to the log naming the file. Package Manager cannot tell why decryption failed, so the warning gives both possible reasons. Either the contents are in plain text, which needs no re-encryption and no action from you, or they are encrypted with a different key. In the second case, re-encrypt them with rspm encrypt and write the result into the file. That case is what an earlier incomplete rotation leaves behind.

An empty file is not reported. If Package Manager cannot read the file at all, the rotation still completes and reports every other value, and a warning naming the file is written to the log.

Only one key rotation can be in progress at a time. If you try to rotate the key while another rotation is in progress, the command will return the following error:

Terminal
an encryption key rotation is already in-progress

If Package Manager starts successfully after restarting the server, the key rotation is complete. If the server fails to start, you can restore the server from a backup to recover.

Back to top