Upgrade and/or Migrate Posit Team — Phase 1, Install upgraded products

flowchart LR
    A((0))
    B((1))
    C(("✔"))
    D((2))
    E(("#9829;"))
    A---B
    B---C
    C---D
    D---E
    A ~~~|"<a style="text-decoration:none" href="index.html">Overview</a>"|A
    B ~~~|"<b>Phase 1\n Install\n Upgraded\n Products</b>"|B
    C ~~~|"<a style="text-decoration:none" href="phase1-test.html">Test\n Phase 1</a>"|C
    D ~~~|"<a style="text-decoration:none" href="phase2-migrate.html">Phase 2\n Migrate\n Data</a>"|D
    E ~~~|"<a style="text-decoration:none" href="rollout.html">Rollout</a>"|E
    
    click A href "index.html"
    %% click B href "phase1-upgrade.html"
    click C href "phase1-test.html"
    click D href "phase2-migrate.html"
    click E href "rollout.html"

    classDef activePage fill:#447099, stroke:#D1DBE5, color:#fff, padding:4px
    classDef inactivePage fill:#D1DBE5, stroke:#447099, color:#151515, padding:4px
    classDef activeDesc font-size:20px
    class A,C,D,E inactivePage
    class B activePage

Installation prerequisites and prework

• Make sure you have sudo permissions on each server in staging and production
• Gather required information about your authentication system. Depending on your desired protocol this could include:
  • A server address, Base DN, and bind account credentials for LDAP.
  • A client ID, client secrets, and Issuer URL for OIDC.
  • A metadata URL and application identifier for SAML
• Provision new staging servers with desired Linux OS for each Posit Team product (Package Manager, Workbench, and Connect).
• Identify existing system libraries installed on the production servers and install on staging servers.
• Recommended: proactively install common system dependencies required by many Python and R packages on the Workbench and Connect staging servers.
• Mirror the architecture of the production environment in staging, e.g.:
  • Same number of server nodes.
  • If the production environment uses PostgreSQL for product metadata, have empty PostgreSQL database available.
  • If the production environment uses shared storage for user/content data, mount an empty directory for shared storage for each product on staging servers.
• If the production environment uses PostgreSQL for the product database, ensure the PostgreSQL database meets minimum requirements for the latest versions of Package Manager, Workbench, and Connect. If upgrade is required, upgrade PostgreSQL in production first, then stand up a matching version in staging

Server verification

Verify network connectivity

• To confirm network connectivity between all Posit Team servers, ping each staging server from the other staging servers.

• Ensure the default ports for each service are open:

  Service	Default Ports	Notes
  Workbench	8787 (HTTP), 443 (HTTPS), 5559
  Connect	3939 (HTTP), 443 (HTTPS)
  Package Manager	4242 (HTTP), 443 (HTTPS)
  LDAP	389
  LDAPS	636
  NFS	2049	If used for shared storage
  PostgreSQL	5432	If used for product database

• For online license activations, allow outbound access to https://wyday.com, port 443.

• For online deployments of Package Manager, allow outbound access to the Posit Package Service at https://rspm-sync.rstudio.com.

• If the production environment uses PostgreSQL for product database, verify connectivity to staging PostgreSQL database from each staging server.

• If the production environment uses shared storage for user/content data, verify connectivity to mounted empty directory for each product on each staging server.

Verify DNS configuration

Verify that nslookup returns the correct IP address for each server, replacing *.example.com with your product DNS addresses:

nslookup connect.example.com
nslookup workbench.example.com
nslookup packagemanager.example.com

Verify file system configuration

Confirm that any network attached storage has been mounted. The following command presents a formatted summary of df -h and related system information:

echo 'Hostname: ' && BLUE='\033[0;34m' && NC='\033[0m' &&  
echo -e ${BLUE}`hostname`${NC} && lscpu | grep -v NUMA |  
grep -E --color 'Architecture|CPU\\(s\\):|Virtualization type' && 
free -h | grep -E --color 'Mem|Swap|total' && df -h | 
grep -E --color 'G|Avail|Size|Filesystem|Use%|Used|Mounted on'

Firewall consideration

A firewall could be in place that prevents access to Posit product package downloads from the CDN. Follow your company’s procedure for downloading software.

Disable SELinux (RHEL only)

setenforce 0 && 
sudo sed -i 's/^SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config

Trust internal SSL CA certificates

• Add internal CA root certificate to the trust store on each staging server:

  • Identify the root certificate as the the last BEGIN CERTIFICATE / END CERTIFICATE block of the certificate file. Copy this into a file called /etc/<rstudio*>/root.crt, where <rstudio*> is rstudio, rstudio-connect, or rstudio-pm.
  • Verify that you have identified the correct root certificate by running:

  openssl x509 -in /etc/<rstudio*>/root.crt -text -noout

  Expected output should be the name and information about the root certificate authority.

• Trust the root certificate on each server:

  RHEL

  cp /etc/<rstudio*>/root.crt /etc/pki/ca-trust/source/anchors && update-ca-trust`

  Ubuntu

  cp /etc/<rstudio*>/root.crt /usr/local/share/ca-certificates && update-ca-certificates`

Install and configure upgraded Package Manager in staging

If you don’t have a professional version of Package Manager, we recommend configuring Workbench and/or Connect to use Posit Public Package Manager. In either case, Package Manager provides precompiled Linux binaries for packages, which accelerates package library rebuilds and avoids many package installation issues due to missing system dependencies.

1. Install the latest version of Posit Package Manager on the staging server.

2. Activate the license.

3. The Git sources functionality allows Package Manager to automatically expose Python and R packages tracked in Git. If you use, or plan to use this functionality:

   1. Install a version of R and Python according to the Package Manager R and Python installation requirements.
   2. Match the same version of Python and R used on production Package Manager, if installed.

4. Check for success: Following the Package Manager Verify Installation instructions, verify the staging Package Manager service successfully starts (sudo systemctl status rstudio-pm), is accessible from the web UI, and you can access the Admin CLI.

5. Configure Package Manager, mirroring the production environment (e.g., review and copy a configuration file from production environment). Don’t add repositories or sources at this time.

6. If the production architecture uses PostgreSQL for the product database, edit the /etc/rstudio-pm/rstudio-pm.gcfg configuration file to point to the empty staging PostgreSQL database.

7. If the production architecture uses external shared storage for content data, edit the /etc/rstudio-pm/rstudio-pm.gcfg configuration file to point to the empty shared storage location (Server.DataDir setting).

8. Check for success: Restart the staging Package Manager service (sudo systemctl restart rstudio-pm) and check status. Verify Package Manager successfully starts and is accessible from the web UI.

   1. If the service fails to restart, look into the server logs located at /var/log/rstudio/rstudio-pm/rstudio-pm.log to identify the errors. There can be a situation where the configuration settings from previous versions were deprecated. Review the Package Manager Release Notes notes for deprecated or added configuration options that are relevant.

9. Configure SSL Certificates:

   1. Place certificate and key in /etc/rstudio-pm to leave them in an accessible place for the rstudio-pm user
   2. Verify permissions on the key are 600
   3. Update rstudio-pm.gcfg with HTTPS.Certificate, HTTPS.Key, HTTPS.Listen, HTTPRedirect.Listen to configure certificates

10. Check for success: Before proceeding, restart the Package Manager service (sudo systemctl restart rstudio-pm) and check the status (sudo systemctl status rstudio-pm). If the service fails to restart, review the logs at /var/log/rstudio/rstudio-pm/rstudio-pm.log to identify and resolve the errors.

Install and configure upgraded Connect in staging

1. Identify Python, R, and Quarto versions on your current Connect production environment. Review product release notes to identify if any versions have been deprecated. Install the supported versions on the staging server and add any additional new versions desired using Posit supported binaries:

   1. Install Python.
   2. Install R.
   3. Install Quarto.

2. If using Java and rJava, refer to the Java and R guidelines and configure each version of R installed to point to the correct Java version using <PATH_TO_R_VERSION>/R CMD javareconf.

3. Install the Posit Professional Drivers on the staging server.

4. Install the latest version of Posit Connect on the staging server.

5. Activate the license.

6. Check for success: Create an Admin User and verify login using basic password authentication. Email and production configuration settings haven’t been applied at this time.

7. Configure Connect, mirroring production environment (e.g., review and copy configuration file from production environment).

8. If your production configuration file has values for Applications.RunAs and/or Applications.SharedRunAsUnixGroup, create these users and groups on the staging server.

9. If your production configuration file has PAM for authentication in order to support current user execution (likely for Kerberos), provision the users on the system, mirroring the production environment.

10. If the production architecture uses PostgreSQL for product database edit the /etc/rstudio-connect/rstudio-connect.gcfg configuration file to point to the empty PostgreSQL database.

11. If the production architecture uses external shared storage for content data, edit the /etc/rstudio-connect/rstudio-connect.gcfg configuration file to point to the blank shared storage location (Server.DataDir setting).

12. Authentication configuration is copied from the production rstudio-connect.gcfg file. Verify authentication configuration and update any server-specific settings. If PAM profiles are also in use, copy them from production server.

13. Check for success: Restart staging Connect service (sudo systemctl restart rstudio-connect) and check status (sudo systemctl status rstudio-connect). Verify Connect successfully starts and that login with configured authentication method is successful.

    1. If the service fails to restart, look in the service logs at /var/log/rstudio/rstudio-connect/rstudio-connect.log to identify the errors. There can be a situation where the configuration settings from previous versions have been deprecated. Review the Connect Release Notes notes for deprecated or added configuration options that may be relevant.

14. Configure Connect to integrate with staging Package Manager, or Public Package Manager:

    1. Configure pip.conf to use Package Manager URL:

       /etc/pip.conf

       [global]
       timeout = 60
       # use the Public Package Manager URL below or your own internal Package Manager URL
       index-url = https://packagemanager.posit.co/pypi/latest/simple 
       trusted-host = packagemanager.posit.co

    2. Configure Connect to use Package Manager for CRAN:

       /etc/rstudio-connect/rstudio-connect.gcfg

       [RPackageRepository "CRAN"]
       URL = <see https://docs.posit.co/connect/admin/integrations/package-manager/#overriding-cran-with-package-manager> 

    3. Configure Connect package URL path rewriting to force binaries. If available, this rewrites all Package Manager URLs to use binary packages. Additionally, this speeds up the post-migration runtime cache rebuilding, accelerates future deployments, and alleviates system dependency burdens when installing packages from source:

       /etc/rstudio-connect/rstudio-connect.gcfg

       [R]
       PositPackageManagerURLRewriting=force-binaries 

15. Verify Connect is configured to use Python.

16. Verify Connect is configured to use Quarto.

17. Check for success: Restart staging Connect service (sudo systemctl restart rstudio-connect) and check status (sudo systemctl status rstudio-connect). If the service fails to restart, look in the service logs at /var/log/rstudio/rstudio-connect/rstudio-connect.log to identify the errors.

18. Configure SSL Certificates:

    1. Place the certificate and key in /etc/rstudio-connect to leave them in an accessible place for the rstudio-connect user.

    2. Verify permissions on the key are 600.

    3. Update rstudio-connect.gcfg with HTTPS.Certificate, HTTPS.Key, HTTPS.Listen, HTTPRedirect.Listen to configure the certificates.

19. Check for success:
    Restart staging Connect service: sudo systemctl restart rstudio-connect and
    check status:sudo systemctl status rstudio-connect

    
    If the service fails to restart, look in the service logs at /var/log/rstudio/rstudio-connect/rstudio-connect.log to identify the errors. Verify Connect functionality and navigation in the web UI.

Install and configure upgraded Workbench in staging

Recommended installation path:

1. Use the Workbench Installer wbi, a CLI tool aimed at streamlining the installation and configuration of Posit Workbench. This tool can be used if you are installing on a single server, are using SQLite for the product database, and have internet access. For all other installations, please follow the manual installation steps mentioned later.

2. Skip to the Configure Workbench, mirroring production environment step below.

Otherwise, conduct a manual installation:

1. Identify Python, R, and Quarto versions on your current Workbench production environment. Check product release notes to identify if any versions have been deprecated. Install the supported versions on the staging server and add any additional new versions desired using Posit supported binaries:

   1. Install Python.

   2. Install R.

   3. Install Quarto.

2. Make Jupyter Kernel available.

3. Install the Posit Professional Drivers on the staging server.

4. Install the latest version of Posit Workbench on the staging server.

5. Activate the license.

6. Check for success: Have a Workbench Admin persona log into staging Workbench. This is a test account and can be created using sudo useradd -m <username> && sudo passwd <username>:

   1. This verifies that basic PAM authentication is working.

   2. Troubleshoot, resolve any login issues. A common solution is to run this command:

      cp /etc/pam.d/rstudio /etc/pam.d/rstudio.bak && cp /etc/pam.d/login /etc/pam.d/rstudio

7. Configure Workbench, mirroring production environment (e.g., review and copy configuration files from production environment).

8. If using Java and rJava, refer to the Java and R guidelines and:

   1. Configure each version of R installed to point to the correct Java version using <PATH_TO_R_VERSION>/R CMD javareconf.

   2. Modify /etc/rstudio/r-versions/ to ensure each version of R installed reads the ldpaths configuration file for that R version.

9. Verify Workbench integrates with new staging environments for Connect and Package Manager:

   1. Configure staging Connect as the default Connect server:

      /etc/rstudio/rsession.conf

      default-rsconnect-server=http://connectserver/ 

   2. Configure Package Manager as default repository for R:

      /etc/rstudio/repos.conf

      CRAN=https://<package-manager-server-address>/<cran-repo-name>/latest

   3. Configure pip.conf to use Package Manager URL:

      /etc/pip.conf

      [global]
      timeout = 60
      # use the Public Package Manager URL below or your own internal Package Manager URL
      index-url = https://packagemanager.posit.co/pypi/latest/simple 
      trusted-host = packagemanager.posit.co

10. Verify that Jupyter is enabled.

11. Check for success: restart the service (sudo rstudio-launcher restart && sudo rstudio-server restart) and verify login. Confirm RStudio Pro, JupyterLab, Jupyter Notebook, and VS Code sessions can be started.

    1. If the service fails to restart, look into the logs to identify the errors. Errors and warnings aggregated in /var/lib/rstudio-server/monitor/log/rstudio-server.log. Executables log under /var/log/rstudio/rstudio-server and Launcher logs to /var/log/rstudio/launcher. There can be a situation where configuration settings from previous versions have been deprecated. Review the Workbench Release Notes notes for deprecated or added configuration options that may be relevant.

12. Configure SSL:

    1. Place the certificate and key in /etc/rstudio.

    2.  Verify permissions on the key are 600.

    3. Update rserver.conf with ssl-certificate and ssl-certificate-key to configure certificates.

13. Check for success: restart the service (sudo rstudio-launcher restart && sudo rstudio-server restart) and verify login.

14. Configure authentication in staging Workbench to mirror production Workbench:

    • If you are using LDAP/AD or SSO authentication, review and mirror the production PAM profiles, sssd.conf, nsswitch.conf and Kerberos configurations, as applicable.

    • If you are using proxied authentication, verify the configuration of your proxy to permit:

      • Publishing from RStudio to staging Connect.
      • Using the Connect server API to interact with staging Connect.

15. Check for success: restart the service (sudo rstudio-launcher restart && sudo rstudio-server restart) and verify login.

    1. If the service fails to restart, look into the logs to identify the errors. Errors and warnings aggregated in /var/lib/rstudio-server/monitor/log/rstudio-server.log. Executables log under /var/log/rstudio/rstudio-server and Launcher logs to /var/log/rstudio/launcher.

Next Step:
Proceed to Phase 1 Testing