Shiny Server Pro to Posit Connect Migration Process

Step-by-step procedure to review and migrate from Shiny Server Pro to Posit Connect

This guide provides a framework for migrating applications from Shiny Server Pro to Posit Connect. Additionally, it provides a feature and configuration review to help you understand the differences between the two platforms in how applications are deployed and managed, and identifies additional functionality that your organization can utilize with Connect.

Shiny Server feature comparison

Feature Shiny Server Open Source Shiny Server Pro Posit Connect
Multiple independent applications ✅ ✅ ✅
User developed Shiny application hosting ✅ ✅ ✅
Auto restart Shiny applications ✅ ✅ ✅
Authenticate users to Shiny applications ❌ ✅ ✅
Scale applications to multiple processes ❌ ✅ ✅
Enable SSL encryption ❌ ✅ ✅
Shiny application monitoring ❌ ✅ ✅
Push-button publishing ❌ ❌ ✅
Built-in content versioning ❌ ❌ ✅
Expanded SSO options ❌ ❌ ✅
Git-backed content ❌ ❌ ✅
Scheduled jobs ❌ ❌ ✅
Off-Host execution ❌ ❌ ✅
Shiny for Python ✅ ❌ ✅
Other Python native and API content types ❌ ❌ ✅
Support for Quarto hosting and scheduling ❌ ❌ ✅

Why Posit Connect?

Posit Connect was developed as the evolution of Shiny Server Pro, because of various ingrained limitations within Shiny Server Pro’s architecture. These shortcomings have been remedied by writing Connect from the ground up. This re-architecture has enabled a number of new features to fix the issues that Shiny Server Pro caused in customer environments. Modern authentication protocols with support for multi-factor authentication like SAML and OIDC weren’t available in Shiny Server Pro, but in Connect they are standard features of all license tiers. On the topic of authentication, Shiny Server Pro really struggled with fine-grained authentication to specific applications, whereas Connect puts this control directly in the hands of the people most knowledgeable about their applications and their data, the data scientists.

With Shiny Server Pro you were extremely limited in the ability to scale individual applications and ability to natively integrate with industry standard infrastructure scaling and high-availability architectures, like Kubernetes. Connect allows your data-scientists to selectively tune their applications to enable the best per-user performance via the Content Runtime settings. This capability will allow your data science teams to provide the best possible user experience for their applications, depending on the needs of that specific application. Additionally, Shiny Server Pro couldn’t easily be configured to support high-availability or Off-Host execution of Shiny applications in Kubernetes. Connect, however, supports multiple high-availability options including Kubernetes. These configurations provide a level of redundancy and scalability that is not currently possible with Shiny Server Pro.

The process of publishing content to Shiny Server Pro can be complex and arduous and frequently requires intervention by server administrators to correctly set permission on publishing files and to make sure that multiple users Shiny applications couldn’t interfere with other Shiny applications running on the server. This administrative overhead is solved by Connect’s streamlined publishing process, including push-button publishing from the RStudio IDE, which will allow your data science teams to more easily publish content, without concerning themselves with fileshare permissions or server side access. Also, because Shiny Server Pro was limited to hosting Shiny application content, it’s utility for many organizations was limited. This is one of the key reasons behind the development of Connect to support a variety of different content types that Shiny Server Pro couldn’t support, such as API-based Content and Python Content Types.

Shiny Server Pro migration prework

Before you start the migration from Shiny Server Pro to Posit Connect, several factors need to be considered by the administration team, with input from the data scientists.

  1. Identify the scope of the migration. This includes:
    • Identifying which applications need to be migrated
    • Understanding your data scientists’ current application deployment workflow
    • Identifying a plan for the cut-over or parallel deployment of applications on an application-by-application basis. The question to answer is, can specific applications be moved to Connect while the old Shiny Server Pro server is still active, or is a hard cut-over from Shiny Server Pro to Connect required?
  2. Verify the installation details for your environment. This includes:
    • Reviewing your Shiny Server Pro configuration file, normally located at /etc/shiny-server/shiny-server.conf to determine what options were used in the configuration and their parallel settings and configurations in Connect. A more exhaustive mapping is included at Section 9 but we’ve highlighted some critical settings to review below:
Shiny Server Pro Settings Connect Settings Description
run_as Applications.RunAs The run_as configuration option defines the Linux user that runs the hosted application at the location specified in this block. This can be critically important to some applications that use this user’s Linux file system permissions to access data files on local or shared drives.
site_dir NA The site_dir directive is used to identify an entire directory of applications that Shiny Server Pro will host. There isn’t a direct parallel in Connect because each application published to Connect is stand-alone and hosted from its own directory.
app_dir NA The app_dir directory specifies a single directory that contains a single application to be hosted by Shiny Server Pro. Like site_dir, there isn’t a direct parallel in Connect, but this option is closest to how Connect individually separates published applications from one another.
user_dirs NA The user_dirs directive specifies an entire directory of user-owned subdirectories that contain applications and content to host. Because Content treats each application as a stand-alone piece of content, this option doesn’t have a direct parallel. However, this setting is important during the migration because user_dirs shows us where users may have stored their applications that need to be re-published to Connect.
Authentication Method Authentication Organizations often change their authentication method during the migration to Connect, since Connect supports most modern Single Sign On methods. But, it is important to know what authentication method Shiny Server Pro used, because that can influence the choice of authentication method used in Connect and whether data scientists need local Linux accounts in Connect for PAM authentication. Local Linux accounts are generally only needed if Application.RunAsCurrentUser is needed by your published applications.
r_path R Versions The r_path Shiny Server Pro configuration setting is used to identify the location of R installed on the server, this location can be used to determine the version of R used by Shiny Server Pro. This version of R should be installed on your Connect server to help maintain compatibility with the applications you migrate.
allow_app_override Content Settings Panel The allow_app_override setting is important because, if this is set to true, the Shiny Server Pro administrator should review the application location in question for files called .shiny_app.conf which can include some critical configuration settings that may need to be replicated to your Connect server. Some configuration directives to look for are r_path, required_user, and required_group.
server_name Server.Address Shiny Server Pro supports serving content at multiple hostnames, and the server_name directive can identify the hostnames that are used for routing to specific applications. Connect only supports hosting content at one URL, with customizable URIs that can replace the server_name directive.

Posit Connect installation

Once the Shiny Server Pro administrator has identified the list of applications and reviewed the Shiny Server Pro configuration file for the critical items listed above, it’s time to install and configure Posit Connect on a new application server. Posit recommends installing Connect on a new server for two key reasons. The first is that deploying Connect to a new server allows you to deploy and test migrating your applications in parallel without the possibility of impacting your production instance of Shiny Server Pro. The second reason is that most instances of Shiny Server Pro are deployed on Linux servers with operating systems that are nearing end of life, such as Ubuntu Bionic 18.04 or Red Hat/CentOS 7 and the migration to Connect provides the opportunity to upgrade to a modern, supported operating system.

Details on the installation and configuration of Connect can be found here:

Posit Connect Installation

Of critical importance with any Connect installation, but especially important in migrating from Shiny Server Pro to Connect, is having a conversation with your core Shiny Server Pro users about the features and functionality of Shiny Server Pro that are critical for their work. This conversation should include all of the R versions they would like to use, as well as a shared understanding of the future authentication model that your organization is planning to use with Connect. This decision may impact their application functionality if they rely on local Linux user features like Kerberos. This conversation should also discuss the use of any run_as users that you discovered during your configuration review, and why that run_as user has been used. Posit recommends reducing the number run_as users if possible, because it will result in maintaining fewer local Linux accounts on your new Connect server. Many use cases for run_as users can be better handled via Connect environment variables specific to each piece of content, such as securely storing credentials as Connect application-specific environment variables.

Once the new Connect server has been provisioned and configured to the satisfaction of both the administrators and the data scientists, a decision has to be made per application about which application migration method to use to publish your Shiny Server Pro applications to Connect.

Shiny application migration options

Publish from the RStudio IDE

The easiest publishing option, by far, is to publish your applications from the RStudio IDE. If the Shiny application will run in the RStudio IDE, then your data science team can use Push-Button Publishing to publish this Shiny application to an appropriately configured instance of Connect. Getting the application running on Connect will likely be an iterative process where parts of the application code and the server configuration may need to be adjusted to allow the application to work within the Connect operating model. The types of errors that occur are usually in the following categories:

These application dependencies are a necessary consideration regardless of the publishing method.

Publish directly from the Shiny Server Pro server

If your data science team cannot run the application inside the RStudio IDE, your Shiny Server Pro administrator can publish the application directly from the Shiny Server Pro server to Connect without an intermediate process. This process should likely only be used for archival applications that your data science team doesn’t anticipate updating in the future. If future updates/changes are expected, then your data science team should undertake the effort required to update the application to get the application running in the RStudio IDE and publish from there using Section 5.1.

Requirements

  1. Root access to the Shiny Server Pro Linux environment
  2. An account on Posit Connect with a minimum of Publisher permissions
  3. A list of the necessary data source dependencies
  4. The directory and all files associated with the Shiny application on the Shiny Server Pro server
  5. A list of required environment variables associated with the application on the Shiny Server Pro server
  6. The named Connect user who will own the application post-publishing

Once your Shiny Server Pro administrator has all the information listed above, they can follow the following steps to publish the application from Shiny Server Pro to Connect.

  1. Connect to the Shiny Server Pro server command line and su to the correct run_as user for the application you need to publish, generally this is the shiny user. Then, change your terminal location to the directory that contains the Shiny application that needs to be published.
# Become the application run_as user
su - shiny
# Change directory to the directory that contains the application source code
cd <path-to-your-application-directory>
  1. Create an .Renviron file with the Connect API key associated with the selected Publisher account and the Server URL for your Connect instance to set them as environment variables during the next step.
vi .Renviron
# Lines to add:
CONNECT_SERVER  = https://<connect-URL>
CONNECT_API_KEY = <my-secret-api-key>
  1. Start an R session in the current directory, and then read in the .Renviron file that you created in the previous step.
R
readRenviron(".Renviron")
  1. Once the R console loads, run the following code to install the rsconnect package, which is required for publishing content to Connect. Then, use the rsconnect package to generate a manifest for the target application.
install.packages("rsconnect")
library(rsconnect)
rsconnect::writeManifest("<path-to-application>")
  1. In the same R session, install the connectapi package, load it, and then create a client connection as well as the application bundle.
install.packages("connectapi")
library(connectapi)
# Creates a client connection to your Connect server, using the CONNECT_SERVER and CONNECT_API_KEY environment variables that we loaded from the .Renviron
client <- connect()
# Creates a Connect application bundle that will be deployed to Connect in the following step
bund <- bundle_dir("<path-to-application>")
  1. Deploy the application to Connect and poll the deployment status.
# Deploys the application bundle created in the previous step using the client connect that was created. <application-name> should be replaced with a uniquely identifying name for your application
dep <- deploy(client, bund, "<application-name>")
# Poll the deployment to get logs and deployment status. This information should be available in the Connect user interface
poll_task(dep)
  1. Optional: Transfer ownership of the application from the Publisher/Administrator who published the application to the permanent owner. The steps to transfer content between users are accomplished via calls to the Connect API.

Once the application has been successfully published to Connect and transferred to the permanent owner, it may not immediately start running. There are several reasons that a published application may fail to start, some of which are outlined below.

Shiny Application publishing considerations

After publishing a Shiny application via either the RStudio IDE or the CLI method, several considerations must be made. Most of these were mentioned earlier in Section 5.1, but a couple of specific considerations require some additional scrutiny.

Run As User

When you initially publish a Shiny application to Connect, Connect tries to build it and run it as the rstudio-connect user. If your application requires a specific run_as user, a Connect Administrator needs to correctly configure the RunAs user and then access the application dashboard for the published piece of content and change the RunAs user. This configuration change needs to be made once after the application has been published. Thus, the initial application run will fail, and the application will likely throw an error until this configuration change is made in Connect.

Absolute/Relative path configuration

Many Shiny applications require external filesystem data or an output directory on the Linux server hosting Shiny Server Pro. These Shiny applications are generally developed with the knowledge of the path to those directories, and these paths are encoded into the Shiny Server Pro instance of the Shiny application. This can cause complications within Connect because Shiny application files are not necessarily stored in the same location on disk between Connect and Shiny Server Pro, and the files may be reorganized slightly on disk during publishing. The result of this environment change is that paths like source("../../some-file.R") are unlikely to be included in the published content bundle and also won’t exist at this relative path on the Connect server. Additionally, if your application is attempting to write persistent files out to disk to some shared location, you need to validate that the relevant filesystem paths and permissions allow writing these files out on the Connect server. The solution to these issues varies based on your specific usage and context.

There are two solutions that commonly resolve this issue. The first is to work with your Connect Linux server administrator to determine the absolute path to your output directories and any files your application expects to be able to read and write. These absolute paths are generally of the form /some/path/to/a/directory/file. This type of path works regardless of the location of the executing Shiny application on the Connect server as long as Linux filesystem permissions allow the appropriate level of access (read, write, or execute). Then the Shiny application owner should update their application to use this absolute path, rather than the current relative path.

The second viable solution that is often used for source R files, and other smaller data files that don’t change frequently, is to include those files in the bundle by copying or moving these files into the Shiny application directory and then referencing them like this source("./some-file.R"). This type of path definition and the file’s location inside the Shiny application folder allows rsconnect and the RStudio IDE to detect them and include them in the application bundle pushed to Connect.

renv/packrat complexity

Shiny applications running under Shiny Server Pro often inherit package dependencies directly from the site library or system cache. This is generally a valid configuration that allows rsconnect and the RStudio IDE to correctly detect the appropriate dependencies and use those when constructing the application bundle manifest.json. Complications can occur when renv or packrat are used with the Shiny application, but the maintainers of the application weren’t careful about where they sourced packages, or in making sure that they had accurately enumerated all of the dependencies for their Shiny application. These complications can include references to packages and package versions that aren’t accessible to the Connect server, references to packages or package versions that are no longer available, and missing dependencies due to a poorly maintained renv/packrat environment. This is particularly impactful for publishing to Connect, because when an renv or packrat configuration is detected in the Shiny application folder it is used directly, without attempting to detect dependencies in the Shiny application files themselves. Essentially, rsconnect and the RStudio IDE assume that your renv or packrat environment is valid and sufficient to run your application, so it uses them blindly. Solutions to this type of issue vary but generally require that any missing packages are made available to Connect via a valid R repository configured inside the Connect configuration file.

Testing migrated applications

After you’ve migrated a new Shiny Server Pro application to Connect, functional and validation testing is required. The combination of changing platforms, servers, and runtime environments often results in application errors or necessary configuration changes to the underlying Connect Linux Server. Testing processes and rigor vary between organizations, but generally, this testing period requires the skills of both the application owner, the Connect administrator, and the Connect Linux server administrator. Some of the most common changes needed are listed below.

  • Package versions/availability
    • Packages installed from private git repositories or inaccessible sources like local files or internal repositories that aren’t configured on Connect need to be made available to your instance of Connect.
  • Data source connectivity, especially when the user_dirs directive was used in Shiny Server Pro
    • When user_dirs is defined, it’s common for individual users to have their own .odbc.ini and .odbcinst.ini stored in their home directory, which defines user-specific ODBC data sources that need to be replicated on the Connect server.
  • Supported R versions in Connect
    • Applications that use R versions before R 3.1 must be updated to at least R 3.1 to work on Connect.

Life after Shiny Server Pro

After and during the migration of your organization’s Shiny applications, your data science team will face many workflow changes and options. The first change they will encounter is that now, rather than copying files/directories to your Shiny Server Pro instance, they have to remotely publish to Connect from their RStudio IDE or the command line using the rsconnect utility from their desktops/remote servers. This first workflow change is usually welcome and results in a quicker, more reliable process once each application’s initial testing and issue resolution period is completed.

Next, a framework for managing Shiny application permissions needs to be developed. Decisions need to be made around which data science team members need the ability to republish/modify which individual applications and what audience should be able to see their published applications. The Content Settings section of the Connect User Guide details their options and limitations.

Data science teams must also shift their mindset from managing application directories to managing individual applications. This means that if a data science team is used to updating an entire directory of applications that are interrelated and tested together, they need to shift their perspective toward an understanding of the critical points where these applications interact and focus on those integration points when testing before publishing one of the included applications to avoid breaking the others. This independent application structure allows for a more micro-service style approach. It can allow for efficient architecture decisions that split the presentation applications from the back-end data APIs that populate it.

Last in this list, but not the last consideration or new functionality they need to consider is Git integration. Connect allows data science publishers to integrate directly with an internal or external Git repository to enable Git-backed publishing. This pull or push-based publishing model allows for rigorous code review or publishing constraints that can increase application traceability and reliability and reduce the risk of security incidents.

Appendix: setting mapping between Shiny Server Pro and Connect

Settings for content deployed to Connect are currently managed individually. Each application and document you publish must be configured separately. The following is a list of settings that you may be using in Shiny Server Pro, with their parallel settings in Connect.

Hosting and administration

  • Custom headers

    Shiny Server Pro: set_header directive in /etc/shiny-server/shiny-server.conf

    Connect: Server.CustomHeader setting in /etc/rstudio-connect/rstudio-connect.gcfg

  • Specify SockJS protocols

    Shiny Server Pro: disable_protocols directive in /etc/shiny-server/shiny-server.conf

    Connect: Applications.DisabledProtocols setting in /etc/rstudio-connect/rstudio-connect.gcfg

  • Customize the run user for each application

    Shiny Server Pro: run_as directive in /etc/shiny-server/shiny-server.conf

    Connect: Administrators can customize the RunAs user on a content-specific level in the Connect dashboard, on the Access tab of the application or document settings.

  • Customize Shiny scaling settings

    Shiny Server Pro: utilization_scheduler directive in /etc/shiny-server/shiny-server.conf

    Connect: The default settings are configured in the Scheduler section of /etc/rstudio-connect/rstudio-connect.gcfg, and custom scheduler settings for individual applications can be set in the Connect dashboard in the Performance tab of an application or document page.

  • Set the R version for a given application

    Shiny Server Pro:  r_path directive within a location in /etc/shiny-server/shiny-server.conf

    Connect: Connect uses Packrat to manage the packages associated with each application, and when you publish an application it will identify a compatible version of R automatically based on the author’s environment. That same R installation is used any time R is needed for that content; there is no need to set the version in a configuration file on the server.

  • Host a directory of applications

    Shiny Server Pro: site_dir directive within a location in /etc/shiny-server/shiny-server.conf

    Connect: Individuals can publish their content into Connect themselves, rendering this feature redundant.

  • Local application configurations

    Shiny Server Pro: allow_app_override directive in /etc/shiny-server/shiny-server.conf, in conjunction with a .shiny_app.conf file in each application

    Connect: Application authors can manage their content independently in the Connect dashboard

Metrics and session management

  • Historical system metrics

    Shiny Server Pro: admin directive in /etc/shiny-server/shiny-server.conf enables the admin dashboard

    Connect: Metrics section of /etc/rstudio-connect/rstudio-connect.gcfg enables the collection of historical metrics, which can then be viewed via Connect’s API and administrative dashboard.

  • Graphite export

    Shiny Server Pro: graphite_enabled directive in /etc/shiny-server/shiny-server.conf

    Connect: Metrics.GraphiteEnabled setting in /etc/rstudio-connect/rstudio-connect.gcfg

  • Health check

    Shiny Server Pro: /__health-check__ endpoint

    Connect: /__health-check__ endpoint

Authentication

Back to top