Launcher Troubleshooting

Troubleshooting

The /status endpoint of the Job Launcher can be used to get the current health status and other connection information. Unlike other Job Launcher endpoints, this endpoint does not require authorization and may be queried by any monitoring or load balancing software to determine the health of a specific Job Launcher node. The status field indicates whether a node is experiencing no issues (“Green”), one or more plugins are restarting or unavailable (“Yellow”) or all plugins have failed and service shutdown is imminent (“Red”). It is recommended that you reroute traffic to another launcher node if you receive a “Yellow” or “Red” status, or if the page fails to load.

If you experience issues related to running Launcher sessions, ad hoc jobs, Jupyter sessions, or VS Code sessions, you can use the Launcher verification tool which will attempt to launch jobs and provide diagnostic output about what could be going wrong. To run the verification process, first stop the server, if it’s currently running:

sudo rstudio-server stop

Then, run the following command:

sudo rstudio-server verify-installation --verify-user=user

Replace the user in --verify-user=user value with a valid username of a user that is setup to run Posit Workbench in your installation. This will cause the test jobs to be started under their account, allowing the verification tool to check additional aspects of launching jobs, including mounting the user’s home directories into containers. You can also specify a specific test to run by using the --verify-test flag, like so:

sudo rstudio-server verify-installation --verify-user=user --verify-test=r-sessions

The above example will only test RStudio Pro Sessions, skipping ad hoc jobs and Jupyter/VS Code sessions. The parameter can be one of r-sessions, adhoc-jobs, jupyter-sessions, or vscode-sessions. If the parameter is unspecified, all tests will be run.

The verify-installation command will not start the server again, so once you’re done troubleshooting, you will need to start it again:

sudo rstudio-server start

Logging

By default, the Job Launcher and its plugins write logs to the system logger. If the service fails to start, check the system log to see if there are any errors, which should help you determine what is going wrong. In general, errors usually result from a misconfiguration of the Job Launcher or one of its plugins. When initially setting up the Launcher, it is sometimes helpful to run it directly from the command line, as opposed to running it via the service. See Running the Launcher for more information.

When running into issues that you are unable to resolve, make sure to enable debug logging for the Job Launcher by adding the line enable-debug-logging=1 to /etc/rstudio/launcher.conf. This will cause the Launcher and all of its plugins to emit debug output. This debug output can be seen on the console (if running the Job Launcher manually in the terminal), or in a debug log file located under the /var/lib/rstudio-launcher folder for the Job Launcher service, and under the plugin’s subdirectory for plugin-specific logging.