5 Launcher Plugin API

The Launcher Plugin API is a set of requests and responses which may be sent from the Launcher to the Plugin (requests) or from the Plugin to the Launcher (responses). This section describes the API calls supported by this version of the RStudio Launcher Plugin SDK. Documentation for the latest version the Launcher Plugin API may be found here.

For each API call, this section will go into detail on how the request corresponds to methods on the IJobSource interface, or on other components of the SDK.

5.1 Common Fields

The following fields are common to all requests and responses:

 

Request

Field Name Description
messageType The type ID of the message. The ID for a given message type can be found in the section for the given message.
requestId The monotonically increasing request ID for this request. This should be saved by the plugin for use in responses.

 

Response

Field Name Description
messageType The type ID of the message. The ID for a given message type can be found in the section for the given message.
requestId The ID of the request that this response is answering. This is used by the Launcher to determine which response belongs to which request.
responseId The monotonically increasing response ID for this response. The first response ID must be 0.

 

The SDK manages all of these fields for each request and response, so the Plugin developer does not need to worry about them.

In addition, many requests may also include username and requestUsername fields, described in the table below. These fields are passed to the appropriate IJobSource method as they impact the behaviour of the Plugin.

 

Field Name Description
username The username of the user that initiated the request. Used for auditing/security purposes. Some requests have the ability to specify ’*’ for this value, indicating all users’ information should be returned. This can occur if a super user (admin user) makes a request, or if authorization is disabled.
requestUsername The actual username used when the request was submitted. This should be ignored in most plugins, and the username value (above) should be used. This field is provided for additional information if needed.

5.2 Requests and Responses

5.2.1 Error

For any request, the Plugin may return an Error response. To return an Error response to the server, the IJobSource implementation should return an Error. The Plugin API component will generally construct the appropriate Error response based on the Error returned by the IJobSource call. For the most part, the Plugin developer does not need to be concerned with Error responses.

 

Error Response

Field Name Description Value
messageType See above. Response::Type::ERROR (-1)
requestId See above. Int
responseId See above. Int
errorCode The error code. More details in the Error Codes section. Int
errorMessage The error message. More details in the Error Codes section. Int

5.2.1.1 Error Codes

The code of the Error response must be one of the error codes supported by the Launcher Plugin API. The message can be an arbitrary string which provides more context for the Launcher user.

Code Name Description Value
UnknownError The request failed for an undetermined reason. Used when the Plugin cannot determine an appropriate error code for the error. 0
RequestNotSupported The request is not supported by the Plugin. The Plugin API component may return this if the Launcher sends a request that is not part of the Plugin’s supported Launcher Plugin API version. 1
InvalidRequest The request is malformed. The Plugin API component may return this if it receives an unexpected message from the Launcher. 2
JobNotFound The job does not exist in the scheduling system. The Plugin should return this if the user specified job ID does not exist. 3
PluignRestarted The request could not be completed because the Plugin had to restart. 4
Timeout The request timed out while waiting for a response from the job scheduling system. 5
JobNotRunning The job exists in the job scheduling system but is not in the running state. 6
JobOutputNotFound The job does not have output. 7
InvalidJobState The job has an invalid job state for the requested action. 8
JobControlFailure The job control action failed. 9
UnsupportedVersion The Launcher is using a Launcher Plugin API version that is not supported by the Plugin. 10

5.2.2 Heartbeat

When the Launcher starts the Plugin, it provides the configured heartbeat-interval-seconds value to the Plugin. If the Plugin does not send the Heartbeat Response every configured amount of time, the Launcher will restart the Plugin. This request is completely handled by the SDK and doesn’t require any action on the part of the Plugin developer. However, when debugging issues in the Plugin, it can become problematic if the Launcher restarts the Plugin while getting the debugger set up. To avoid this, heartbeat-interval-seconds can be set to 0 in /etc/rstudio/launcher.conf, which will disable heartbeating. This is not a recommended configuration for production because unresponsive Plugins will not be handled correctly.

Despite the fact that there will be many heartbeats sent to and from the Launcher, every Heartbeat Request and Response will have request ID and response IDs of 0. This is because it is Heartbeat Response do not strictly respond to a particular request – they must merely be sent to the Launcher every configured time interval.

 

Heartbeat Request

Field Name Description Value
messageType See above. Request::Type::HEARTBEAT (0)
requestId See above. 0

 

Heartbeat Response

Field Name Description Value
messageType See above. Response::Type::HEARTBEAT (0)
requestId See above. 0
responseId See above. 0

5.2.3 Bootstrap

This request is sent to the Plugin once, immediately after the Plugin is launched. During this request, the SDK will verify version compatibility with the Launcher and then trigger the population of existing jobs by invoking IJobSource::getJobs.

 

Bootstrap Request

Field Name Description Value
messageType See above. Request::Type::BOOTSTRAP (1)
requestId See above. 0
version The Launcher Plugin API version in use by the Launcher. Version

 

Bootstrap Response

Field Name Description Value
messageType See above. Response::Type::BOOTSTRAP (1)
requestId See above. 0
responseId See above. 0
version The Launcher Plugin API version in use by the Plugin. Version

5.2.4 Jobs

There are two API requests which return a Job State Response: the Submit Job and Job State requests. The SDK maintains an internal cache of all jobs in the system. When a Job State request is received, the SDK searches the cache for any jobs that match the request and sends them back to the Launcher. The SDK will also manage filtering the jobs and restricting the job fields in the response, if necessary. For more information about how the job cache is maintained, see the Customizing the Job Repository section of the Advanced Features chapter.

 

Submit Job Request

Field Name Description Value
messageType See above. Request::Type::SUBMIT_JOB (2)
requestId See above. Int
username See above. String
requestUsername See above. String
job The job object which describes the job to be launched. Job

 

Job State Request

Field Name Description Value
messageType See above. Request::Type::GET_JOB (3)
requestId See above. Int
username See above. String
requestUsername See above. String
jobId The ID of the job to be retrieved. If this field is ’*’ all jobs should be retrieved and filters may be used. String
encodedJobId The Launcher generated encoded job ID which contains extra metadata. Provides extra information only. String
tags If present, the set of tags by which to filter the jobs. Only jobs which have all these tags will be returned. String Array
startTime If present, only jobs which were submitted after this UTC time will be returned. Format: YYYY-MM-DDThh:mm:ss. String
endTime If present, only jobs which were submitted before this UTC time will be returned. Format: YYYY-MM-DDThh:mm:ss. String
statuses If present, only jobs which have one of the specified statuses will be returned. JobState Array
fields If present, only the job fields included in this list will be included in the response, excepted ‘id’, which will always be returned. String Array

 

Job State Response

Field Name Description Value
messageType See above. Response::Type::JOB_STATE (2)
jobs The list of jobs that met the request criteria. Job Array

5.2.5 Job Status Stream

This API call is responsible for streaming Job status updates as they occur. The SDK maintains an internal cache of all the Jobs in the Job Scheduling System and uses the JobStatusNotifier to allow components of the SDK to be notified when a Job’s status is updated. Components which wish to be notified of a Job status update may subscribe to the notifier, and components which wish to notify other components about new updates should post the updates to the JobStatusNotifier.

The SDK provides AbstractJobStatusWatcher and AbstractTimedJobStatusWatcher to help facilitate the implementation of keeping Job statuses updated within the Plugin. For more details about implementing Job status updates, see the Job Status Updates of the Advanced Features chapter, or TODOs #’s 9 - 11 of the RStudio Launcher Plugin SDK QuickStart Guide.

The SDK will manage all Job status streams, including closing them on request. The Plugin developer is only responsible for posting correct Job status updates to the JobStatusNotifier either directly or indirectly.

Rather than sending a response per open stream, the Plugin sends the Launcher a response with a field that indicates which stream requests are answered by this response. A sequence ID is kept per request to ensure the Launcher can keep the responses in the right order.

 

Job Status Stream Request

Field Name Description Value
messageType See above. Request::Type::GET_JOB_STATUS (4)
requestId See above. Int
username See above. String
requestUsername See above. String
jobId The ID of the job to be for which to stream statuses. If this field is ’*’ the statuses of all jobs will be streamed. String
encodedJobId The Launcher generated encoded job ID which contains extra metadata. Provides extra information only. String
cancel Whether the stream should be started (false) or canceled (true). Default: false. Boolean

 

Job Status Stream Response

Field Name Description Value
messageType See above. Response::Type::JOB_STATUS (3)
sequences An array of requests which are waiting for this stream response and the sequence ID of this response, relative to each request ID. StreamSequence Array
id The ID of the job. String
name The name of the job. String
status The status of the job. JobState
statusMessage The status message of the job, if any. String

5.2.6 Control Job

This API call can be used to control the state of a job. Currently supported operations are cancel, stop, kill, suspend, and resume. The Job must be in the correct state for a given operation to be valid (respectively, pending, running, running, running, and suspended). The SDK will handle Job State validation. The Plugin is only responsible for a best-effort attempt to perform the given operation on the job. If the Plugin indicates that the operation is not supported, the SDK will return an Error Response.

 

Control Job Request

Field Name Description Value
messageType See above. Request::Type::CONTROL_JOB (5)
requestId See above. Int
username See above. String
requestUsername See above. String
jobId The ID of the job to control. This field cannot be ’*’. String
encodedJobId The Launcher generated encoded job ID which contains extra metadata. Provides extra information only. String
operation The operation to be performed on the Job. JobOperation

 

Control Job Response

Field Name Description Value
messageType See above. Response::Type::CONTROL_JOB (4)
requestId See above. Int
responseId See above. Int
statusMessage A message describing the status of the control job operation, if any. String
operationComplete Whether the control job operation completed successfully or not. Boolean

5.2.7 Job Output Stream

This API call is responsible for streaming a Job’s output data. The caller may request the type of output they wish to see (Standard Out, Standard Error, or both) and the Plugin must do its best to report only that type of data.

The SDK will manage all the output streams. The Plugin developer needs to implement IJobSource::createOutputStream and may either use the provided FileOutputStream class or implement a custom concrete AbstractOutputStream. An example using the FileOutputStream can be found in ‘TODO #8’ of the ‘RStudio Launcher Plugin SDK QuickStart Guide’. For more details on implementing a custom AbstractOutputStream class, see the Custom Output Streams section.

 

Job Output Stream Request

Field Name Description Value
messageType See above. Request::Type::GET_JOB_OUTPUT (6)
requestId See above. Int
username See above. String
requestUsername See above. String
jobId The ID of the job to be for which to stream output. String
encodedJobId The Launcher generated encoded job ID which contains extra metadata. Provides extra information only. String
outputType The type of output to be streamed. Standard out (0), Standard error (1), or both (2). Int: 0, 1, or 2
cancel Whether the stream should be started (false) or canceled (true). Default: false. Boolean

 

Job Output Stream Response

Field Name Description Value
messageType See above. Response::Type::JOB_OUTPUT (5)
requestId See above. Int
responseId See above. Int
seqId The ID of this response in the stream of output responses. Int
outputType The type of output to in the response, if any. Standard out, Standard error, or mixed. String: “stdout”, “stderr”, or “mixed”
output The job output, if any. String
complete Whether the output stream is complete. Boolean

5.2.8 Job Resource Utilization Stream

This API call is responsible for streaming a Job’s resource utilization metrics. Resource utilization metrics may only be streamed when the Job is in the Running state. If the Job is not running, a JobNotRunning error will be returned by the SDK. Otherwise, the SDK will invoke AbstactResourceStream::initialize and the Plugin should commence streaming resource utilization metrics.

The SDK will manage all the resource utilization streams. The Plugin developer needs to implement AbstactResourceStream::initialize and may either extend AbstractResourceStream directly, or extend AbstractTimedResourceStream if resource utilization data must be polled. An example of extending AbstractResourceStream directly can be found in ‘TODO #16’ of the ‘RStudio Launcher Plugin SDK QuickStart Guide’. The LocalResourceStream provided with the sample Local Plugin provides an example of extending AbstractTimedResourceStream.

 

Job Resource Utilization Stream Request

Field Name Description Value
messageType See above. Request::Type::GET_JOB_RESOURCE_UTIL (7)
requestId See above. Int
username See above. String
requestUsername See above. String
jobId The ID of the job for which to stream resource utilization metrics. String
encodedJobId The Launcher generated encoded job ID which contains extra metadata. Provides extra information only. String
cancel Whether the stream should be started (false) or canceled (true). Default: false. Boolean

 

Job Resource Utilization Stream Response

Field Name Description Value
messageType See above. Response::Type::JOB_RESOURCE_UTIL (6)
requestId See above. Int
responseId See above. Int
seqId The ID of this response in the stream of resource utilization responses. Int
cpuPercent The percentage of CPU utilization of the Job, if available. Optional. String
cpuSeconds The total run time of the Job, in seconds, if available. Optional. String
virtualMemory The current size of virtual memory in use by the Job, in MB, if available. Optional. String
residentMemory The current size of the resident set in use by the Job, in MB, if available. Optional. String
complete Whether the resource utilization stream is complete. Boolean

5.2.9 Job Network

This API call is responsible for providing network information about the machine running the specified job to the caller. When this API call is received, the SDK will invoke IJobSource::getNetworkInfo. The Plugin is responsible for resolving the hostname and the IP addresses of the machine running the job.

For an example of how to implement IJobSource::getNetworkInfo, see ‘TODO #14’ of the ‘RStudio Launcher Plugin SDK QuickStart Guide’.

 

Job Network Request

Field Name Description Value
messageType See above. Request::Type::GET_JOB_NETWORK (8)
requestId See above. Int
username See above. String
requestUsername See above. String
jobId The ID of the job to be for which to retreive network information. String
encodedJobId The Launcher generated encoded job ID which contains extra metadata. Provides extra information only. String

 

Job Network Response

Field Name Description Value
messageType See above. Response::Type::GET_JOB_NETWORK (7)
requestId See above. Int
responseId See above. Int
host The hostname of the machine running the job. String
ipAddresses An array of the non-local IP addresses for the machine running the job. String Array

5.2.10 Cluster Info

This API call is responsible for reporting the configuration and capabilities of the job scheduling system. RStudio applications use this call to determine what UI to surface to the end user when they launch jobs. When this request is received, the SDK will invoke the following method to compose the correct response:

  • IJobSource::getConfiguration

The Plugin may return an error from this method if it encounters any issues populating the fields on the JobSourceConfiguration object. Additionally, the method is provided with a system::User object, which represents the user who initiated the Cluster Info request. This optionally allows the Plugin Developer to provide different results on a per-user or per-group basis. This may be useful to allow system administrator to set different maximum and default Resource Limits for different groups of user via the User Profiles feature. It also may be useful if the job scheduling system allows administrators to make similar rule sets.

For a simple example of implementing this method, see ‘TODO #7’ in the ‘RStudio Launcher Plugin SDK QuickStart Guide’.

 

Cluster Info Request

Field Name Description Value
messageType See above. Request::Type::GET_CLUSTER_INFO (9)
requestId See above. Int
username See above. String
requestUsername See above. String

 

Cluster Info Response

Field Name Description Value
messageType See above. Response::Type::CLUSTER_INFO (8)
requestId See above. Int
responseId See above. Int
supportsContainers Whether or not the job scheduling system supports containers. Boolean
config Any custom configuration values which may be set per Job. JobConfig Array
placementConstraints Any custom placement constraints which may be requested for a Job. PlacementConstraint Array
queues The queues which are available to run Jobs, if the job scheduling system supports queues. String Array
resourceLimits The types of resource limits which may be requested for a Job, including default and maximum values, if any. ResourceLimit Array
images If the job scheduling system supports containers, the list of container images which may be used when launching a job. String Array
defaultImage If the job scheduling system supports containers, the default container image to use if none is selected. String
allowUnknownImages If the job scheduling system supports containers, whether to allow Jobs to be run on images which are not known. Boolean

5.3 Complex API Types

This section describes the details of API types which are not simple types, such as String, Int, or Boolean.

5.3.1 Container

Field Name Description Value
image The name of the container image to use. String
runAsUserId The ID of the user to run the container as. Optional. Int
runAsGroupId The ID of the group to run the container as. Optional. Int
supplememtalGroupIds The list of additional group IDs to be added to the run-as user in the container. Optional. Int Array

5.3.2 Environment

Field Name Description Value
name The name of the environment variable. String
value The value of the environment variable. String

5.3.3 ExposedPort

Field Name Description Value
targetPort The target port, within the container. Int
publishedPort The published port, if different from the container port. Int
protocol The network protocol to use. Default: TCP. String

5.3.4 Job

Field Name Description Value
args The arguments of the ‘command’ or ‘exe’ of the Job. String Array
cluster The cluster of the Job. String
command The shell command of the Job. Mutually exclusive with ‘exe’. String
config The custom configuration values of the Job. JobConfig Array
container The container configuration of the Job, if the Cluster supports containers. Container Array
environment The environment variables for the Job. Environment Array
exe The executable of the Job. Mutually exclusive with ‘command’. String
exitCode The exit code of the ‘command’ or ‘exe’ of the Job. Int
exposedPorts The exposed ports of the Job, if containers are used. ExposedPort Array
host The host on which the Job was (or is being) run. String
id The unique ID of the Job. String
lastUpdateTime The time of the last update to the Job. String (ISO 8601 format DateTime)
mounts The file system mounts to apply when the Job is run. Mount Array
name The name of the Job. String
pid The process ID of the Job, if applicable. Int
id The unique ID of the Job. String
placementConstraints The list of placement constraints that were selected for the Job. PlacementConstraint Array
queues The list of queues that may be used to launch the Job, or the queue that was used to run the Job. String Array
stdin The standard input to be passed to the ‘command’ or ‘exe’ of the Job. String
stderr The location of the file which contains the standard error output of the Job. String
stdout The location of the file which contains the standard output of the Job. String
status The current status of the Job. JobState
statusMessage The message or reason of the current status of the Job. String
submissionTime The time at which the Job was submitted to the Cluster. String (ISO 8601 format DateTime)
tags The tags that were set for the Job. Used for filtering Jobs. String Array
user The username of the user who launched the Job. String
workingDirectory The directory to use as the working directory for the ‘command’ or ‘exe’. String

5.3.5 JobConfig

Field Name Description Value
name The name of the custom configuration value. String
valueType The type of the custom configuration value. Optional. “string”, “int”, “float”, or “enum”
value The value of the custom configuration value. Optional. String (convertible to valueType)

5.3.6 JobOperation

This is a JSON Integer value with a limited set of values. The possible values of a JobOperation field and their meanings are listed in the table below.

Value Description
0 The Job should be suspended. Only valid if the Job is currently in the “Running” JobState.
1 The Job should be resumed. Only valid if the Job is currently in the “Suspended” JobState.
2 The Job should be stopped. Only valid if the Job is currently in the “Running” JobState.
3 The Job should be killed. Only valid if the Job is currently in the “Running” JobState.
4 The Job should be canceled. Only valid if the Job is currently in the “Pending” JobState.

5.3.7 JobState

This type is a JSON String value with limited set of valid values. The possible values of a JobState field and their meanings are listed in the table below.

Value Description
Canceled The job was canceled by the user before it began to run.
Failed The job could not be launched due to an error. This status does not refer to jobs where the process exited with a non-zero exit code.
Finished The job was launched and finished executing. This includes jobs where the process exited with a non-zero exit code.
Killed The job was forcibly killed while it was running, i.e. the job process received SIGKILL.
Pending The job was successfully submitted to the job scheduling system but has not started running yet.
Running The job is currently running.
Suspended The job was running, but execution was paused and may be resumed at a later time.

5.3.8 Mount

Field Name Description Value
mountPath The destination path of the mount. String
readOnly Whether the source path should be mounted with write permissions (false) or not (true). Default: false. Boolean
type The type of mount. The default supported options are ‘azureFile’, ‘cephFs’, ‘glusterFs’, ‘host’, and ‘nfs’. The ‘passthrough’ value or a custom value may be used for other mount types. String
source The mount source description. Must match the specified mount type. MountSource

5.3.9 MountSource

This object will have a different schema depending on the type of mount specified in the ‘type’ field of the Mount object. Below, the source object for each of the default supported mount types will be described. If the mount type is ‘passthrough’ or a custom value, the object definition is determined by the Plugin.

5.3.9.1 AzureFileMountSource

This represents an Azure File source. If the ‘type’ field in the Mount object is ‘azureFile’ and the ‘source’ object follows this schema, the Plugin should attempt to mount the requested share from the Azure File account specified in the secret.

Field Name Description Value
secretName The name of the secret that contains both the Azure storage account name and the key. String
shareName The share name to be used. String

5.3.9.2 CephFsMountSource

This represents a Ceph File System source. If the ‘type’ field in the Mount object is ‘cephFs’ and the ‘source’ object follows this schema, the Plugin should attempt to mount the requested path as the specified user from the list of Ceph monitor addresses.

Field Name Description Value
monitors A comma-separated list of Ceph monitor addresses. For example: 192.168.1.200:8765,192.168.1.200:8766 String Array
path The path within the Ceph filesystem to mount. String
user The Ceph username to use. String
secretFile The file which contains the Ceph keyring for authentication. String
secretRef Reference to Ceph authentication secrets, which overrides SecretFile if specified. String

5.3.9.3 GlusterFsMountSource

This represents a Glusterfs mount source. If the ‘type’ field in the Mount object is ‘glusterFs’ and the ‘source’ object follows this schema, the Plugin should attempt to mount the requested path from the specified Glusterfs endpoints.

Field Name Description Value
endpoints The name of the endpoints object that represents a Gluster cluster configuration. String
path The name of the GlusterFs volume. String

5.3.9.4 HostMountSource

This represents a host mount source. If the ‘type’ field in the Mount object is ‘host’ and the ‘source’ object follows this schema, the Plugin should attempt to mount the requested path from the Launcher host, or reject the request on the basis that mounting host paths is not supported.

Field Name Description Value
path The source path to be mounted. String

5.3.9.5 NfsMountSource

This represents an NFS server mount source. If the ‘type’ field in the Mount object is ‘nfs’ and the ‘source’ object follows this schema, the Plugin should attempt to mount the requested path from the specified NFS server, or reject the request on the basis that mounting NFS paths is not supported.

Field Name Description Value
host The host of the NFS server. String
path The source path of the mount, on the NFS server. String

5.3.10 PlacementConstraint

Field Name Description Value
name The name of the placement constraint. String
value One of the possible values of the placement constraint. Optional. String

5.3.11 ResourceLimit

Field Name Description Value
limitType The type of the resource. “cpuCount”, “cpuTime”, “memory”, or “memorySwap”
defaultValue The default value of the resource type. String
maxValue The maximum value of the resource type. String
value The requested value of the resource. String

5.3.12 StreamSequence

Field Name Description Value
requestId The ID of the request for which this streamed response is being sent. Int
sequenceId The ordered ID of this stream response in the sequence of stream response for the given ‘requestId’. Int

5.3.13 Version

Field Name Description Value
major The major component of the supported version. Int
minor The minor component of the supported version. Int
patch The patch component of the supported version. Int